You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by wa...@apache.org on 2016/04/20 07:09:07 UTC
svn commit: r1740048 [6/10] - in
/incubator/singa/site/trunk/content/markdown: ./ develop/ docs/ docs/kr/
v0.3.0/ v0.3.0/jp/ v0.3.0/kr/ v0.3.0/zh/
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/layer.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/layer.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/layer.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/layer.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,614 @@
+# Layers
+
+---
+
+Layer is a core abstraction in SINGA. It performs a variety of feature
+transformations for extracting high-level features, e.g., loading raw features,
+parsing RGB values, doing convolution transformation, etc.
+
+The *Basic user guide* section introduces the configuration of a built-in
+layer. *Advanced user guide* explains how to extend the base Layer class to
+implement users' functions.
+
+## Basic user guide
+
+### Layer configuration
+
+Configuration of two example layers are shown below,
+
+ layer {
+ name: "data"
+ type: kCSVRecord
+ store_conf { }
+ }
+ layer{
+ name: "fc1"
+ type: kInnerProduct
+ srclayers: "data"
+ innerproduct_conf{ }
+ param{ }
+ }
+
+There are some common fields for all kinds of layers:
+
+ * `name`: a string used to differentiate two layers in a neural net.
+ * `type`: an integer used for identifying a specific Layer subclass. The types of built-in
+ layers are listed in LayerType (defined in job.proto).
+ For user-defined layer subclasses, `user_type` should be used instead of `type`.
+ * `srclayers`: names of the source layers.
+ In SINGA, all connections are [converted](neural-net.html) to directed connections.
+ * `param`: configuration for a [Param](param.html) instance.
+ There can be multiple Param objects in one layer.
+
+Different layers may have different configurations. These configurations
+are defined in `<type>_conf`. E.g., "fc1" layer has
+`innerproduct_conf`. The subsequent sections
+explain the functionality of each built-in layer and how to configure it.
+
+### Built-in Layer subclasses
+SINGA has provided many built-in layers, which can be used directly to create neural nets.
+These layers are categorized according to their functionalities,
+
+ * Input layers for loading records (e.g., images) from disk files, HDFS or network into memory.
+ * Neuron layers for feature transformation, e.g., [convolution](../api/classsinga_1_1ConvolutionLayer.html), [pooling](../api/classsinga_1_1PoolingLayer.html), dropout, etc.
+ * Loss layers for measuring the training objective loss, e.g., Cross Entropy loss or Euclidean loss.
+ * Output layers for outputting the prediction results (e.g., probabilities of each category) or features into persistent storage, e.g., disk or HDFS.
+ * Connection layers for connecting layers when the neural net is partitioned.
+
+#### Input layers
+
+Input layers load training/test data from disk or other places (e.g., HDFS or network)
+into memory.
+
+##### StoreInputLayer
+
+[StoreInputLayer](../api/classsinga_1_1StoreInputLayer.html) is a base layer for
+loading data from data store. The data store can be a KVFile or TextFile (LMDB,
+LevelDB, HDFS, etc., will be supported later). Its `ComputeFeature` function reads
+batchsize (string:key, string:value) tuples. Each tuple is parsed by a `Parse` function
+implemented by its subclasses.
+
+The configuration for this layer is in `store_conf`,
+
+ store_conf {
+ backend: # "kvfile" or "textfile"
+ path: # path to the data store
+ batchsize :
+ ...
+ }
+
+##### SingleLabelRecordLayer
+
+It is a subclass of StoreInputLayer. It assumes the (key, value) tuple loaded
+from a data store contains a feature vector (and a label) for one data instance.
+All feature vectors are of the same fixed length. The shape of one instance
+is configured through the `shape` field, e.g., the following configuration
+specifies the shape for the CIFAR10 images.
+
+ store_conf {
+ shape: 3 #channels
+ shape: 32 #height
+ shape: 32 #width
+ }
+
+It may do some preprocessing like [standardization](http://ufldl.stanford.edu/wiki/index.php/Data_Preprocessing).
+The data for preprocessing is loaded by and parsed in a virtual function, which is implemented by
+its subclasses.
+
+##### RecordInputLayer
+
+It is a subclass of SingleLabelRecordLayer. It parses the value field from one
+tuple into a RecordProto, which is generated by Google Protobuf according
+to common.proto. It can be used to store features for images (e.g., using the pixel field)
+or other objects (using the data field). The key field is not parsed.
+
+ type: kRecordInput
+ store_conf {
+ has_label: # default is true
+ ...
+ }
+
+##### CSVInputLayer
+
+It is a subclass of SingleLabelRecordLayer. The value field from one tuple is parsed
+as a CSV line (separated by comma). The first number would be parsed as a label if
+`has_label` is configured in `store_conf`. Otherwise, all numbers would be parsed
+into one row of the `data_` Blob.
+
+ type: kCSVInput
+ store_conf {
+ has_label: # default is true
+ ...
+ }
+
+##### ImagePreprocessLayer
+
+This layer does image preprocessing, e.g., cropping, mirroring and scaling, against
+the data Blob from its source layer. It deprecates the RGBImageLayer which
+works on the Record from ShardDataLayer. It still uses the same configuration as
+RGBImageLayer,
+
+ type: kImagePreprocess
+ rgbimage_conf {
+ scale: float
+ cropsize: int # cropping each image to keep the central part with this size
+ mirror: bool # mirror the image by set image[i,j]=image[i,len-j]
+ meanfile: "Image_Mean_File_Path"
+ }
+
+##### ShardDataLayer (Deprected)
+Deprected! Please use ProtoRecordInputLayer or CSVRecordInputLayer.
+
+[ShardDataLayer](../api/classsinga_1_1ShardDataLayer.html) is a subclass of DataLayer,
+which reads Records from disk file. The file should be created using
+[DataShard](../api/classsinga_1_1DataShard.html)
+class. With the data file prepared, users configure the layer as
+
+ type: kShardData
+ sharddata_conf {
+ path: "path to data shard folder"
+ batchsize: int
+ random_skip: int
+ }
+
+`batchsize` specifies the number of records to be trained for one mini-batch.
+The first `rand() % random_skip` `Record`s will be skipped at the first
+iteration. This is to enforce that different workers work on different Records.
+
+##### LMDBDataLayer (Deprected)
+Deprected! Please use ProtoRecordInputLayer or CSVRecordInputLayer.
+
+[LMDBDataLayer] is similar to ShardDataLayer, except that the Records are
+loaded from LMDB.
+
+ type: kLMDBData
+ lmdbdata_conf {
+ path: "path to LMDB folder"
+ batchsize: int
+ random_skip: int
+ }
+
+##### ParserLayer (Deprected)
+Deprected! Please use ProtoRecordInputLayer or CSVRecordInputLayer.
+
+It get a vector of Records from DataLayer and parse features into
+a Blob.
+
+ virtual void ParseRecords(Phase phase, const vector<Record>& records, Blob<float>* blob) = 0;
+
+
+##### LabelLayer (Deprected)
+Deprected! Please use ProtoRecordInputLayer or CSVRecordInputLayer.
+
+[LabelLayer](../api/classsinga_1_1LabelLayer.html) is a subclass of ParserLayer.
+It parses a single label from each Record. Consequently, it
+will put $b$ (mini-batch size) values into the Blob. It has no specific configuration fields.
+
+
+##### MnistImageLayer (Deprected)
+Deprected! Please use ProtoRecordInputLayer or CSVRecordInputLayer.
+[MnistImageLayer] is a subclass of ParserLayer. It parses the pixel values of
+each image from the MNIST dataset. The pixel
+values may be normalized as `x/norm_a - norm_b`. For example, if `norm_a` is
+set to 255 and `norm_b` is set to 0, then every pixel will be normalized into
+[0, 1].
+
+ type: kMnistImage
+ mnistimage_conf {
+ norm_a: float
+ norm_b: float
+ }
+
+##### RGBImageLayer (Deprected)
+Deprected! Please use the ImagePreprocessLayer.
+[RGBImageLayer](../api/classsinga_1_1RGBImageLayer.html) is a subclass of ParserLayer.
+It parses the RGB values of one image from each Record. It may also
+apply some transformations, e.g., cropping, mirroring operations. If the
+`meanfile` is specified, it should point to a path that contains one Record for
+the mean of each pixel over all training images.
+
+ type: kRGBImage
+ rgbimage_conf {
+ scale: float
+ cropsize: int # cropping each image to keep the central part with this size
+ mirror: bool # mirror the image by set image[i,j]=image[i,len-j]
+ meanfile: "Image_Mean_File_Path"
+ }
+
+##### PrefetchLayer
+
+[PrefetchLayer](../api/classsinga_1_1PrefetchLayer.html) embeds other input layers
+to do data prefeching. It will launch a thread to call the embedded layers to load and extract features.
+It ensures that the I/O task and computation task can work simultaneously.
+One example PrefetchLayer configuration is,
+
+ layer {
+ name: "prefetch"
+ type: kPrefetch
+ sublayers {
+ name: "data"
+ type: kShardData
+ sharddata_conf { }
+ }
+ sublayers {
+ name: "rgb"
+ type: kRGBImage
+ srclayers:"data"
+ rgbimage_conf { }
+ }
+ sublayers {
+ name: "label"
+ type: kLabel
+ srclayers: "data"
+ }
+ exclude:kTest
+ }
+
+The layers on top of the PrefetchLayer should use the name of the embedded
+layers as their source layers. For example, the "rgb" and "label" should be
+configured to the `srclayers` of other layers.
+
+
+#### Output Layers
+
+Output layers get data from their source layers and write them to persistent storage,
+e.g., disk files or HDFS (to be supported).
+
+##### RecordOutputLayer
+
+This layer gets data (and label if it is available) from its source layer and converts it into records of type
+RecordProto. Records are written as (key = instance No., value = serialized record) tuples into Store, e.g., KVFile. The configuration of this layer
+should include the specifics of the Store backend via `store_conf`.
+
+ layer {
+ name: "output"
+ type: kRecordOutput
+ srclayers:
+ store_conf {
+ backend: "kvfile"
+ path:
+ }
+ }
+
+##### CSVOutputLayer
+This layer gets data (and label if it available) from its source layer and converts it into
+a string per instance with fields separated by commas (i.e., CSV format). The shape information
+is not kept in the string. All strings are written into
+Store, e.g., text file. The configuration of this layer should include the specifics of the Store backend via `store_conf`.
+
+ layer {
+ name: "output"
+ type: kCSVOutput
+ srclayers:
+ store_conf {
+ backend: "textfile"
+ path:
+ }
+ }
+
+#### Neuron Layers
+
+Neuron layers conduct feature transformations.
+
+##### ConvolutionLayer
+
+[ConvolutionLayer](../api/classsinga_1_1ConvolutionLayer.html) conducts convolution transformation.
+
+ type: kConvolution
+ convolution_conf {
+ num_filters: int
+ kernel: int
+ stride: int
+ pad: int
+ }
+ param { } # weight/filter matrix
+ param { } # bias vector
+
+The int value `num_filters` stands for the count of the applied filters; the int
+value `kernel` stands for the convolution kernel size (equal width and height);
+the int value `stride` stands for the distance between the successive filters;
+the int value `pad` pads each with a given int number of pixels border of
+zeros.
+
+##### InnerProductLayer
+
+[InnerProductLayer](../api/classsinga_1_1InnerProductLayer.html) is fully connected with its (single) source layer.
+Typically, it has two parameter fields, one for weight matrix, and the other
+for bias vector. It rotates the feature of the source layer (by multiplying with weight matrix) and
+shifts it (by adding the bias vector).
+
+ type: kInnerProduct
+ innerproduct_conf {
+ num_output: int
+ }
+ param { } # weight matrix
+ param { } # bias vector
+
+
+##### PoolingLayer
+
+[PoolingLayer](../api/classsinga_1_1PoolingLayer.html) is used to do a normalization (or averaging or sampling) of the
+feature vectors from the source layer.
+
+ type: kPooling
+ pooling_conf {
+ pool: AVE|MAX // Choose whether use the Average Pooling or Max Pooling
+ kernel: int // size of the kernel filter
+ pad: int // the padding size
+ stride: int // the step length of the filter
+ }
+
+The pooling layer has two methods: Average Pooling and Max Pooling.
+Use the enum AVE and MAX to choose the method.
+
+ * Max Pooling selects the max value for each filtering area as a point of the
+ result feature blob.
+ * Average Pooling averages all values for each filtering area at a point of the
+ result feature blob.
+
+##### ReLULayer
+
+[ReLuLayer](../api/classsinga_1_1ReLULayer.html) has rectified linear neurons, which conducts the following
+transformation, `f(x) = Max(0, x)`. It has no specific configuration fields.
+
+##### STanhLayer
+
+[STanhLayer](../api/classsinga_1_1TanhLayer.html) uses the scaled tanh as activation function, i.e., `f(x)=1.7159047* tanh(0.6666667 * x)`.
+It has no specific configuration fields.
+
+##### SigmoidLayer
+
+[SigmoidLayer] uses the sigmoid (or logistic) as activation function, i.e.,
+`f(x)=sigmoid(x)`. It has no specific configuration fields.
+
+
+##### Dropout Layer
+[DropoutLayer](../api/asssinga_1_1DropoutLayer.html) is a layer that randomly dropouts some inputs.
+This scheme helps deep learning model away from over-fitting.
+
+ type: kDropout
+ dropout_conf {
+ dropout_ratio: float # dropout probability
+ }
+
+##### LRNLayer
+[LRNLayer](../api/classsinga_1_1LRNLayer.html), (Local Response Normalization), normalizes over the channels.
+
+ type: kLRN
+ lrn_conf {
+ local_size: int
+ alpha: float // scaling parameter
+ beta: float // exponential number
+ }
+
+`local_size` specifies the quantity of the adjoining channels which will be summed up.
+ For `WITHIN_CHANNEL`, it means the side length of the space region which will be summed up.
+
+
+#### Loss Layers
+
+Loss layers measures the objective training loss.
+
+##### SoftmaxLossLayer
+
+[SoftmaxLossLayer](../api/classsinga_1_1SoftmaxLossLayer.html) is a combination of the Softmax transformation and
+Cross-Entropy loss. It applies Softmax firstly to get a prediction probability
+for each output unit (neuron) and compute the cross-entropy against the ground truth.
+It is generally used as the final layer to generate labels for classification tasks.
+
+ type: kSoftmaxLoss
+ softmaxloss_conf {
+ topk: int
+ }
+
+The configuration field `topk` is for selecting the labels with `topk`
+probabilities as the prediction results. It is tedious for users to view the
+prediction probability of every label.
+
+#### ConnectionLayer
+
+Subclasses of ConnectionLayer are utility layers that connects other layers due
+to neural net partitioning or other cases.
+
+##### ConcateLayer
+
+[ConcateLayer](../api/classsinga_1_1ConcateLayer.html) connects more than one source layers to concatenate their feature
+blob along given dimension.
+
+ type: kConcate
+ concate_conf {
+ concate_dim: int // define the dimension
+ }
+
+##### SliceLayer
+
+[SliceLayer](../api/classsinga_1_1SliceLayer.html) connects to more than one destination layers to slice its feature
+blob along given dimension.
+
+ type: kSlice
+ slice_conf {
+ slice_dim: int
+ }
+
+##### SplitLayer
+
+[SplitLayer](../api/classsinga_1_1SplitLayer.html) connects to more than one destination layers to replicate its
+feature blob.
+
+ type: kSplit
+ split_conf {
+ num_splits: int
+ }
+
+##### BridgeSrcLayer & BridgeDstLayer
+
+[BridgeSrcLayer](../api/classsinga_1_1BridgeSrcLayer.html) &
+[BridgeDstLayer](../api/classsinga_1_1BridgeDstLayer.html) are utility layers assisting data (e.g., feature or
+gradient) transferring due to neural net partitioning. These two layers are
+added implicitly. Users typically do not need to configure them in their neural
+net configuration.
+
+### OutputLayer
+
+It write the prediction results or the extracted features into file, HTTP stream
+or other places. Currently SINGA has not implemented any specific output layer.
+
+## Advanced user guide
+
+The base Layer class is introduced in this section, followed by how to
+implement a new Layer subclass.
+
+### Base Layer class
+
+#### Members
+
+ LayerProto layer_conf_;
+ Blob<float> data_, grad_;
+ vector<AuxType> aux_data_;
+
+The base layer class keeps the user configuration in `layer_conf_`.
+Almost all layers has $b$ (mini-batch size) feature vectors, which are stored
+in the `data_` [Blob](../api/classsinga_1_1Blob.html) (A Blob is a chunk of memory space, proposed in
+[Caffe](http://caffe.berkeleyvision.org/)).
+There are layers without feature vectors; instead, they share the data from
+source layers.
+The `grad_` Blob is for storing the gradients of the
+objective loss w.r.t. the `data_` Blob. It is necessary in [BP algorithm](../api/classsinga_1_1BPWorker.html),
+hence we put it as a member of the base class. For [CD algorithm](../api/classsinga_1_1CDWorker.html), the `grad_`
+field is not used; instead, the layers for the RBM model may have a Blob for the positive
+phase feature and a Blob for the negative phase feature. For a recurrent layer
+in RNN, one row of the feature blob corresponds to the feature of one internal layer.
+The `aux_data_` stores the auxiliary data, e.g., image label (set `AuxType` to int).
+If images have variant number of labels, the AuxType can be defined to `vector<int>`.
+Currently, we hard code `AuxType` to int. It will be added as a template argument of Layer class later.
+
+If a layer has parameters, these parameters are declared using type
+[Param](param.html). Since some layers do not have
+parameters, we do not declare any `Param` in the base layer class.
+
+#### Functions
+
+ virtual void Setup(const LayerProto& conf, const vector<Layer*>& srclayers);
+ virtual void ComputeFeature(int flag, const vector<Layer*>& srclayers) = 0;
+ virtual void ComputeGradient(int flag, const vector<Layer*>& srclayers) = 0;
+
+The `Setup` function reads user configuration, i.e. `conf`, and information
+from source layers, e.g., mini-batch size, to set the
+shape of the `data_` (and `grad_`) field as well
+as some other layer specific fields.
+<!---
+If `npartitions` is larger than 1, then
+users need to reduce the sizes of `data_`, `grad_` Blobs or Param objects. For
+example, if the `partition_dim=0` and there is no source layer, e.g., this
+layer is a (bottom) data layer, then its `data_` and `grad_` Blob should have
+`b/npartitions` feature vectors; If the source layer is also partitioned on
+dimension 0, then this layer should have the same number of feature vectors as
+the source layer. More complex partition cases are discussed in
+[Neural net partitioning](neural-net.html#neural-net-partitioning). Typically, the
+Setup function just set the shapes of `data_` Blobs and Param objects.
+-->
+Memory will not be allocated until computation over the data structure happens.
+
+The `ComputeFeature` function evaluates the feature blob by transforming (e.g.
+convolution and pooling) features from the source layers. `ComputeGradient`
+computes the gradients of parameters associated with this layer. These two
+functions are invoked by the [TrainOneBatch](train-one-batch.html)
+function during training. Hence, they should be consistent with the
+`TrainOneBatch` function. Particularly, for feed-forward and RNN models, they are
+trained using [BP algorithm](train-one-batch.html#back-propagation),
+which requires each layer's `ComputeFeature`
+function to compute `data_` based on source layers, and requires each layer's
+`ComputeGradient` to compute gradients of parameters and source layers'
+`grad_`. For energy models, e.g., RBM, they are trained by
+[CD algorithm](train-one-batch.html#contrastive-divergence), which
+requires each layer's `ComputeFeature` function to compute the feature vectors
+for the positive phase or negative phase depending on the `phase` argument, and
+requires the `ComputeGradient` function to only compute parameter gradients.
+For some layers, e.g., loss layer or output layer, they can put the loss or
+prediction result into the `metric` argument, which will be averaged and
+displayed periodically.
+
+### Implementing a new Layer subclass
+
+Users can extend the Layer class or other subclasses to implement their own feature transformation
+logics as long as the two virtual functions are overridden to be consistent with
+the `TrainOneBatch` function. The `Setup` function may also be overridden to
+read specific layer configuration.
+
+The [RNNLM](rnn.html) provides a couple of user-defined layers. You can refer to them as examples.
+
+#### Layer specific protocol message
+
+To implement a new layer, the first step is to define the layer specific
+configuration. Suppose the new layer is `FooLayer`, the layer specific
+google protocol message `FooLayerProto` should be defined as
+
+ # in user.proto
+ package singa
+ import "job.proto"
+ message FooLayerProto {
+ optional int32 a = 1; // specific fields to the FooLayer
+ }
+
+In addition, users need to extend the original `LayerProto` (defined in job.proto of SINGA)
+to include the `foo_conf` as follows.
+
+ extend LayerProto {
+ optional FooLayerProto foo_conf = 101; // unique field id, reserved for extensions
+ }
+
+If there are multiple new layers, then each layer that has specific
+configurations would have a `<type>_conf` field and takes one unique extension number.
+SINGA has reserved enough extension numbers, e.g., starting from 101 to 1000.
+
+ # job.proto of SINGA
+ LayerProto {
+ ...
+ extensions 101 to 1000;
+ }
+
+With user.proto defined, users can use
+[protoc](https://developers.google.com/protocol-buffers/) to generate the `user.pb.cc`
+and `user.pb.h` files. In users' code, the extension fields can be accessed via,
+
+ auto conf = layer_proto_.GetExtension(foo_conf);
+ int a = conf.a();
+
+When defining configurations of the new layer (in job.conf), users should use
+`user_type` for its layer type instead of `type`. In addition, `foo_conf`
+should be enclosed in brackets.
+
+ layer {
+ name: "foo"
+ user_type: "kFooLayer" # Note user_type of user-defined layers is string
+ [foo_conf] { # Note there is a pair of [] for extension fields
+ a: 10
+ }
+ }
+
+#### New Layer subclass declaration
+
+The new layer subclass can be implemented like the built-in layer subclasses.
+
+ class FooLayer : public singa::Layer {
+ public:
+ void Setup(const LayerProto& conf, const vector<Layer*>& srclayers) override;
+ void ComputeFeature(int flag, const vector<Layer*>& srclayers) override;
+ void ComputeGradient(int flag, const vector<Layer*>& srclayers) override;
+
+ private:
+ // members
+ };
+
+Users must override the two virtual functions to be called by the
+`TrainOneBatch` for either BP or CD algorithm. Typically, the `Setup` function
+will also be overridden to initialize some members. The user configured fields
+can be accessed through `layer_conf_` as shown in the above paragraphs.
+
+#### New Layer subclass registration
+
+The newly defined layer should be registered in [main.cc](http://singa.incubator.apache.org/docs/programming-guide) by adding
+
+ driver.RegisterLayer<FooLayer, std::string>("kFooLayer"); // "kFooLayer" should be matched to layer configurations in job.conf.
+
+After that, the [NeuralNet](neural-net.html) can create instances of the new Layer subclass.
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mesos.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mesos.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mesos.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mesos.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,84 @@
+#Distributed Training on Mesos
+
+This guide explains how to start SINGA distributed training on a Mesos cluster. It assumes that both Mesos and HDFS are already running, and every node has SINGA installed.
+We assume the architecture depicted below, in which a cluster nodes are Docker container. Refer to [Docker guide](docker.html) for details of how to start individual nodes and set up network connection between them (make sure [weave](http://weave.works/guides/weave-docker-ubuntu-simple.html) is running at each node, and the cluster's headnode is running in container `node0`)
+
+
+
+---
+
+## Start HDFS and Mesos
+Go inside each container, using:
+````
+docker exec -it nodeX /bin/bash
+````
+and configure it as follows:
+
+* On container `node0`
+
+ hadoop namenode -format
+ hadoop-daemon.sh start namenode
+ /opt/mesos-0.22.0/build/bin/mesos-master.sh --work_dir=/opt --log_dir=/opt --quiet > /dev/null &
+ zk-service.sh start
+
+* On container `node1, node2, ...`
+
+ hadoop-daemon.sh start datanode
+ /opt/mesos-0.22.0/build/bin/mesos-slave.sh --master=node0:5050 --log_dir=/opt --quiet > /dev/null &
+
+To check if the setup has been successful, check that HDFS namenode has registered `N` datanodes, via:
+
+````
+hadoop dfsadmin -report
+````
+
+#### Mesos logs
+Mesos logs are stored at `/opt/lt-mesos-master.INFO` on `node0` and `/opt/lt-mesos-slave.INFO` at other nodes.
+
+---
+
+## Starting SINGA training on Mesos
+Assumed that Mesos and HDFS are already started, SINGA job can be launched at **any** container.
+
+#### Launching job
+
+1. Log in to any container, then
+ cd incubator-singa/tool/mesos
+<a name="job_start"></a>
+2. Check that configuration files are correct:
+ + `scheduler.conf` contains information about the master nodes
+ + `singa.conf` contains information about Zookeeper node0
+ + Job configuration file `job.conf` **contains full path to the examples directories (NO RELATIVE PATH!).**
+3. Start the job:
+ + If starting for the first time:
+
+ ./scheduler <job config file> -scheduler_conf <scheduler config file> -singa_conf <SINGA config file>
+ + If not the first time:
+
+ ./scheduler <job config file>
+
+**Notes.** Each running job is given a `frameworkID`. Look for the log message of the form:
+
+ Framework registered with XXX-XXX-XXX-XXX-XXX-XXX
+
+#### Monitoring and Debugging
+
+Each Mesos job is given a `frameworkID` and a *sandbox* directory is created for each job.
+The directory is in the specified `work_dir` (or `/tmp/mesos`) by default. For example, the error
+during SINGA execution can be found at:
+
+ /tmp/mesos/slaves/xxxxx-Sx/frameworks/xxxxx/executors/SINGA_x/runs/latest/stderr
+
+Other artifacts, like files downloaded from HDFS (`job.conf`) and `stdout` can be found in the same
+directory.
+
+#### Stopping
+
+There are two way to kill the running job:
+
+1. If the scheduler is running in the foreground, simply kill it (using `Ctrl-C`, for example).
+
+2. If the scheduler is running in the background, kill it using Mesos's REST API:
+
+ curl -d "frameworkId=XXX-XXX-XXX-XXX-XXX-XXX" -X POST http://<master>/master/shutdown
+
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mlp.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mlp.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mlp.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/mlp.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,195 @@
+# MLP Example
+
+---
+
+Multilayer perceptron (MLP) is a subclass of feed-forward neural networks.
+A MLP typically consists of multiple directly connected layers, with each layer fully
+connected to the next one. In this example, we will use SINGA to train a
+[simple MLP model proposed by Ciresan](http://arxiv.org/abs/1003.0358)
+for classifying handwritten digits from the [MNIST dataset](http://yann.lecun.com/exdb/mnist/).
+
+## Running instructions
+
+Please refer to the [installation](installation.html) page for
+instructions on building SINGA, and the [quick start](quick-start.html)
+for instructions on starting zookeeper.
+
+We have provided scripts for preparing the training and test dataset in *examples/cifar10/*.
+
+ # in examples/mnist
+ $ cp Makefile.example Makefile
+ $ make download
+ $ make create
+
+After the datasets are prepared, we start the training by
+
+ ./bin/singa-run.sh -conf examples/mnist/job.conf
+
+After it is started, you should see output like
+
+ Record job information to /tmp/singa-log/job-info/job-1-20150817-055231
+ Executing : ./singa -conf /xxx/incubator-singa/examples/mnist/job.conf -singa_conf /xxx/incubator-singa/conf/singa.conf -singa_job 1
+ E0817 07:15:09.211885 34073 cluster.cc:51] proc #0 -> 192.168.5.128:49152 (pid = 34073)
+ E0817 07:15:14.972231 34114 server.cc:36] Server (group = 0, id = 0) start
+ E0817 07:15:14.972520 34115 worker.cc:134] Worker (group = 0, id = 0) start
+ E0817 07:15:24.462602 34073 trainer.cc:373] Test step-0, loss : 2.341021, accuracy : 0.109100
+ E0817 07:15:47.341076 34073 trainer.cc:373] Train step-0, loss : 2.357269, accuracy : 0.099000
+ E0817 07:16:07.173364 34073 trainer.cc:373] Train step-10, loss : 2.222740, accuracy : 0.201800
+ E0817 07:16:26.714855 34073 trainer.cc:373] Train step-20, loss : 2.091030, accuracy : 0.327200
+ E0817 07:16:46.590946 34073 trainer.cc:373] Train step-30, loss : 1.969412, accuracy : 0.442100
+ E0817 07:17:06.207080 34073 trainer.cc:373] Train step-40, loss : 1.865466, accuracy : 0.514800
+ E0817 07:17:25.890033 34073 trainer.cc:373] Train step-50, loss : 1.773849, accuracy : 0.569100
+ E0817 07:17:51.208935 34073 trainer.cc:373] Test step-60, loss : 1.613709, accuracy : 0.662100
+ E0817 07:17:53.176766 34073 trainer.cc:373] Train step-60, loss : 1.659150, accuracy : 0.652600
+ E0817 07:18:12.783370 34073 trainer.cc:373] Train step-70, loss : 1.574024, accuracy : 0.666000
+ E0817 07:18:32.904942 34073 trainer.cc:373] Train step-80, loss : 1.529380, accuracy : 0.670500
+ E0817 07:18:52.608111 34073 trainer.cc:373] Train step-90, loss : 1.443911, accuracy : 0.703500
+ E0817 07:19:12.168465 34073 trainer.cc:373] Train step-100, loss : 1.387759, accuracy : 0.721000
+ E0817 07:19:31.855865 34073 trainer.cc:373] Train step-110, loss : 1.335246, accuracy : 0.736500
+ E0817 07:19:57.327133 34073 trainer.cc:373] Test step-120, loss : 1.216652, accuracy : 0.769900
+
+After the training of some steps (depends on the setting) or the job is
+finished, SINGA will [checkpoint](checkpoint.html) the model parameters.
+
+## Details
+
+To train a model in SINGA, you need to prepare the datasets,
+and a job configuration which specifies the neural net structure, training
+algorithm (BP or CD), SGD update algorithm (e.g. Adagrad),
+number of training/test steps, etc.
+
+### Data preparation
+
+Before using SINGA, you need to write a program to pre-process the dataset you
+use to a format that SINGA can read. Please refer to the
+[Data Preparation](data.html) to get details about preparing
+this MNIST dataset.
+
+
+### Neural net
+
+<div style = "text-align: center">
+<img src = "../images/example-mlp.png" style = "width: 230px">
+<br/><strong>Figure 1 - Net structure of the MLP example. </strong></img>
+</div>
+
+
+Figure 1 shows the structure of the simple MLP model, which is constructed following
+[Ciresan's paper](http://arxiv.org/abs/1003.0358). The dashed circle contains
+two layers which represent one feature transformation stage. There are 6 such
+stages in total. They sizes of the [InnerProductLayer](layer.html#innerproductlayer)s in these circles decrease from
+2500->2000->1500->1000->500->10.
+
+Next we follow the guide in [neural net page](neural-net.html)
+and [layer page](layer.html) to write the neural net configuration.
+
+* We configure an input layer to read the training/testing records from a disk file.
+
+ layer {
+ name: "data"
+ type: kRecordInput
+ store_conf {
+ backend: "kvfile"
+ path: "examples/mnist/train_data.bin"
+ random_skip: 5000
+ batchsize: 64
+ shape: 784
+ std_value: 127.5
+ mean_value: 127.5
+ }
+ exclude: kTest
+ }
+
+ layer {
+ name: "data"
+ type: kRecordInput
+ store_conf {
+ backend: "kvfile"
+ path: "examples/mnist/test_data.bin"
+ batchsize: 100
+ shape: 784
+ std_value: 127.5
+ mean_value: 127.5
+ }
+ exclude: kTrain
+ }
+
+
+* All [InnerProductLayer](layer.html#innerproductlayer)s are configured similarly as,
+
+ layer{
+ name: "fc1"
+ type: kInnerProduct
+ srclayers:"data"
+ innerproduct_conf{
+ num_output: 2500
+ }
+ param{
+ name: "w1"
+ ...
+ }
+ param{
+ name: "b1"
+ ..
+ }
+ }
+
+ with the `num_output` decreasing from 2500 to 10.
+
+* A [STanhLayer](layer.html#stanhlayer) is connected to every InnerProductLayer
+except the last one. It transforms the feature via scaled tanh function.
+
+ layer{
+ name: "tanh1"
+ type: kSTanh
+ srclayers:"fc1"
+ }
+
+* The final [Softmax loss layer](layer.html#softmaxloss) connects
+to LabelLayer and the last STanhLayer.
+
+ layer{
+ name: "loss"
+ type:kSoftmaxLoss
+ softmaxloss_conf{ topk:1 }
+ srclayers:"fc6"
+ srclayers:"data"
+ }
+
+### Updater
+
+The [normal SGD updater](updater.html#updater) is selected.
+The learning rate shrinks by 0.997 every 60 steps (i.e., one epoch).
+
+ updater{
+ type: kSGD
+ learning_rate{
+ base_lr: 0.001
+ type : kStep
+ step_conf{
+ change_freq: 60
+ gamma: 0.997
+ }
+ }
+ }
+
+### TrainOneBatch algorithm
+
+The MLP model is a feed-forward model, hence
+[Back-propagation algorithm](train-one-batch#back-propagation)
+is selected.
+
+ train_one_batch {
+ alg: kBP
+ }
+
+### Cluster setting
+
+The following configuration set a single worker and server for training.
+[Training frameworks](frameworks.html) page introduces configurations of a couple of distributed
+training frameworks.
+
+ cluster {
+ nworker_groups: 1
+ nserver_groups: 1
+ }
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/model-config.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/model-config.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/model-config.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/model-config.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,294 @@
+# Model Configuration
+
+---
+
+SINGA uses the stochastic gradient descent (SGD) algorithm to train parameters
+of deep learning models. For each SGD iteration, there is a
+[Worker](architecture.html) computing
+gradients of parameters from the NeuralNet and a [Updater]() updating parameter
+values based on gradients. Hence the model configuration mainly consists these
+three parts. We will introduce the NeuralNet, Worker and Updater in the
+following paragraphs and describe the configurations for them. All model
+configuration is specified in the model.conf file in the user provided
+workspace folder. E.g., the [cifar10 example folder](https://github.com/apache/incubator-singa/tree/master/examples/cifar10)
+has a model.conf file.
+
+
+## NeuralNet
+
+### Uniform model (neuralnet) representation
+
+<img src = "../images/model-categorization.png" style = "width: 400px"> Fig. 1:
+Deep learning model categorization</img>
+
+Many deep learning models have being proposed. Fig. 1 is a categorization of
+popular deep learning models based on the layer connections. The
+[NeuralNet](https://github.com/apache/incubator-singa/blob/master/include/neuralnet/neuralnet.h)
+abstraction of SINGA consists of multiple directly connected layers. This
+abstraction is able to represent models from all the three categorizations.
+
+ * For the feed-forward models, their connections are already directed.
+
+ * For the RNN models, we unroll them into directed connections, as shown in
+ Fig. 2.
+
+ * For the undirected connections in RBM, DBM, etc., we replace each undirected
+ connection with two directed connection, as shown in Fig. 3.
+
+<div style = "height: 200px">
+<div style = "float:left; text-align: center">
+<img src = "../images/unroll-rbm.png" style = "width: 280px"> <br/>Fig. 2: Unroll RBM </img>
+</div>
+<div style = "float:left; text-align: center; margin-left: 40px">
+<img src = "../images/unroll-rnn.png" style = "width: 550px"> <br/>Fig. 3: Unroll RNN </img>
+</div>
+</div>
+
+In specific, the NeuralNet class is defined in
+[neuralnet.h](https://github.com/apache/incubator-singa/blob/master/include/neuralnet/neuralnet.h) :
+
+ ...
+ vector<Layer*> layers_;
+ ...
+
+The Layer class is defined in
+[base_layer.h](https://github.com/apache/incubator-singa/blob/master/include/neuralnet/base_layer.h):
+
+ vector<Layer*> srclayers_, dstlayers_;
+ LayerProto layer_proto_; // layer configuration, including meta info, e.g., name
+ ...
+
+
+The connection with other layers are kept in the `srclayers_` and `dstlayers_`.
+Since there are many different feature transformations, there are many
+different Layer implementations correspondingly. For layers that have
+parameters in their feature transformation functions, they would have Param
+instances in the layer class, e.g.,
+
+ Param weight;
+
+
+### Configure the structure of a NeuralNet instance
+
+To train a deep learning model, the first step is to write the configurations
+for the model structure, i.e., the layers and connections for the NeuralNet.
+Like [Caffe](http://caffe.berkeleyvision.org/), we use the [Google Protocol
+Buffer](https://developers.google.com/protocol-buffers/) to define the
+configuration protocol. The
+[NetProto](https://github.com/apache/incubator-singa/blob/master/src/proto/model.proto)
+specifies the configuration fields for a NeuralNet instance,
+
+message NetProto {
+ repeated LayerProto layer = 1;
+ ...
+}
+
+The configuration is then
+
+ layer {
+ // layer configuration
+ }
+ layer {
+ // layer configuration
+ }
+ ...
+
+To configure the model structure, we just configure each layer involved in the model.
+
+ message LayerProto {
+ // the layer name used for identification
+ required string name = 1;
+ // source layer names
+ repeated string srclayers = 3;
+ // parameters, e.g., weight matrix or bias vector
+ repeated ParamProto param = 12;
+ // the layer type from the enum above
+ required LayerType type = 20;
+ // configuration for convolution layer
+ optional ConvolutionProto convolution_conf = 30;
+ // configuration for concatenation layer
+ optional ConcateProto concate_conf = 31;
+ // configuration for dropout layer
+ optional DropoutProto dropout_conf = 33;
+ ...
+ }
+
+A sample configuration for a feed-forward model is like
+
+ layer {
+ name : "input"
+ type : kRecordInput
+ }
+ layer {
+ name : "conv"
+ type : kInnerProduct
+ srclayers : "input"
+ param {
+ // configuration for parameter
+ }
+ innerproduct_conf {
+ // configuration for this specific layer
+ }
+ ...
+ }
+
+The layer type list is defined in
+[LayerType](https://github.com/apache/incubator-singa/blob/master/src/proto/model.proto).
+One type (kFoo) corresponds to one child class of Layer (FooLayer) and one
+configuration field (foo_conf). All built-in layers are introduced in the [layer page](layer.html).
+
+## Worker
+
+At the beginning, the Work will initialize the values of Param instances of
+each layer either randomly (according to user configured distribution) or
+loading from a [checkpoint file](). For each training iteration, the worker
+visits layers of the neural network to compute gradients of Param instances of
+each layer. Corresponding to the three categories of models, there are three
+different algorithm to compute the gradients of a neural network.
+
+ 1. Back-propagation (BP) for feed-forward models
+ 2. Back-propagation through time (BPTT) for recurrent neural networks
+ 3. Contrastive divergence (CD) for RBM, DBM, etc models.
+
+SINGA has provided these three algorithms as three Worker implementations.
+Users only need to configure in the model.conf file to specify which algorithm
+should be used. The configuration protocol is
+
+ message ModelProto {
+ ...
+ enum GradCalcAlg {
+ // BP algorithm for feed-forward models, e.g., CNN, MLP, RNN
+ kBP = 1;
+ // BPTT for recurrent neural networks
+ kBPTT = 2;
+ // CD algorithm for RBM, DBM etc., models
+ kCd = 3;
+ }
+ // gradient calculation algorithm
+ required GradCalcAlg alg = 8 [default = kBackPropagation];
+ ...
+ }
+
+These algorithms override the TrainOneBatch function of the Worker. E.g., the
+BPWorker implements it as
+
+ void BPWorker::TrainOneBatch(int step, Metric* perf) {
+ Forward(step, kTrain, train_net_, perf);
+ Backward(step, train_net_);
+ }
+
+The Forward function passes the raw input features of one mini-batch through
+all layers, and the Backward function visits the layers in reverse order to
+compute the gradients of the loss w.r.t each layer's feature and each layer's
+Param objects. Different algorithms would visit the layers in different orders.
+Some may traverses the neural network multiple times, e.g., the CDWorker's
+TrainOneBatch function is:
+
+ void CDWorker::TrainOneBatch(int step, Metric* perf) {
+ PostivePhase(step, kTrain, train_net_, perf);
+ NegativePhase(step, kTran, train_net_, perf);
+ GradientPhase(step, train_net_);
+ }
+
+Each `*Phase` function would visit all layers one or multiple times.
+All algorithms will finally call two functions of the Layer class:
+
+ /**
+ * Transform features from connected layers into features of this layer.
+ *
+ * @param phase kTrain, kTest, kPositive, etc.
+ */
+ virtual void ComputeFeature(Phase phase, Metric* perf) = 0;
+ /**
+ * Compute gradients for parameters (and connected layers).
+ *
+ * @param phase kTrain, kTest, kPositive, etc.
+ */
+ virtual void ComputeGradient(Phase phase) = 0;
+
+All [Layer implementations]() must implement the above two functions.
+
+
+## Updater
+
+Once the gradients of parameters are computed, the Updater will update
+parameter values. There are many SGD variants for updating parameters, like
+[AdaDelta](http://arxiv.org/pdf/1212.5701v1.pdf),
+[AdaGrad](http://www.magicbroom.info/Papers/DuchiHaSi10.pdf),
+[RMSProp](http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf),
+[Nesterov](http://scholar.google.com/citations?view_op=view_citation&hl=en&user=DJ8Ep8YAAAAJ&citation_for_view=DJ8Ep8YAAAAJ:hkOj_22Ku90C)
+and SGD with momentum. The core functions of the Updater is
+
+ /**
+ * Update parameter values based on gradients
+ * @param step training step
+ * @param param pointer to the Param object
+ * @param grad_scale scaling factor for the gradients
+ */
+ void Update(int step, Param* param, float grad_scale=1.0f);
+ /**
+ * @param step training step
+ * @return the learning rate for this step
+ */
+ float GetLearningRate(int step);
+
+SINGA provides several built-in updaters and learning rate change methods.
+Users can configure them according to the UpdaterProto
+
+ message UpdaterProto {
+ enum UpdaterType{
+ // noraml SGD with momentum and weight decay
+ kSGD = 1;
+ // adaptive subgradient, http://www.magicbroom.info/Papers/DuchiHaSi10.pdf
+ kAdaGrad = 2;
+ // http://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf
+ kRMSProp = 3;
+ // Nesterov first optimal gradient method
+ kNesterov = 4;
+ }
+ // updater type
+ required UpdaterType type = 1 [default=kSGD];
+ // configuration for RMSProp algorithm
+ optional RMSPropProto rmsprop_conf = 50;
+
+ enum ChangeMethod {
+ kFixed = 0;
+ kInverseT = 1;
+ kInverse = 2;
+ kExponential = 3;
+ kLinear = 4;
+ kStep = 5;
+ kFixedStep = 6;
+ }
+ // change method for learning rate
+ required ChangeMethod lr_change= 2 [default = kFixed];
+
+ optional FixedStepProto fixedstep_conf=40;
+ ...
+ optional float momentum = 31 [default = 0];
+ optional float weight_decay = 32 [default = 0];
+ // base learning rate
+ optional float base_lr = 34 [default = 0];
+ }
+
+
+## Other model configuration fields
+
+Some other important configuration fields for training a deep learning model is
+listed:
+
+ // model name, e.g., "cifar10-dcnn", "mnist-mlp"
+ string name;
+ // displaying training info for every this number of iterations, default is 0
+ int32 display_freq;
+ // total num of steps/iterations for training
+ int32 train_steps;
+ // do test for every this number of training iterations, default is 0
+ int32 test_freq;
+ // run test for this number of steps/iterations, default is 0.
+ // The test dataset has test_steps * batchsize instances.
+ int32 test_steps;
+ // do checkpoint for every this number of training steps, default is 0
+ int32 checkpoint_freq;
+
+The pages of [checkpoint and restore](checkpoint.html) has details on checkpoint related fields.
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neural-net.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neural-net.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neural-net.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neural-net.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,326 @@
+# Neural Net
+
+---
+
+`NeuralNet` in SINGA represents an instance of user's neural net model. As the
+neural net typically consists of a set of layers, `NeuralNet` comprises
+a set of unidirectionally connected [Layer](layer.html)s.
+This page describes how to convert an user's neural net into
+the configuration of `NeuralNet`.
+
+<img src="../../images/model-category.png" align="center" width="200px"/>
+<span><strong>Figure 1 - Categorization of popular deep learning models.</strong></span>
+
+## Net structure configuration
+
+Users configure the `NeuralNet` by listing all layers of the neural net and
+specifying each layer's source layer names. Popular deep learning models can be
+categorized as Figure 1. The subsequent sections give details for each
+category.
+
+### Feed-forward models
+
+<div align = "left">
+<img src="../../images/mlp-net.png" align="center" width="200px"/>
+<span><strong>Figure 2 - Net structure of a MLP model.</strong></span>
+</div>
+
+Feed-forward models, e.g., CNN and MLP, can easily get configured as their layer
+connections are undirected without circles. The
+configuration for the MLP model shown in Figure 1 is as follows,
+
+ net {
+ layer {
+ name : 'data"
+ type : kData
+ }
+ layer {
+ name : 'image"
+ type : kImage
+ srclayer: 'data'
+ }
+ layer {
+ name : 'label"
+ type : kLabel
+ srclayer: 'data'
+ }
+ layer {
+ name : 'hidden"
+ type : kHidden
+ srclayer: 'image'
+ }
+ layer {
+ name : 'softmax"
+ type : kSoftmaxLoss
+ srclayer: 'hidden'
+ srclayer: 'label'
+ }
+ }
+
+### Energy models
+
+<img src="../../images/rbm-rnn.png" align="center" width="500px"/>
+<span><strong>Figure 3 - Convert connections in RBM and RNN.</strong></span>
+
+
+For energy models including RBM, DBM,
+etc., their connections are undirected (i.e., Category B). To represent these models using
+`NeuralNet`, users can simply replace each connection with two directed
+connections, as shown in Figure 3a. In other words, for each pair of connected layers, their source
+layer field should include each other's name.
+The full [RBM example](rbm.html) has
+detailed neural net configuration for a RBM model, which looks like
+
+ net {
+ layer {
+ name : "vis"
+ type : kVisLayer
+ param {
+ name : "w1"
+ }
+ srclayer: "hid"
+ }
+ layer {
+ name : "hid"
+ type : kHidLayer
+ param {
+ name : "w2"
+ share_from: "w1"
+ }
+ srclayer: "vis"
+ }
+ }
+
+### RNN models
+
+For recurrent neural networks (RNN), users can remove the recurrent connections
+by unrolling the recurrent layer. For example, in Figure 3b, the original
+layer is unrolled into a new layer with 4 internal layers. In this way, the
+model is like a normal feed-forward model, thus can be configured similarly.
+The [RNN example](rnn.html) has a full neural net
+configuration for a RNN model.
+
+
+## Configuration for multiple nets
+
+Typically, a training job includes three neural nets for
+training, validation and test phase respectively. The three neural nets share most
+layers except the data layer, loss layer or output layer, etc.. To avoid
+redundant configurations for the shared layers, users can uses the `exclude`
+filed to filter a layer in the neural net, e.g., the following layer will be
+filtered when creating the testing `NeuralNet`.
+
+
+ layer {
+ ...
+ exclude : kTest # filter this layer for creating test net
+ }
+
+
+
+## Neural net partitioning
+
+A neural net can be partitioned in different ways to distribute the training
+over multiple workers.
+
+### Batch and feature dimension
+
+<img src="../images/partition_fc.png" align="center" width="400px"/>
+<span><strong>Figure 4 - Partitioning of a fully connected layer.</strong></span>
+
+
+Every layer's feature blob is considered a matrix whose rows are feature
+vectors. Thus, one layer can be split on two dimensions. Partitioning on
+dimension 0 (also called batch dimension) slices the feature matrix by rows.
+For instance, if the mini-batch size is 256 and the layer is partitioned into 2
+sub-layers, each sub-layer would have 128 feature vectors in its feature blob.
+Partitioning on this dimension has no effect on the parameters, as every
+[Param](param.html) object is replicated in the sub-layers. Partitioning on dimension
+1 (also called feature dimension) slices the feature matrix by columns. For
+example, suppose the original feature vector has 50 units, after partitioning
+into 2 sub-layers, each sub-layer would have 25 units. This partitioning may
+result in [Param](param.html) object being split, as shown in
+Figure 4. Both the bias vector and weight matrix are
+partitioned into two sub-layers.
+
+
+### Partitioning configuration
+
+There are 4 partitioning schemes, whose configurations are give below,
+
+ 1. Partitioning each singe layer into sub-layers on batch dimension (see
+ below). It is enabled by configuring the partition dimension of the layer to
+ 0, e.g.,
+
+ # with other fields omitted
+ layer {
+ partition_dim: 0
+ }
+
+ 2. Partitioning each singe layer into sub-layers on feature dimension (see
+ below). It is enabled by configuring the partition dimension of the layer to
+ 1, e.g.,
+
+ # with other fields omitted
+ layer {
+ partition_dim: 1
+ }
+
+ 3. Partitioning all layers into different subsets. It is enabled by
+ configuring the location ID of a layer, e.g.,
+
+ # with other fields omitted
+ layer {
+ location: 1
+ }
+ layer {
+ location: 0
+ }
+
+
+ 4. Hybrid partitioning of strategy 1, 2 and 3. The hybrid partitioning is
+ useful for large models. An example application is to implement the
+ [idea proposed by Alex](http://arxiv.org/abs/1404.5997).
+ Hybrid partitioning is configured like,
+
+ # with other fields omitted
+ layer {
+ location: 1
+ }
+ layer {
+ location: 0
+ }
+ layer {
+ partition_dim: 0
+ location: 0
+ }
+ layer {
+ partition_dim: 1
+ location: 0
+ }
+
+Currently SINGA supports strategy-2 well. Other partitioning strategies are
+are under test and will be released in later version.
+
+## Parameter sharing
+
+Parameters can be shared in two cases,
+
+ * sharing parameters among layers via user configuration. For example, the
+ visible layer and hidden layer of a RBM shares the weight matrix, which is configured through
+ the `share_from` field as shown in the above RBM configuration. The
+ configurations must be the same (except name) for shared parameters.
+
+ * due to neural net partitioning, some `Param` objects are replicated into
+ different workers, e.g., partitioning one layer on batch dimension. These
+ workers share parameter values. SINGA controls this kind of parameter
+ sharing automatically, users do not need to do any configuration.
+
+ * the `NeuralNet` for training and testing (and validation) share most layers
+ , thus share `Param` values.
+
+If the shared `Param` instances resident in the same process (may in different
+threads), they use the same chunk of memory space for their values. But they
+would have different memory spaces for their gradients. In fact, their
+gradients will be averaged by the stub or server.
+
+## Advanced user guide
+
+### Creation
+
+ static NeuralNet* NeuralNet::Create(const NetProto& np, Phase phase, int num);
+
+The above function creates a `NeuralNet` for a given phase, and returns a
+pointer to the `NeuralNet` instance. The phase is in {kTrain,
+kValidation, kTest}. `num` is used for net partitioning which indicates the
+number of partitions. Typically, a training job includes three neural nets for
+training, validation and test phase respectively. The three neural nets share most
+layers except the data layer, loss layer or output layer, etc.. The `Create`
+function takes in the full net configuration including layers for training,
+validation and test. It removes layers for phases other than the specified
+phase based on the `exclude` field in
+[layer configuration](layer.html):
+
+ layer {
+ ...
+ exclude : kTest # filter this layer for creating test net
+ }
+
+The filtered net configuration is passed to the constructor of `NeuralNet`:
+
+ NeuralNet::NeuralNet(NetProto netproto, int npartitions);
+
+The constructor creates a graph representing the net structure firstly in
+
+ Graph* NeuralNet::CreateGraph(const NetProto& netproto, int npartitions);
+
+Next, it creates a layer for each node and connects layers if their nodes are
+connected.
+
+ void NeuralNet::CreateNetFromGraph(Graph* graph, int npartitions);
+
+Since the `NeuralNet` instance may be shared among multiple workers, the
+`Create` function returns a pointer to the `NeuralNet` instance .
+
+### Parameter sharing
+
+ `Param` sharing
+is enabled by first sharing the Param configuration (in `NeuralNet::Create`)
+to create two similar (e.g., the same shape) Param objects, and then calling
+(in `NeuralNet::CreateNetFromGraph`),
+
+ void Param::ShareFrom(const Param& from);
+
+It is also possible to share `Param`s of two nets, e.g., sharing parameters of
+the training net and the test net,
+
+ void NeuralNet:ShareParamsFrom(NeuralNet* other);
+
+It will call `Param::ShareFrom` for each Param object.
+
+### Access functions
+`NeuralNet` provides a couple of access function to get the layers and params
+of the net:
+
+ const std::vector<Layer*>& layers() const;
+ const std::vector<Param*>& params() const ;
+ Layer* name2layer(string name) const;
+ Param* paramid2param(int id) const;
+
+
+### Partitioning
+
+
+#### Implementation
+
+SINGA partitions the neural net in `CreateGraph` function, which creates one
+node for each (partitioned) layer. For example, if one layer's partition
+dimension is 0 or 1, then it creates `npartition` nodes for it; if the
+partition dimension is -1, a single node is created, i.e., no partitioning.
+Each node is assigned a partition (or location) ID. If the original layer is
+configured with a location ID, then the ID is assigned to each newly created node.
+These nodes are connected according to the connections of the original layers.
+Some connection layers will be added automatically.
+For instance, if two connected sub-layers are located at two
+different workers, then a pair of bridge layers is inserted to transfer the
+feature (and gradient) blob between them. When two layers are partitioned on
+different dimensions, a concatenation layer which concatenates feature rows (or
+columns) and a slice layer which slices feature rows (or columns) would be
+inserted. These connection layers help making the network communication and
+synchronization transparent to the users.
+
+#### Dispatching partitions to workers
+
+Each (partitioned) layer is assigned a location ID, based on which it is dispatched to one
+worker. Particularly, the pointer to the `NeuralNet` instance is passed
+to every worker within the same group, but each worker only computes over the
+layers that have the same partition (or location) ID as the worker's ID. When
+every worker computes the gradients of the entire model parameters
+(strategy-2), we refer to this process as data parallelism. When different
+workers compute the gradients of different parameters (strategy-3 or
+strategy-1), we call this process model parallelism. The hybrid partitioning
+leads to hybrid parallelism where some workers compute the gradients of the
+same subset of model parameters while other workers compute on different model
+parameters. For example, to implement the hybrid parallelism in for the
+[DCNN model](http://arxiv.org/abs/1404.5997), we set `partition_dim = 0` for
+lower layers and `partition_dim = 1` for higher layers.
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neuralnet-partition.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neuralnet-partition.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neuralnet-partition.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/neuralnet-partition.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,54 @@
+# Neural Net Partition
+
+---
+
+The purposes of partitioning neural network is to distribute the partitions onto
+different working units (e.g., threads or nodes, called workers in this article)
+and parallelize the processing.
+Another reason for partition is to handle large neural network which cannot be
+hold in a single node. For instance, to train models against images with high
+resolution we need large neural networks (in terms of training parameters).
+
+Since *Layer* is the first class citizen in SIGNA, we do the partition against
+layers. Specifically, we support partitions at two levels. First, users can configure
+the location (i.e., worker ID) of each layer. In this way, users assign one worker
+for each layer. Secondly, for one layer, we can partition its neurons or partition
+the instances (e.g, images). They are called layer partition and data partition
+respectively. We illustrate the two types of partitions using an simple convolutional neural network.
+
+<img src="../images/conv-mnist.png" style="width: 220px"/>
+
+The above figure shows a convolutional neural network without any partition. It
+has 8 layers in total (one rectangular represents one layer). The first layer is
+DataLayer (data) which reads data from local disk files/databases (or HDFS). The second layer
+is a MnistLayer which parses the records from MNIST data to get the pixels of a batch
+of 8 images (each image is of size 28x28). The LabelLayer (label) parses the records to get the label
+of each image in the batch. The ConvolutionalLayer (conv1) transforms the input image to the
+shape of 8x27x27. The ReLULayer (relu1) conducts elementwise transformations. The PoolingLayer (pool1)
+sub-samples the images. The fc1 layer is fully connected with pool1 layer. It
+mulitplies each image with a weight matrix to generate a 10 dimension hidden feature which
+is then normalized by a SoftmaxLossLayer to get the prediction.
+
+<img src="../images/conv-mnist-datap.png" style="width: 1000px"/>
+
+The above figure shows the convolutional neural network after partitioning all layers
+except the DataLayer and ParserLayers, into 3 partitions using data partition.
+The read layers process 4 images of the batch, the black and blue layers process 2 images
+respectively. Some helper layers, i.e., SliceLayer, ConcateLayer, BridgeSrcLayer,
+BridgeDstLayer and SplitLayer, are added automatically by our partition algorithm.
+Layers of the same color resident in the same worker. There would be data transferring
+across different workers at the boundary layers (i.e., BridgeSrcLayer and BridgeDstLayer),
+e.g., between s-slice-mnist-conv1 and d-slice-mnist-conv1.
+
+<img src="../images/conv-mnist-layerp.png" style="width: 1000px"/>
+
+The above figure shows the convolutional neural network after partitioning all layers
+except the DataLayer and ParserLayers, into 2 partitions using layer partition. We can
+see that each layer processes all 8 images from the batch. But different partitions process
+different part of one image. For instance, the layer conv1-00 process only 4 channels. The other
+4 channels are processed by conv1-01 which residents in another worker.
+
+
+Since the partition is done at the layer level, we can apply different partitions for
+different layers to get a hybrid partition for the whole neural network. Moreover,
+we can also specify the layer locations to locate different layers to different workers.
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/overview.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/overview.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/overview.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/overview.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,70 @@
+# ê°ì
+
+---
+
+SINGAë ëê·ëª¨ ë°ì´í° ë¶ìì ìí ë¥ë¬ë 모ë¸ì í¸ë ì´ëì 목ì ì¼ë¡ í "ë¶ì° ë¥ë¬ë íë«í¼" ì
ëë¤. 모ë¸ì´ ëë ë´ë´ë¤í¸ìí¬ì "Layer" ê°ë
ì ë°ë¼ì ì§ê´ì ì¸ íë¡ê·¸ëë°ì í ì ìëë¡ ëìì¸ëì´ ììµëë¤.
+
+* Convolutional Neural Network ì ê°ì í¼ëí¬ìë ë¤í¸ìí¬ì Restricted Boltzmann Machine ê³¼ ê°ì ìëì§ ëª¨ë¸, Recurrent Neural Network ëª¨ë¸ ë± ë¤ìí 모ë¸ì ì§ìí©ëë¤.
+
+* ë¤ìí 기ë¥ì ê°ì§ë "Layer"ë¤ì´ Built-in Layer ë¡ ì¤ë¹ëì´ ììµëë¤.
+
+* SINGA ìí¤í
ì²ë synchronous (ë기), asynchronous (ë¹ë기), ê·¸ë¦¬ê³ hybrid (íì´ë¸ë¦¬ë) í¸ë ì´ëì í ì ìëë¡ ì¤ê³ëì´ ììµëë¤.
+
+* ëí 모ë¸ì í¸ë ì´ëì ë³ë ¬ííë ë¤ìí partition ì¤í´ (ë°°ì¹ ë° í¹ì§ ë¶í )ì ì§ìí©ëë¤.
+
+
+## 목ì
+
+íì¥ì± : ë¶ì° ìì¤í
ì¼ë¡ì¨ ë ë§ì ììì ì´ì©íì¬ í¹ì ì ë°ëì ëë¬ í ëê¹ì§ í¸ë ì´ë ìë를 í¥ììí¨ë¤.
+
+ì ì©ì± : ëê·ëª¨ ë¶ì° 모ë¸ì í¨ì¨ì ì¸ í¸ë ì´ëì íìí ë°ì´í°ì 모ë¸ì ë¶í , ë¤í¸ìí¬ íµì ë± íë¡ê·¸ë머ì ìì
ì ë¨ìííê³ , ë³µì¡í ëª¨ë¸ ë° ìê³ ë¦¬ì¦ì 구ì¶ì ì½ê² íë¤.
+
+
+## ì¤ê³ ì´ë
+
+íì¥ì±ì ë¶ì° ë¥ë¬ëìì ì¤ìí ì°êµ¬ ê³¼ì ì
ëë¤.
+SINGAë ë¤ìí í¸ë ì´ë íë ììí¬ì íì¥ì±ì ì ì§í ì ìëë¡ ì¤ê³ëì´ ììµëë¤.
+
+* Synchronous (ë기) : í¸ë ì´ëì 1ë¨ê³ìì ì»ì ììë í¨ê³¼ë¥¼ ëì
ëë¤.
+
+* Asynchronous (ë¹ë기) : í¸ë ì´ëì ìë ´ ìë를 í¥ììíµëë¤.
+
+* Hybrid (íì´ë¸ë¦¬ë) : ì½ì¤í¸ ë° ë¦¬ìì¤ (í´ë¬ì¤í° í¬ê¸° ë±)ì ë§ë í¨ê³¼ì ìë ´ ìëì ê· íì ì¡ê³ íì¥ì±ì í¥ììíµëë¤.
+
+SINGAë ë¥ë¬ë 모ë¸ì ë¤í¸ìí¬ "ë ì´ì´" ê°ë
ì ë°ë¼ ì§ê´ì ì¼ë¡ íë¡ê·¸ëë°ì í ì ìëë¡ ëìì¸ëì´ ììµëë¤. ë¤ìí 모ë¸ì ì½ê² 구ì¶íê³ í¸ë ì´ë í ì ììµëë¤.
+
+## ìì¤í
ê°ì
+
+<img src = "../../images/sgd.png" align="center" width="400px"/>
+<span><strong> Figure 1 - SGD Flow </strong></span>
+
+"ë¥ë¬ë 모ë¸ì íìµíë¤"ë ê²ì í¹ì ìì
(ë¶ë¥, ì측 ë±)ì ë¬ì±í기 ìíì¬ ì¬ì©ëë í¹ì§ë(feature)ì ìì±íë ë³í í¨ìì ìµì íë¼ë¯¸í°ë¥¼ ì°¾ëë¤ë ê²ì
ëë¤.
+ê·¸ ë³ìì ì¢ê³ ëì¨ì, Cross-Entropy Loss (https://en.wikipedia.org/wiki/Cross_entropy) ë±ì loss function (ìì¤ í¨ì)ì¼ë¡ íì¸í©ëë¤. ì´ í¨ìë ì¼ë°ì ì¼ë¡ ë¹ì í ëë ë¹ ë³¼ë¡ í¨ìì´ë¯ë¡ é解ì ì°¾ê¸°ê° íëëë¤.
+
+ê·¸ëì Stochastic Gradient Descent (íë¥ ì 구배ê°íë²)ì ì´ì©í©ëë¤.
+Figure 1ê³¼ ê°ì´ ëë¤ì¼ë¡ ì´ê¸°í ë íë¼ë¯¸í° ê°ì, ìì¤ í¨ìê° ìì ì§ëë¡ ë°ë³µ ì
ë°ì´í¸íê³ ììµëë¤.
+
+<img src = "../../images/overview.png" align="center" width="400px"/>
+<span> <strong> Figure 2 - SINGA Overview </strong> </span>
+
+í¸ë ì´ëì íìí ìí¬ë¡ëë workers ì servers ì ë¶ì°ë©ëë¤.
+Figure 2ì ê°ì´ 루í(iteration)ë§ë¤ workers ë *TrainOneBatch* í¨ì를 ë¶ë¬ì íë¼ë¯¸í° 구배를 ê³ì°í©ëë¤.
+*TrainOneBatch* ë ë´ë´ë¤í¸ìí¬ì êµ¬ì¡°ê° ê¸°ì ë *NeuralNet* ì ë³´ì ë°ë¼ì "Layer"를 ì°¨ë¡ë¡ ëë¬ë´
ëë¤.
+ê³ì° ë 구배ë ë¡ì»¬ë
¸ëì stub ì ë³´ë´ì ¸ ì§ê³ ë í, í´ë¹ servers ì ì ì¡ë©ëë¤. Servers ë ì
ë°ì´í¸ ë íë¼ë¯¸í° ê°ì workers íí
ëëë ¤ì£¼ê³ , ë¤ì 루í(iteration)를 ì§íí©ëë¤.
+
+
+## Job
+
+SINGAìì "Job"ì´ë ë´ë´ë¤í¸ìí¬ ëª¨ë¸ê³¼ ë°ì´í° í¸ë ì´ë ë°©ë², í´ë¬ì¤í° í í´ë¡ì§ ë±ì´ 기ì ë "Job Configuration"ì ë§í©ëë¤.
+Job configuration ì Figure 2ì ê·¸ë ¤ì§ ë¤ìì 4ê°ì§ ìì를 ê°ì§ëë¤.
+
+* [NeuralNet](neural-net.html) : ë´ë´ë¤í¸ìí¬ì 구조ì ê° "ë ì´ì´"ì ì¤ì ì 기ì í©ëë¤.
+* [TrainOneBatch](train-one-batch.html) : ëª¨ë¸ ì¹´í
ê³ ë¦¬ì ì í©í ìê³ ë¦¬ì¦ì 기ì í©ëë¤.
+* [Updater](updater.html) : serverìì ë§¤ê° ë³ì를 ì
ë°ì´í¸íë ë°©ë²ì 기ì í©ëë¤.
+* [Cluster Topology](distributed-training.html) : workersì servers ë¶ì° í í´ë¡ì§ë¥¼ 기ì í©ëë¤.
+
+[main í¨ì](programming-guide.html)ì SINGA ëë¼ì´ë²ë¥¼ ì¨ì Job ì ëê¹ëë¤.
+
+ì´ íë¡ì¸ì¤ë Hadoopììì Job ìë¸ë¯¸ì
ê³¼ ë¹ì·í©ëë¤.
+ì ì ê° main í¨ììì ìì
ì¤ì ì í©ëë¤.
+Hadoop ì ì ë ìì ì mapperì reducer를 ì¤ì íì§ë§ SINGA ììë ì ì ì Layer ë Updater ë±ì ì¤ì í©ëë¤.
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/param.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/param.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/param.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/param.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,226 @@
+# Parameters
+
+---
+
+A `Param` object in SINGA represents a set of parameters, e.g., a weight matrix
+or a bias vector. *Basic user guide* describes how to configure for a `Param`
+object, and *Advanced user guide* provides details on implementing users'
+parameter initialization methods.
+
+## Basic user guide
+
+The configuration of a Param object is inside a layer configuration, as the
+`Param` are associated with layers. An example configuration is like
+
+ layer {
+ ...
+ param {
+ name : "p1"
+ init {
+ type : kConstant
+ value: 1
+ }
+ }
+ }
+
+The [SGD algorithm](overview.html) starts with initializing all
+parameters according to user specified initialization method (the `init` field).
+For the above example,
+all parameters in `Param` "p1" will be initialized to constant value 1. The
+configuration fields of a Param object is defined in [ParamProto](../api/classsinga_1_1ParamProto.html):
+
+ * name, an identifier string. It is an optional field. If not provided, SINGA
+ will generate one based on layer name and its order in the layer.
+ * init, field for setting initialization methods.
+ * share_from, name of another `Param` object, from which this `Param` will share
+ configurations and values.
+ * lr_scale, float value to be multiplied with the learning rate when
+ [updating the parameters](updater.html)
+ * wd_scale, float value to be multiplied with the weight decay when
+ [updating the parameters](updater.html)
+
+There are some other fields that are specific to initialization methods.
+
+### Initialization methods
+
+Users can set the `type` of `init` use the following built-in initialization
+methods,
+
+ * `kConst`, set all parameters of the Param object to a constant value
+
+ type: kConst
+ value: float # default is 1
+
+ * `kGaussian`, initialize the parameters following a Gaussian distribution.
+
+ type: kGaussian
+ mean: float # mean of the Gaussian distribution, default is 0
+ std: float # standard variance, default is 1
+ value: float # default 0
+
+ * `kUniform`, initialize the parameters following an uniform distribution
+
+ type: kUniform
+ low: float # lower boundary, default is -1
+ high: float # upper boundary, default is 1
+ value: float # default 0
+
+ * `kGaussianSqrtFanIn`, initialize `Param` objects with two dimensions (i.e.,
+ matrix) using `kGaussian` and then
+ multiple each parameter with `1/sqrt(fan_in)`, where`fan_in` is the number of
+ columns of the matrix.
+
+ * `kUniformSqrtFanIn`, the same as `kGaussianSqrtFanIn` except that the
+ distribution is an uniform distribution.
+
+ * `kUniformFanInOut`, initialize matrix `Param` objects using `kUniform` and then
+ multiple each parameter with `sqrt(6/(fan_in + fan_out))`, where`fan_in +
+ fan_out` sums up the number of columns and rows of the matrix.
+
+For all above initialization methods except `kConst`, if their `value` is not
+1, every parameter will be multiplied with `value`. Users can also implement
+their own initialization method following the *Advanced user guide*.
+
+
+## Advanced user guide
+
+This sections describes the details on implementing new parameter
+initialization methods.
+
+### Base ParamGenerator
+All initialization methods are implemented as
+subclasses of the base `ParamGenerator` class.
+
+ class ParamGenerator {
+ public:
+ virtual void Init(const ParamGenProto&);
+ void Fill(Param*);
+
+ protected:
+ ParamGenProto proto_;
+ };
+
+Configurations of the initialization method is in `ParamGenProto`. The `Fill`
+function fills the `Param` object (passed in as an argument).
+
+### New ParamGenerator subclass
+
+Similar to implement a new Layer subclass, users can define a configuration
+protocol message,
+
+ # in user.proto
+ message FooParamProto {
+ optional int32 x = 1;
+ }
+ extend ParamGenProto {
+ optional FooParamProto fooparam_conf =101;
+ }
+
+The configuration of `Param` would be
+
+ param {
+ ...
+ init {
+ user_type: 'FooParam" # must use user_type for user defined methods
+ [fooparam_conf] { # must use brackets for configuring user defined messages
+ x: 10
+ }
+ }
+ }
+
+The subclass could be declared as,
+
+ class FooParamGen : public ParamGenerator {
+ public:
+ void Fill(Param*) override;
+ };
+
+Users can access the configuration fields in `Fill` by
+
+ int x = proto_.GetExtension(fooparam_conf).x();
+
+To use the new initialization method, users need to register it in the
+[main function](programming-guide.html).
+
+ driver.RegisterParamGenerator<FooParamGen>("FooParam") # must be consistent with the user_type in configuration
+
+{% comment %}
+### Base Param class
+
+### Members
+
+ int local_version_;
+ int slice_start_;
+ vector<int> slice_offset_, slice_size_;
+
+ shared_ptr<Blob<float>> data_;
+ Blob<float> grad_;
+ ParamProto proto_;
+
+Each Param object has a local version and a global version (inside the data
+Blob). These two versions are used for synchronization. If multiple Param
+objects share the same values, they would have the same `data_` field.
+Consequently, their global version is the same. The global version is updated
+by [the stub thread](communication.html). The local version is
+updated in `Worker::Update` function which assigns the global version to the
+local version. The `Worker::Collect` function is blocked until the global
+version is larger than the local version, i.e., when `data_` is updated. In
+this way, we synchronize workers sharing parameters.
+
+In Deep learning models, some Param objects are 100 times larger than others.
+To ensure the load-balance among servers, SINGA slices large Param objects. The
+slicing information is recorded by `slice_*`. Each slice is assigned a unique
+ID starting from 0. `slice_start_` is the ID of the first slice of this Param
+object. `slice_offset_[i]` is the offset of the i-th slice in this Param
+object. `slice_size_[i]` is the size of the i-th slice. These slice information
+is used to create messages for transferring parameter values or gradients to
+different servers.
+
+Each Param object has a `grad_` field for gradients. Param objects do not share
+this Blob although they may share `data_`. Because each layer containing a
+Param object would contribute gradients. E.g., in RNN, the recurrent layers
+share parameters values, and the gradients used for updating are averaged from all recurrent
+these recurrent layers. In SINGA, the stub thread will aggregate local
+gradients for the same Param object. The server will do a global aggregation
+of gradients for the same Param object.
+
+The `proto_` field has some meta information, e.g., name and ID. It also has a
+field called `owner` which is the ID of the Param object that shares parameter
+values with others.
+
+### Functions
+The base Param class implements two sets of functions,
+
+ virtual void InitValues(int version = 0); // initialize values according to `init_method`
+ void ShareFrom(const Param& other); // share `data_` from `other` Param
+ --------------
+ virtual Msg* GenGetMsg(bool copy, int slice_idx);
+ virtual Msg* GenPutMsg(bool copy, int slice_idx);
+ ... // other message related functions.
+
+Besides the functions for processing the parameter values, there is a set of
+functions for generating and parsing messages. These messages are for
+transferring parameter values or gradients between workers and servers. Each
+message corresponds to one Param slice. If `copy` is false, it means the
+receiver of this message is in the same process as the sender. In such case,
+only pointers to the memory of parameter value (or gradient) are wrapped in
+the message; otherwise, the parameter values (or gradients) should be copied
+into the message.
+
+
+## Implementing Param subclass
+Users can extend the base Param class to implement their own parameter
+initialization methods and message transferring protocols. Similar to
+implementing a new Layer subclasses, users can create google protocol buffer
+messages for configuring the Param subclass. The subclass, denoted as FooParam
+should be registered in main.cc,
+
+ dirver.RegisterParam<FooParam>(kFooParam); // kFooParam should be different to 0, which is for the base Param type
+
+
+ * type, an integer representing the `Param` type. Currently SINGA provides one
+ `Param` implementation with type 0 (the default type). If users want
+ to use their own Param implementation, they should extend the base Param
+ class and configure this field with `kUserParam`
+
+{% endcomment %}
Added: incubator/singa/site/trunk/content/markdown/v0.3.0/kr/programmer-guide.md
URL: http://svn.apache.org/viewvc/incubator/singa/site/trunk/content/markdown/v0.3.0/kr/programmer-guide.md?rev=1740048&view=auto
==============================================================================
--- incubator/singa/site/trunk/content/markdown/v0.3.0/kr/programmer-guide.md (added)
+++ incubator/singa/site/trunk/content/markdown/v0.3.0/kr/programmer-guide.md Wed Apr 20 05:09:06 2016
@@ -0,0 +1,97 @@
+# Programmer Guide
+
+---
+
+To submit a training job, users must provide the configuration of the
+four components shown in Figure 1:
+
+ * a [NeuralNet](neural-net.html) describing the neural net structure with the detailed layer setting and their connections;
+ * a [TrainOneBatch](train-one-batch.html) algorithm which is tailored for different model categories;
+ * an [Updater](updater.html) defining the protocol for updating parameters at the server side;
+ * a [Cluster Topology](distributed-training.html) specifying the distributed architecture of workers and servers.
+
+The *Basic user guide* section describes how to submit a training job using
+built-in components; while the *Advanced user guide* section presents details
+on writing user's own main function to register components implemented by
+themselves. In addition, the training data must be prepared, which has the same
+[process](data.html) for both advanced users and basic users.
+
+<img src="../../images/overview.png" align="center" width="400px"/>
+<span><strong>Figure 1 - SINGA overview.</strong></span>
+
+
+
+## Basic user guide
+
+Users can use the default main function provided by SINGA to submit the training
+job. For this case, a job configuration file written as a google protocol
+buffer message for the [JobProto](../api/classsinga_1_1JobProto.html) must be provided in the command line,
+
+ ./bin/singa-run.sh -conf <path to job conf> [-resume] [-test]
+
+* `-resume` is for continuing the training from last [checkpoint](checkpoint.html).
+* `-test` is for testing the performance of previously trained model and extracting features for new data,
+more details are available [here](test.html).
+
+The [MLP](mlp.html) and [CNN](cnn.html)
+examples use built-in components. Please read the corresponding pages for their
+job configuration files. The subsequent pages will illustrate the details on
+each component of the configuration.
+
+## Advanced user guide
+
+If a user's model contains some user-defined components, e.g.,
+[Updater](updater.html), he has to write a main function to
+register these components. It is similar to Hadoop's main function. Generally,
+the main function should
+
+ * initialize SINGA, e.g., setup logging.
+
+ * register user-defined components.
+
+ * create and pass the job configuration to SINGA driver
+
+An example main function is like
+
+ #include <string>
+ #include "singa.h"
+ #include "user.h" // header for user code
+
+ int main(int argc, char** argv) {
+ singa::Driver driver;
+ driver.Init(argc, argv);
+ bool resume;
+ // parse resume option from argv.
+
+ // register user defined layers
+ driver.RegisterLayer<FooLayer, std::string>("kFooLayer");
+ // register user defined updater
+ driver.RegisterUpdater<FooUpdater, std::string>("kFooUpdater");
+ ...
+ auto jobConf = driver.job_conf();
+ // update jobConf
+
+ driver.Submit(resume, jobConf);
+ return 0;
+ }
+
+The Driver class' `Init` method will load a job configuration file provided by
+users as a command line argument (`-conf <job conf>`). It contains at least the
+cluster topology and returns the `jobConf` for users to update or fill in
+configurations of neural net, updater, etc. If users define subclasses of
+Layer, Updater, Worker and Param, they should register them through the driver.
+Finally, the job configuration is submitted to the driver which starts the
+training.
+
+We will provide helper functions to make the configuration easier in the
+future, like [keras](https://github.com/fchollet/keras).
+
+Users need to compile and link their code (e.g., layer implementations and the main
+file) with SINGA library (*.libs/libsinga.so*) to generate an
+executable file, e.g., with name *mysinga*. To launch the program, users just pass the
+path of the *mysinga* and base job configuration to *./bin/singa-run.sh*.
+
+ ./bin/singa-run.sh -conf <path to job conf> -exec <path to mysinga> [other arguments]
+
+The [RNN application](rnn.html) provides a full example of
+implementing the main function for training a specific RNN model.