You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by bu...@apache.org on 2015/08/15 12:56:47 UTC
svn commit: r961914 - in /websites/staging/singa/trunk/content: ./
community/ develop/ docs/
Author: buildbot
Date: Sat Aug 15 10:56:46 2015
New Revision: 961914
Log:
Staging update by buildbot for singa
Removed:
websites/staging/singa/trunk/content/docs/neuralnet.html
Modified:
websites/staging/singa/trunk/content/ (props changed)
websites/staging/singa/trunk/content/community.html
websites/staging/singa/trunk/content/community/issue-tracking.html
websites/staging/singa/trunk/content/community/mail-lists.html
websites/staging/singa/trunk/content/community/source-repository.html
websites/staging/singa/trunk/content/community/team-list.html
websites/staging/singa/trunk/content/develop/contribute-code.html
websites/staging/singa/trunk/content/develop/contribute-docs.html
websites/staging/singa/trunk/content/develop/how-contribute.html
websites/staging/singa/trunk/content/develop/schedule.html
websites/staging/singa/trunk/content/docs.html
websites/staging/singa/trunk/content/docs/architecture.html
websites/staging/singa/trunk/content/docs/checkpoint.html
websites/staging/singa/trunk/content/docs/cnn.html
websites/staging/singa/trunk/content/docs/code-structure.html
websites/staging/singa/trunk/content/docs/communication.html
websites/staging/singa/trunk/content/docs/data.html
websites/staging/singa/trunk/content/docs/datashard.html
websites/staging/singa/trunk/content/docs/debug.html
websites/staging/singa/trunk/content/docs/examples.html
websites/staging/singa/trunk/content/docs/hdfs.html
websites/staging/singa/trunk/content/docs/installation.html
websites/staging/singa/trunk/content/docs/layer.html
websites/staging/singa/trunk/content/docs/lmdb.html
websites/staging/singa/trunk/content/docs/mlp.html
websites/staging/singa/trunk/content/docs/model-config.html
websites/staging/singa/trunk/content/docs/neuralnet-partition.html
websites/staging/singa/trunk/content/docs/programmer-guide.html
websites/staging/singa/trunk/content/docs/rbm.html
websites/staging/singa/trunk/content/docs/rnn.html
websites/staging/singa/trunk/content/index.html
websites/staging/singa/trunk/content/introduction.html
websites/staging/singa/trunk/content/quick-start.html
Propchange: websites/staging/singa/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Sat Aug 15 10:56:46 2015
@@ -1 +1 @@
-1694636
+1696034
Modified: websites/staging/singa/trunk/content/community.html
==============================================================================
--- websites/staging/singa/trunk/content/community.html (original)
+++ websites/staging/singa/trunk/content/community.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Community</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/community/issue-tracking.html
==============================================================================
--- websites/staging/singa/trunk/content/community/issue-tracking.html (original)
+++ websites/staging/singa/trunk/content/community/issue-tracking.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Issue Tracking</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/community/mail-lists.html
==============================================================================
--- websites/staging/singa/trunk/content/community/mail-lists.html (original)
+++ websites/staging/singa/trunk/content/community/mail-lists.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Project Mailing Lists</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/community/source-repository.html
==============================================================================
--- websites/staging/singa/trunk/content/community/source-repository.html (original)
+++ websites/staging/singa/trunk/content/community/source-repository.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Source Repository</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/community/team-list.html
==============================================================================
--- websites/staging/singa/trunk/content/community/team-list.html (original)
+++ websites/staging/singa/trunk/content/community/team-list.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – The SINGA Team</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/develop/contribute-code.html
==============================================================================
--- websites/staging/singa/trunk/content/develop/contribute-code.html (original)
+++ websites/staging/singa/trunk/content/develop/contribute-code.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – How to Contribute Code</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/develop/contribute-docs.html
==============================================================================
--- websites/staging/singa/trunk/content/develop/contribute-docs.html (original)
+++ websites/staging/singa/trunk/content/develop/contribute-docs.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – How to Contribute Documentation</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/develop/how-contribute.html
==============================================================================
--- websites/staging/singa/trunk/content/develop/how-contribute.html (original)
+++ websites/staging/singa/trunk/content/develop/how-contribute.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – How to Contribute to SINGA</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/develop/schedule.html
==============================================================================
--- websites/staging/singa/trunk/content/develop/schedule.html (original)
+++ websites/staging/singa/trunk/content/develop/schedule.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Development Schedule</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs.html
==============================================================================
--- websites/staging/singa/trunk/content/docs.html (original)
+++ websites/staging/singa/trunk/content/docs.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Documentation</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/architecture.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/architecture.html (original)
+++ websites/staging/singa/trunk/content/docs/architecture.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – System Architecture</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/checkpoint.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/checkpoint.html (original)
+++ websites/staging/singa/trunk/content/docs/checkpoint.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Checkpoint and Resume</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/cnn.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/cnn.html (original)
+++ websites/staging/singa/trunk/content/docs/cnn.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/code-structure.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/code-structure.html (original)
+++ websites/staging/singa/trunk/content/docs/code-structure.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Code Structure</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/communication.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/communication.html (original)
+++ websites/staging/singa/trunk/content/docs/communication.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Communication</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
@@ -404,7 +404,7 @@
<div class="section">
<h2><a name="Communication"></a>Communication</h2>
<hr />
-<p>Different messaging libraries has different benefits and drawbacks. For instance, MPI provides fast message passing between GPUs (using GPUDirect), but does not support fault-tolerance well. On the contrary, systems using ZeroMQ can be fault-tolerant. But ZeroMQ does not support GPUDirect. The AllReduce function of MPI is also missing in ZeroMQ which is efficient for data aggregation for distributed training. In Singa, we provide general messaging APIs for communication between threads within a process and across processes, and let users choose the underlying implementation (MPI or ZeroMQ) that meets their requirements.</p>
+<p>Different messaging libraries has different benefits and drawbacks. For instance, MPI provides fast message passing between GPUs (using GPUDirect), but does not support fault-tolerance well. On the contrary, systems using ZeroMQ can be fault-tolerant, but does not support GPUDirect. The AllReduce function of MPI is also missing in ZeroMQ which is efficient for data aggregation for distributed training. In Singa, we provide general messaging APIs for communication between threads within a process and across processes, and let users choose the underlying implementation (MPI or ZeroMQ) that meets their requirements.</p>
<p>Singa’s messaging library consists of two components, namely the message, and the socket to send and receive messages. <b>Socket</b> refers to a Singa defined data structure instead of the Linux Socket. We will introduce the two components in detail with the following figure as an example architecture.</p>
<p><img src="../images/arch/arch2.png" style="width: 550px" alt="" /> <img src="../images/arch/comm.png" style="width: 550px" alt="" />
<p><b> Fig.1 - Example physical architecture and network connection</b></p>
@@ -413,7 +413,7 @@
<h3><a name="Message"></a>Message</h3>
<p><object type="image/svg+xml" style="width: 100px" data="../images/msg.svg"> Not supported </object>
<p><b> Fig.2 - Logical message format</b></p>
-<p>Fig.2 shows the logical message format which has two parts, the header and the content. The message header includes the sender and receiver’s ID consisting of the group ID and the worker/server ID within the group. The stub forwards messages by looking up an address table based on the receiver’s ID. There are two sets of messages, their message types are:</p>
+<p>Fig.2 shows the logical message format which has two parts, the header and the content. The message header includes the sender’s and receiver’s IDs, each consisting of the group ID and the worker/server ID within the group. The stub forwards messages by looking up an address table based on the receiver’s ID. There are two sets of messages according to the message type defined below.</p>
<ul>
@@ -423,97 +423,259 @@
<li>
<p>kFeaBlob/kGradBlob for messages about transferring feature and gradient blobs of one layer to its neighboring layer</p></li>
</ul>
-<p>There is a target ID in the header. If the message is related to parameters, the target ID is then the parameter ID. Otherwise the message is related to layer feature or gradient, and the target ID consists of the layer ID and the blob ID of that layer. The message content has multiple frames to store the parameter or feature data.</p>
+<p>There is a target ID in the header. If the message body is parameters, the target ID is then the parameter ID. Otherwise the message is related to layer feature or gradient, and the target ID consists of the layer ID and the blob ID of that layer. The message content has multiple frames to store the parameter or feature data.</p>
<p>The API for the base Msg is:</p>
<div class="source">
-<div class="source"><pre class="prettyprint">class Msg{
+<div class="source"><pre class="prettyprint">/**
+ * Msg used to transfer Param info (gradient or value), feature blob, etc
+ * between workers, stubs and servers.
+ *
+ * Each msg has a source addr and dest addr identified by a unique integer.
+ * It is also associated with a target field (value and version) for ease of
+ * getting some meta info (e.g., parameter id) from the msg.
+ *
+ * Other data is added into the message as frames.
+ */
+class Msg {
public:
+ ~Msg();
+ Msg();
/**
- * Destructor to free memory
+ * Construct the msg providing source and destination addr.
*/
- virtual ~Msg()=0;
+ Msg(int src, int dst);
/**
- * @param group_id worker/server group id
- * @param id worker/server id within the group
- * @param flag 0 for server, 1 for worker, 2 for stub
- */
- virtual void set_src(int group_id, int id, int flag)=0;
- virtual void set_dst(int group_id, int id, int flag)=0;
- virtual void src_group_id()=0;
- virtual int src_id() const=0;
- virtual int dst_group_id() const=0;
- virtual int dst_id() const=0;
- virtual int src_flag() const=0;
- virtual int dst_flag() const=0;
- virtual void set_type(int type)=0;
- virtual void set_target(int param_id)=0;
- virtual void set_target(int layer_id, blob_id)=0;
- virtual int type() const=0;
- /**
- * @return true if the msg is about parameter, otherwise false
- */
- virtual bool is_param_msg() const=0;
- virtual int param_id() const=0;
- virtual int layer_id() const=0;
- virtual int blob_id() const=0;
-
- virtual void add_frame(void*, int nBytes)=0;
- virtual int frame_size()=0;
- virtual void* frame_data()=0;
+ * Copy constructor.
+ */
+ Msg(const Msg& msg);
+ /**
+ * Swap the src/dst addr
+ */
+ void SwapAddr();
+ /**
+ * Add a frame (a chunk of bytes) into the message
+ */
+ void AddFrame(const void* addr, int nBytes);
+ /**
+ * @return num of bytes of the current frame.
+ */
+ int FrameSize();
+ /**
+ * @return the pointer to the current frame data.
+ */
+ void* FrameData();
+ /**
+ * @return the data of the current frame as c string
+ */
+ char* FrameStr();
+ /**
+ * Move the cursor to the first frame.
+ */
+ void FirstFrame();
+ /**
+ * Move the cursor to the last frame.
+ */
+ void LastFrame();
/**
* Move the cursor to the next frame
* @return true if the next frame is not NULL; otherwise false
*/
- virtual bool next_frame()=0;
- virtual int SerializeTo(string* buf);
- virtual int ParseFrom(const string& buf);
+ bool NextFrame();
+ /**
+ * Add a 'format' frame to the msg (like CZMQ's zsock_send).
+ *
+ * The format is a string that defines the type of each field.
+ * The format can contain any of these characters, each corresponding to
+ * one or two arguments:
+ * i = int (signed)
+ * 1 = uint8_t
+ * 2 = uint16_t
+ * 4 = uint32_t
+ * 8 = uint64_t
+ * p = void * (sends the pointer value, only meaningful over inproc)
+ * s = char**
+ *
+ * Returns size of the added content.
+ */
+ int AddFormatFrame(const char *format, ...);
+ /**
+ * Parse the current frame added using AddFormatFrame(const char*, ...).
+ *
+ * The format is a string that defines the type of each field.
+ * The format can contain any of these characters, each corresponding to
+ * one or two arguments:
+ * i = int (signed)
+ * 1 = uint8_t
+ * 2 = uint16_t
+ * 4 = uint32_t
+ * 8 = uint64_t
+ * p = void * (sends the pointer value, only meaningful over inproc)
+ * s = char**
+ *
+ * Returns size of the parsed content.
+ */
+ int ParseFormatFrame(const char* format, ...);
+
+#ifdef USE_ZMQ
+ void ParseFromZmsg(zmsg_t* msg);
+ zmsg_t* DumpToZmsg();
+#endif
+
+ /**
+ * @return msg size in terms of bytes, ignore meta info.
+ */
+ int size() const;
+ /**
+ * Set source addr.
+ * @param addr unique identify one worker/server/stub in the current job
+ */
+ void set_src(int addr) { src_ = addr; }
+ /**
+ * @return source addr.
+ */
+ int src() const { return src_; }
+ /**
+ * Set destination addr.
+ * @param addr unique identify one worker/server/stub in the current job
+ */
+ void set_dst(int addr) { dst_ = addr; }
+ /**
+ * @return dst addr.
+ */
+ int dst() const { return dst_; }
+ /**
+ * Set msg type, e.g., kPut, kGet, kUpdate, kRequest
+ */
+ void set_type(int type) { type_ = type; }
+ /**
+ * @return msg type.
+ */
+ int type() const { return type_; }
+ /**
+ * Set msg target.
+ *
+ * One msg has a target to identify some entity in worker/server/stub.
+ * The target is associated with a version, e.g., Param version.
+ */
+ void set_trgt(int val, int version) {
+ trgt_val_ = val;
+ trgt_version_ = version;
+ }
+ int trgt_val() const {
+ return trgt_val_;
+ }
+ int trgt_version() const {
+ return trgt_version_;
+ }
+
};
+</pre></div></div>
+<p>In order for a Msg object to be routed, the source and dest address should be attached. This is achieved by calling the set_src and set_dst methods of the Msg object. The address parameter passed to these two methods can be manipulated via a set of helper functions, shown as below.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">/**
+ * Wrapper to generate message address
+ * @param grp worker/server group id
+ * @param id_or_proc worker/server id or procs id
+ * @param type msg type
+ */
+inline int Addr(int grp, int id_or_proc, int type) {
+ return (grp << 16) | (id_or_proc << 8) | type;
+}
+
+/**
+ * Parse group id from addr.
+ *
+ * @return group id
+ */
+inline int AddrGrp(int addr) {
+ return addr >> 16;
+}
+/**
+ * Parse worker/server id from addr.
+ *
+ * @return id
+ */
+inline int AddrID(int addr) {
+ static const int mask = (1 << 8) - 1;
+ return (addr >> 8) & mask;
+}
+
+/**
+ * Parse worker/server procs from addr.
+ *
+ * @return procs id
+ */
+inline int AddrProc(int addr) {
+ return AddrID(addr);
+}
+/**
+ * Parse msg type from addr
+ * @return msg type
+ */
+inline int AddrType(int addr) {
+ static const int mask = (1 << 8) -1;
+ return addr & mask;
+}
</pre></div></div></div>
<div class="section">
<h3><a name="Socket"></a>Socket</h3>
-<p>In Singa, there are two types of sockets, the Dealer Socket and the Router Socket. The names are from ZeroMQ. All connections are of the same type, i.e., Dealer<–>Router. The communication between dealers and routers are asynchronous. In other words, one Dealer socket can talk with multiple Router sockets, and one Router socket can talk with multiple Dealer sockets.</p>
+<p>In Singa, there are two types of sockets, the Dealer Socket and the Router Socket, whose names are adapted from ZeroMQ. All connections are of the same type, i.e., Dealer<–>Router. The communication between dealers and routers are asynchronous. In other words, one Dealer socket can talk with multiple Router sockets, and one Router socket can talk with multiple Dealer sockets.</p>
<div class="section">
<h4><a name="Base_Socket"></a>Base Socket</h4>
<p>The basic functions of a Singa Socket is to send and receive messages. The APIs are:</p>
<div class="source">
-<div class="source"><pre class="prettyprint">class Socket{
+<div class="source"><pre class="prettyprint">class SocketInterface {
public:
+ virtual ~SocketInterface() {}
/**
- * @param args depending on the underlying implementation.
- */
- Socket(void* args);
+ * Send a message to connected socket(s), non-blocking. The message
+ * will be deallocated after sending, thus should not be used after
+ * calling Send();
+ *
+ * @param msg The message to be sent
+ * @return 1 for success queuing the message for sending, 0 for failure
+ */
+ virtual int Send(Msg** msg) = 0;
/**
- * Send a message to connected socket(s), non-blocking. The message will
- * be deallocated after sending, thus should not be used after calling Send();
- * @param the message to be sent
- * @param dst the identifier of the connected socket. By default, it is
- * -1, which means sending this message to all connected sockets.
- * @return 1 for success queuing the message for sending, 0 for failure
- */
- virtual int Send(Msg** msg, int dst=-1)=0;
+ * Receive a message from any connected socket.
+ *
+ * @return a message pointer if success; nullptr if failure
+ */
+ virtual Msg* Receive() = 0;
/**
- * Receive a message
- * @return a message pointer if success; nullptr if failure
+ * @return Identifier of the implementation dependent socket. E.g., zsock_t*
+ * for ZeroMQ implementation and rank for MPI implementation.
*/
- virtual Message* Receive()=0;
+ virtual void* InternalID() const = 0;
};
+</pre></div></div>
+<p>A poller class is provided to enable asynchronous communication between routers and dealers. One can register a set of SocketInterface objects with a poller instance via calling its Add method, and then call the Wait method of this poll object to wait for the registered SocketInterface objects to be ready for sending and receiving messages. The APIs of the poller class is shown below.</p>
-class Poller{
+<div class="source">
+<div class="source"><pre class="prettyprint">class Poller {
public:
+ Poller();
+ Poller(SocketInterface* socket);
/**
- * Add a socket for polling; Multiple sockets can be polled together by
- * adding them into the same poller.
- */
- void Add(Socket* socket);
+ * Add a socket for polling; Multiple sockets can be polled together by
+ * adding them into the same poller.
+ */
+ void Add(SocketInterface* socket);
/**
- * Poll for all sockets added into this poller.
- * @param duration stop after this number of milliseconds
- * @return pointer to the socket if it has one message in the receiving
- * queue; nullptr if no message in any sockets,
+ * Poll for all sockets added into this poller.
+ * @param timeout Stop after this number of mseconds
+ * @return pointer To the socket if it has one message in the receiving
+ * queue; nullptr if no message in any sockets,
+ */
+ SocketInterface* Wait(int duration);
+
+ /**
+ * @return true if the poller is terminated due to process interupt
*/
- Socket* Poll(int duation);
+ virtual bool Terminated();
};
</pre></div></div></div>
<div class="section">
@@ -521,51 +683,69 @@ class Poller{
<p>The Dealer socket inherits from the base Socket. In Singa, every Dealer socket only connects to one Router socket as shown in Fig.1. The connection is set up by connecting the Dealer socket to the endpoint of a Router socket.</p>
<div class="source">
-<div class="source"><pre class="prettyprint">class Dealer : public Socket{
+<div class="source"><pre class="prettyprint">class Dealer : public SocketInterface {
public:
- /**
- * Blocking operation to setup the connection with the router, called
- * only once.
- * @param endpoint identifier of the router. For intra-process
- * connection, the endpoint follows the format of ZeroMQ, i.e.,
- * starting with "inproc://"; in Singa, since each process has one
- * router, hence we can fix the endpoint to be "inproc://router" for
- * intra-process. For inter-process, the endpoint follows ZeroMQ's
- * format, i.e., IP:port, where IP is the connected process.
- * @return 1 connection sets up successfully; 0 otherwise
- /
- int Connect(string endpoint);
/*
- * Since the Dealer socket connects to only one router, it must send to
- * the connected router, thus the dst argument is useless.
+ * @param id Local dealer ID within a procs if the dealer is from worker or
+ * server thread, starts from 1 (0 is used by the router); or the connected
+ * remote procs ID for inter-process dealers from the stub thread.
*/
- virtual int Send(Msg** msg, int dst=-1);
- virtual Message* Receive();
+ Dealer();
+ explicit Dealer(int id);
+ ~Dealer() override;
+ /**
+ * Setup the connection with the router.
+ *
+ * @param endpoint Identifier of the router. For intra-process
+ * connection, the endpoint follows the format of ZeroMQ, i.e.,
+ * starting with "inproc://"; in Singa, since each process has one
+ * router, hence we can fix the endpoint to be "inproc://router" for
+ * intra-process. For inter-process, the endpoint follows ZeroMQ's
+ * format, i.e., IP:port, where IP is the connected process.
+ * @return 1 connection sets up successfully; 0 otherwise
+ */
+ int Connect(const std::string& endpoint);
+ int Send(Msg** msg) override;
+ Msg* Receive() override;
+ void* InternalID() const override;
};
</pre></div></div></div>
<div class="section">
<h4><a name="Router_Socket"></a>Router Socket</h4>
-<p>The Router socket inherits from the base Socket. One Router socket connects to at least one Dealer socket.</p>
+<p>The Router socket inherits from the base Socket. One Router socket connects to at least one Dealer socket. Upon receiving a message, the router forwards it to the appropriate dealer according to the receiver’s ID of this message.</p>
<div class="source">
-<div class="source"><pre class="prettyprint">class Router : public Socket{
- /**
- * Blocking operation to setup the connection with dealers.
- * It automatically binds to the endpoint for intra-process communication,
- * i.e., "inproc://router".
- *
- * @param endpoint the identifier for the Dealer socket in other process
- * to connect. It has the format IP:Port, where IP is the host machine.
- * If endpoint is empty, it means that all connections are
- * intra-process connection.
- * @param expected_connections total number of connections. This function
- * exits after receiving this number of connections from dealers or after
- * a timeout (1 minutes).
- * @return number of connected dealers.
- */
- int Bind(string endpoint, int expected_connections);
- virtual int Send(Msg** msg, int dst=-1);
- virtual Message* Receive();
+<div class="source"><pre class="prettyprint">class Router : public SocketInterface {
+ public:
+ Router();
+ /**
+ * There is only one router per procs, hence its local id is 0 and is not set
+ * explicitly.
+ *
+ * @param bufsize Buffer at most this number of messages
+ */
+ explicit Router(int bufsize);
+ ~Router() override;
+ /**
+ * Setup the connection with dealers.
+ *
+ * It automatically binds to the endpoint for intra-process communication,
+ * i.e., "inproc://router".
+ *
+ * @param endpoint The identifier for the Dealer socket in other process
+ * to connect. It has the format IP:Port, where IP is the host machine.
+ * If endpoint is empty, it means that all connections are
+ * intra-process connection.
+ * @return number of connected dealers.
+ */
+ int Bind(const std::string& endpoint);
+ /**
+ * If the destination socket has not connected yet, buffer this the message.
+ */
+ int Send(Msg** msg) override;
+ Msg* Receive() override;
+ void* InternalID() const override;
+
};
</pre></div></div></div></div>
<div class="section">
@@ -595,7 +775,7 @@ class SafeQueue{
bool empty();
};
</pre></div></div>
-<p>For inter-process communication, we serialize the message and call MPI’s send/receive functions to transferring them. All inter-process connections are setup by MPI at the beginning. Consequently, the Connect and Bind functions do nothing for both inter-process and intra-process communication.</p>
+<p>For inter-process communication, we serialize the message and call MPI’s send/receive functions to transfer them. All inter-process connections are setup by MPI at the beginning. Consequently, the Connect and Bind functions do nothing for both inter-process and intra-process communication.</p>
<p>MPI’s AllReduce function is efficient for data aggregation in distributed training. For example, <a class="externalLink" href="http://arxiv.org/abs/1501.02876">DeepImage of Baidu</a> uses AllReduce to aggregate the updates of parameter from all workers. It has similar architecture as <a href="architecture.html">Fig.2</a>, where every process has a server group and is connected with all other processes. Hence, we can implement DeepImage in Singa by simply using MPI’s AllReduce function for inter-process communication.</p>
<!-- #### Server socket
Modified: websites/staging/singa/trunk/content/docs/data.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/data.html (original)
+++ websites/staging/singa/trunk/content/docs/data.html Sat Aug 15 10:56:46 2015
@@ -1,15 +1,15 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – Data Preparation</title>
+ <title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -183,7 +183,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active ">Data Preparation</li>
+ <li class="active "></li>
@@ -401,120 +401,13 @@
<div id="bodyColumn" class="span10" >
- <div class="section">
-<h2><a name="Data_Preparation"></a>Data Preparation</h2>
-<p>To submit a training job, users need to convert raw data (e.g., images, text documents) into records that can be recognized by SINGA. SINGA uses a DataLayer to load these records into memory and uses ParserLayer to parse features (e.g., image pixels and labels) from these records. The records could be organized and stored using many different ways, e.g., a file, a light database, or HDFS, as long as there is a corresponding DataLayer that can load the records.</p>
-<div class="section">
-<h3><a name="DataShard"></a>DataShard</h3>
-<p>To create shard for your own data, users may need to implement or modify the following files</p>
-
-<ul>
-
-<li>common.proto</li>
-
-<li>create_shard.cc</li>
-
-<li>Makefile</li>
-</ul>
-<p><b>1. Define record</b></p>
-<p>Record class is inherited from Message class whose format follows Google protocol buffers. Please refer to the <a class="externalLink" href="https://developers.google.com/protocol-buffers/docs/cpptutorial">Tutorial</a>. </p>
-<p>Your record will be defined in a file, SINGAfolder/src/proto/common.proto</p>
-<p>(a) Define the record</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">message UserRecord {
- repeated int userVAR1 = 1; // unique id
- optional string userVAR2 = 2; // unique id
- ...
-}
-</pre></div></div>
-<p>(b) Declare user own record in Record</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">message Record {
- optional UserRecord user_record = 1; // unique id
- ...
-}
-</pre></div></div>
-<p>(c) Compile SINGA</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">cd SINGAfolder
-./configure
-make
-</pre></div></div>
-<p><b>2. Create shard</b></p>
-<p>(a) Create a folder for dataset, e.g., we call it “USERDATAfolder”.</p>
-<p>(b) Source files for creating shard will be in SINGAfolder/USERDATAfolder/</p>
-
-<ul>
-
-<li>For example of RNNLM, create_shard.cc is in SINGAfolder/examples/rnnlm</li>
-</ul>
-<p>(c) Create shard</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">singa::DataShard myShard( outputpath, mode);
-</pre></div></div>
-
-<ul>
-
-<li><tt>string outputpath</tt>, where user wants to create shard.</li>
-
-<li><tt>int mode := kRead | kCreate | kAppend</tt>, is defined in SINGAfolder/include/utils/data_shard.h</li>
-</ul>
-<p><b>3. Store record into shard</b></p>
-<p>(a) xxx</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">singa::Record record;
-singa::UserRecord *myRecord = record.mutable_user_record();
-</pre></div></div>
-<p><tt>mutable_user_record()</tt> method is automatically generated after compiling SINGA at Step 1-(c).</p>
-<p>(b) Set/Add values into the record</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">myRecord->add_userVAR1( int_val );
-myRecord->set_userVAR2( string_val );
-</pre></div></div>
-<p>(c) Store the record to shard</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">myShard.Insert( key, myRecord );
-</pre></div></div>
-
-<ul>
-
-<li><tt>String key</tt>, will be a unique id for a message</li>
-</ul>
-<p><b>Example of RNNLM</b></p>
-<p>You can refer to RNNLM example at SINGAfolder/example/rnnlm/</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">message SingleWordRecord {
- optional string word = 1;
- optional int32 word_index = 2;
- optional int32 class_index =3;`
-}
-
-message Record {
- optional SingleWordRecord word_record = 4;
-}
-
-make download
-to download raw data from https://www.rnnlm.org
-</pre></div></div>
-<p>In this example, rnnlm-0.4b is used.</p>
-
-<div class="source">
-<div class="source"><pre class="prettyprint">make create
-</pre></div></div>
-<p>to process input text file, create records, and store it into shard</p>
-<p>We create 3 shards for training data, which are class_shard, vocab_shard, word_shard.</p></div>
-<div class="section">
-<h3><a name="LMDB"></a>LMDB</h3></div>
-<div class="section">
-<h3><a name="HDFS"></a>HDFS</h3></div></div>
+ <p>— trunk/content/markdown/docs/data.md (revision 1693059) +++ trunk/content/markdown/docs/data.md (working copy) @@ -1,18 +1,125 @@ ## Data Preparation</p>
+<p>-To submit a training job, users need to convert raw data (e.g., images, text -documents) into records that can be recognized by SINGA. SINGA uses a DataLayer -to load these records into memory and uses ParserLayer to parse features (e.g., -image pixels and labels) from these records. The records could be organized and -stored using many different ways, e.g., using a light database, or a file or -HDFS, as long as there is a corresponding DataLayer that can load the records. +To submit a training job, users need to convert raw data (e.g., images, text documents) into records that can be recognized by SINGA. SINGA uses a DataLayer +to load these records into memory and uses ParserLayer to parse features (e.g., image pixels and labels) from these records. The records could be organized and +stored using many different ways, e.g., a file, a light database, or HDFS, as long as there is a corresponding DataLayer that can load the records.</p>
+<p>### DataShard</p>
+<p>+To create shard for your own data, users may need to implement or modify the following files</p>
+<p>+- user.proto +- create_shard.cc +- Makefile</p>
+<p>+<b>1. Define record</b> + +Record class is inherited from Message class whose format follows Google protocol buffers. Please refer to the [Tutorial][1]. + +Your record will be defined in a file, “user.proto”, as an extension of Record written in “SINGA_ROOT/src/proto/common.proto”. + +(a) Declare the user-defined record as an extension of Record. Please refer to the [Tutorial][2] for Extension. + + package singa;<br />+ + import “common.proto”; // required to import common.proto + + extend Record { + optional UserRecord1 user_record1 = 100; // unique extension field id, reserved for users (e.g., 100-199) + optional UserRecord2 user_record2 = 101; // unique extension field id, reserved for users (e.g., 100-199) + } + +(b) Define the user own record + + message UserRecord1 { + repeated int userVAR1 = 1; // unique field id + optional string userVAR2 = 2; // unique field id + … + } + + message UserRecord2 { + … + } + +<i>Note</
i> + +There is an alternative way to define the proto extension. + + message UserRecord { + extend Record { + optional UserRecord user_record = 100; // unique extension field id, reserved for users (e.g., 100-199) + } + repeated int userVAR1 = 1; // unique field id + optional string userVAR2 = 2; // unique field id + … + } + +In this way, you should be careful of the scope of fields and how to access the fields, which are different from the above. + +<b>2. Compile proto/Singa</b> + +Once you defined protos, you need to compile them as follow. + + protoc -I=IMPORT_DIR –cpp_out=DST_DIR [proto1] [proto2] … + + - IMPORT_DIR: path to contain proto files + - DST_DIR: path to generate C++ codes (.pb.h and .pb.cc) + +Assume that users will implement their own [Layers][3] as well. The following commands help you to compile Singa sources as well as proto. + + cd SINGA_ROOT + ./configure + make + +<b>3. Create shard</b> + +(a) Create a folder for dataset and shard, e.g., w
e call it “USER_DATA”. (This folder would be the same as WORKSPACE.) + +(b) Source files, e.g., “create_shard.cc”, will be in “SINGA_ROOT/USER_DATA/” + +DataShard object + + singa::DataShard myShard( outputpath, mode ); + + - <tt>string outputpath</tt>, where user wants to create shard. + - <tt>int mode := kRead | kCreate | kAppend</tt>, is defined in “SINGA_ROOT/include/utils/data_shard.h” + +Record object + + singa::Record record; + + singa::UserRecord <i>myRecord = record.MutableExtension(singa::user_record); + + - <tt>MutableExtension()</tt> is templated method, generated after compile at Step 2 and processed at runtime. + +(c) Set/Add values into your record + + myRecord->add_userVAR1( int_val ); // for repeated field + myRecord->set_userVAR2( string_val ); + +(d) Write the record to shard + + myShard.Insert( key, myRecord ); +- <tt>String key</tt>, will be a unique id for a message + +</i><i>Note: Preparing Makefile
</i>* + +There are some notes when you write your own Makefile + +- Specify <tt>outputpath</tt> of output directory for shard. + +- Make sure you copy “user.proto” into “SINGA_ROOT/src/proto”. + +Examples of creating shard can be found at [RNNLM][4]. + ### LMDB</p>
+<p>### HDFS + + + [1]: <a class="externalLink" href="https://developers.google.com/protocol-buffers/docs/cpptutorial">https://developers.google.com/protocol-buffers/docs/cpptutorial</a> + [2]: <a class="externalLink" href="https://developers.google.com/protocol-buffers/docs/reference/cpp-generated?hl=en#extension">https://developers.google.com/protocol-buffers/docs/reference/cpp-generated?hl=en#extension</a> + [3]: <a class="externalLink" href="http://singa.incubator.apache.org/docs/layer.html">http://singa.incubator.apache.org/docs/layer.html</a> + [4]: <a class="externalLink" href="http://singa.incubator.apache.org/docs/examples.html">http://singa.incubator.apache.org/docs/examples.html</a></p>
</div>
</div>
</div>
Modified: websites/staging/singa/trunk/content/docs/datashard.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/datashard.html (original)
+++ websites/staging/singa/trunk/content/docs/datashard.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/debug.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/debug.html (original)
+++ websites/staging/singa/trunk/content/docs/debug.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – How to Debug</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/examples.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/examples.html (original)
+++ websites/staging/singa/trunk/content/docs/examples.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Example Models</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/hdfs.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/hdfs.html (original)
+++ websites/staging/singa/trunk/content/docs/hdfs.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/installation.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/installation.html (original)
+++ websites/staging/singa/trunk/content/docs/installation.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Installation</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/layer.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/layer.html (original)
+++ websites/staging/singa/trunk/content/docs/layer.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Layers Instruction</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/lmdb.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/lmdb.html (original)
+++ websites/staging/singa/trunk/content/docs/lmdb.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/mlp.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/mlp.html (original)
+++ websites/staging/singa/trunk/content/docs/mlp.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/model-config.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/model-config.html (original)
+++ websites/staging/singa/trunk/content/docs/model-config.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Model Configuration</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/neuralnet-partition.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/neuralnet-partition.html (original)
+++ websites/staging/singa/trunk/content/docs/neuralnet-partition.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Neural Network Partition</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/programmer-guide.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/programmer-guide.html (original)
+++ websites/staging/singa/trunk/content/docs/programmer-guide.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Programmer Guide</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/rbm.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/rbm.html (original)
+++ websites/staging/singa/trunk/content/docs/rbm.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – </title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/docs/rnn.html
==============================================================================
--- websites/staging/singa/trunk/content/docs/rnn.html (original)
+++ websites/staging/singa/trunk/content/docs/rnn.html Sat Aug 15 10:56:46 2015
@@ -1,15 +1,15 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
- <title>Apache SINGA – </title>
+ <title>Apache SINGA – Recurrent neural networks (RNN)</title>
<link rel="stylesheet" href="../css/apache-maven-fluido-1.4.min.css" />
<link rel="stylesheet" href="../css/site.css" />
<link rel="stylesheet" href="../css/print.css" media="print" />
@@ -183,7 +183,7 @@
Apache SINGA</a>
<span class="divider">/</span>
</li>
- <li class="active "></li>
+ <li class="active ">Recurrent neural networks (RNN)</li>
@@ -403,7 +403,118 @@
<div id="bodyColumn" class="span10" >
-
+ <div class="section">
+<h2><a name="Recurrent_neural_networks_RNN"></a>Recurrent neural networks (RNN)</h2>
+<p>Example files for RNN can be found in “SINGA_ROOT/examples/rnnlm”, which we assume to be WORKSPACE.</p>
+<div class="section">
+<h3><a name="Create_DataShard"></a>Create DataShard</h3>
+<p>(a) Define your own record. Please refer to <a class="externalLink" href="http://singa.incubator.apache.org/docs/data.html">Data Preparation</a> for details.</p>
+<p>Records for RNN example are defined in “user.proto” as an extension.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">package singa;
+
+import "common.proto"; // Record message for SINGA is defined
+import "job.proto"; // Layer message for SINGA is defined
+
+extend Record {
+ optional WordClassRecord wordclass = 101;
+ optional SingleWordRecord singleword = 102;
+}
+
+message WordClassRecord {
+ optional int32 class_index = 1; // the index of this class
+ optional int32 start = 2; // the index of the start word in this class;
+ optional int32 end = 3; // the index of the end word in this class
+}
+
+message SingleWordRecord {
+ optional string word = 1;
+ optional int32 word_index = 2; // the index of this word in the vocabulary
+ optional int32 class_index = 3; // the index of the class corresponding to this word
+}
+</pre></div></div>
+<p>(b) Download raw data</p>
+<p>This example downloads rnnlm-0.4b from <a href="www.rnnlm.org">www.rnnlm.org</a> by a command </p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">make download
+</pre></div></div>
+<p>The raw data is stored in a folder “rnnlm-0.4b/train” and “rnnlm-0.4b/test”.</p>
+<p>(c) Create data shard for training and testing</p>
+<p>Data shards (e.g., “shard.dat”) will be created in “rnnlm_class_shard”, “rnnlm_vocab_shard”, “rnnlm_word_shard_train” and “rnnlm_word_shard_test” by a command</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">make create
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Define_Layers"></a>Define Layers</h3>
+<p>Similar to records, layers are also defined in “user.proto” as an extension.</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">package singa;
+
+import "common.proto"; // Record message for SINGA is defined
+import "job.proto"; // Layer message for SINGA is defined
+
+//For implementation of RNNLM application
+extend LayerProto {
+ optional RnnlmComputationProto rnnlmcomputation_conf = 201;
+ optional RnnlmSigmoidProto rnnlmsigmoid_conf = 202;
+ optional RnnlmInnerproductProto rnnlminnerproduct_conf = 203;
+ optional RnnlmWordinputProto rnnlmwordinput_conf = 204;
+ optional RnnlmDataProto rnnlmdata_conf = 207;
+}
+
+// 1-Message that stores parameters used by RnnlmComputationLayer
+message RnnlmComputationProto {
+ optional bool bias_term = 1 [default = true]; // use bias vector or not
+}
+
+// 2-Message that stores parameters used by RnnlmSigmoidLayer
+message RnnlmSigmoidProto {
+ optional bool bias_term = 1 [default = true]; // use bias vector or not
+}
+
+// 3-Message that stores parameters used by RnnlmInnerproductLayer
+message RnnlmInnerproductProto {
+ required int32 num_output = 1; // number of outputs for the layer
+ optional bool bias_term = 30 [default = true]; // use bias vector or not
+}
+
+// 4-Message that stores parameters used by RnnlmWordinputLayer
+message RnnlmWordinputProto {
+ required int32 word_length = 1; // vector length for each input word
+ optional bool bias_term = 30 [default = true]; // use bias vector or not
+}
+
+// 5-Message that stores parameters used by RnnlmWordparserLayer - nothing needs to be configured
+//message RnnlmWordparserProto {
+//}
+
+// 6-Message that stores parameters used by RnnlmClassparserLayer - nothing needs to be configured
+//message RnnlmClassparserProto {
+//}
+
+// 7-Message that stores parameters used by RnnlmDataLayer
+message RnnlmDataProto {
+ required string class_path = 1; // path to the data file/folder, absolute or relative to the workspace
+ required string word_path = 2;
+ required int32 window_size = 3; // window size.
+}
+</pre></div></div></div>
+<div class="section">
+<h3><a name="Configure_Job"></a>Configure Job</h3>
+<p>Job configuration is written in “job.conf”.</p>
+<p>Note: Extended field names should be embraced with square-parenthesis [], e.g., [singa.rnnlmdata_conf].</p></div>
+<div class="section">
+<h3><a name="Run_Training"></a>Run Training</h3>
+<p>Start training by the following commands</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">cd SINGA_ROOT
+./bin/singa-run.sh -workspace=examples/rnnlm
+</pre></div></div></div></div>
</div>
</div>
</div>
Modified: websites/staging/singa/trunk/content/index.html
==============================================================================
--- websites/staging/singa/trunk/content/index.html (original)
+++ websites/staging/singa/trunk/content/index.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Welcome to Apache SINGA</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/introduction.html
==============================================================================
--- websites/staging/singa/trunk/content/introduction.html (original)
+++ websites/staging/singa/trunk/content/introduction.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Introduction</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
Modified: websites/staging/singa/trunk/content/quick-start.html
==============================================================================
--- websites/staging/singa/trunk/content/quick-start.html (original)
+++ websites/staging/singa/trunk/content/quick-start.html Sat Aug 15 10:56:46 2015
@@ -1,13 +1,13 @@
<!DOCTYPE html>
<!--
- | Generated by Apache Maven Doxia at 2015-08-07
+ | Generated by Apache Maven Doxia at 2015-08-15
| Rendered using Apache Maven Fluido Skin 1.4
-->
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
- <meta name="Date-Revision-yyyymmdd" content="20150807" />
+ <meta name="Date-Revision-yyyymmdd" content="20150815" />
<meta http-equiv="Content-Language" content="en" />
<title>Apache SINGA – Quick Start</title>
<link rel="stylesheet" href="./css/apache-maven-fluido-1.4.min.css" />
@@ -447,20 +447,33 @@ model configuration file (*model.conf*)
training data shard, test data shard and the mean file. -->
<p>Since all modules used for training this CNN model are provided by SINGA as built-in modules, there is no need to write any code. You just execute the script (<i>../../bin/singa-run.sh</i>) by providing the workspace which includes the job configuration file (<i>job.conf</i>). If you want to implement your own modules, then you have to register your modules in the <a href="user-guide.html">driver program</a>.</p></div>
<div class="section">
+<h4><a name="Start_Zookeeper"></a>Start Zookeeper</h4>
+<p>SINGA uses <a class="externalLink" href="https://zookeeper.apache.org/">zookeeper</a> to coordinate the training. Please make sure the zookeeper service is on before running SINGA.</p>
+<p>If you installed the zookeeper using our thirdparty script, you can simply start it by:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">#goto top level folder
+cd ../..
+./bin/zk-service start
+</pre></div></div>
+<p>Otherwise, if you launched a zookeeper by yourself but not used the default port, please edit the <tt>conf/singa.conf</tt>:</p>
+
+<div class="source">
+<div class="source"><pre class="prettyprint">zookeeper_host: "localhost:YOUR_PORT"
+</pre></div></div></div>
+<div class="section">
<h4><a name="Start_Training"></a>Start Training</h4>
<p>Start the training by running:</p>
<div class="source">
<div class="source"><pre class="prettyprint">#goto top level folder
-cd ../..
./bin/singa-run.sh -workspace=examples/cifar10
</pre></div></div>
<p>Note: we have changed the command line arguments from <tt>-cluster.. -model..</tt> to <tt>-workspace</tt>. The <tt>workspace</tt> folder must have a job.conf file which specifies the cluster (number of workers, number of servers, etc) and model configuration.</p>
<p>Some training information will be shown on the screen like:</p>
<div class="source">
-<div class="source"><pre class="prettyprint">Starting zookeeper ... already running as process 21660.
-Generate host list to SINGA_ROOT/examples/cifar10/job.hosts
+<div class="source"><pre class="prettyprint">Generate host list to SINGA_ROOT/examples/cifar10/job.hosts
Generate job id to SINGA_ROOT/examples/cifar10/job.id [job_id = 1]
Executing : ./singa -workspace=SINGA_ROOT/examples/cifar10 -job=1
proc #0 -> 10.10.10.14:49152 (pid = 26724)
@@ -532,7 +545,7 @@ singa-node3
</pre></div></div></li>
<li>
-<p>The zookeeper location must be configured in conf/singa.conf, e.g.,</p>
+<p>The zookeeper must be on and configured in conf/singa.conf, e.g.,</p>
<p>zookeeper_host: “singa-node1:2181”</p></li>
<li>
@@ -617,26 +630,6 @@ All other settings are the same as runni
./bin/singa-run.sh -model=examples/cifar10/model.conf -cluster=examples/cifar10/cluster.conf
-
-
-#### Training in a cluster
-
-To run the distributed Hogwild framework, configure the cluster.conf as:
-
- nworker_groups: 2
- nserver_groups: 2
-
-and start one process as,
-
- ./bin/singa-run.sh -model=examples/cifar10/model.conf -cluster=examples/cifar10/cluster.conf
-
-and then start another process as,
-
- ./singa -model=examples/cifar10/model.conf -cluster=examples/cifar10/cluster.conf
-
-Note that the two commands are different! The first one will start the zookeeper. Currently we assume
-that the example/cifar10 folder is in NFS.
-
### Run with Mesos
*in working*...