You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by wa...@apache.org on 2015/09/16 06:19:40 UTC
[01/21] incubator-singa git commit: SINGA-67 Add singatest into build
targets Modified Makefile.am to add a target named "singatest"i. Now "make"
will generate 5 files/libs: libsinga.la libgtest.la singa singatool
singatest. User can also specify the tar
Repository: incubator-singa
Updated Branches:
refs/heads/tutorial 7306bcffb -> 5a8a1a316 (forced update)
SINGA-67 Add singatest into build targets
Modified Makefile.am to add a target named "singatest"i.
Now "make" will generate 5 files/libs: libsinga.la libgtest.la singa singatool singatest.
User can also specify the target he wants to generate, for example, "make singa".
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/bc400a74
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/bc400a74
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/bc400a74
Branch: refs/heads/tutorial
Commit: bc400a7422a9671c9d7672b898f4f0ac9f271a77
Parents: 50deedd
Author: xiezl <xi...@comp.nus.edu.sg>
Authored: Sun Sep 6 12:54:14 2015 +0800
Committer: xiezl <xi...@comp.nus.edu.sg>
Committed: Sun Sep 6 12:54:14 2015 +0800
----------------------------------------------------------------------
Makefile.am | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/bc400a74/Makefile.am
----------------------------------------------------------------------
diff --git a/Makefile.am b/Makefile.am
index deca6e6..732b84e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -78,6 +78,14 @@ SINGA_HDRS := include/singa.h \
include/communication/msg.h \
include/communication/socket.h
+GTEST_SRCS := include/gtest/gtest-all.cc
+GTEST_HRDS := include/gtest/gtest.h
+TEST_SRCS := include/gtest/gtest_main.cc \
+ src/test/test_cluster.cc \
+ src/test/test_msg.cc \
+ src/test/test_neuralnet.cc \
+ src/test/test_paramslicer.cc \
+ src/test/test_shard.cc
lib_LTLIBRARIES = libsinga.la
libsinga_la_SOURCES = $(PROTO_HDRS) $(PROTO_SRCS) $(SINGA_HDRS) $(SINGA_SRCS)
@@ -89,6 +97,17 @@ libsinga_la_CXXFLAGS += -DUSE_LMDB
endif
libsinga_la_LDFLAGS = -I./include
+lib_LTLIBRARIES += libgtest.la
+libgtest_la_SOURCES = $(GTEST_HDRS) $(GTEST_SRCS)
+libgtest_la_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -Wno-unknown-pragmas \
+ $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -gdwarf-2 -msse3 \
+ -gstrict-dwarf -Woverloaded-virtual -DTHREADED -fpermissive
+if LMDB
+libgtest_la_CXXFLAGS += -DUSE_LMDB
+endif
+libgtest_la_LDFLAGS = -I./include
+
+
bin_PROGRAMS = singa
singa_SOURCES = src/main.cc
singa_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -MMD -Wno-unknown-pragmas \
@@ -116,6 +135,24 @@ singatool_LDFLAGS = -I./include \
-lprotobuf \
-lzookeeper_mt
+bin_PROGRAMS += singatest
+singatest_SOURCES = $(GTEST_HDRS) $(TEST_SRCS)
+singatest_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -MMD -Wno-unknown-pragmas \
+ $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -DTHREADED
+singatest_LDFLAGS = -I./include \
+ -lsinga \
+ -lglog \
+ -lprotobuf \
+ -lrt \
+ -lopenblas \
+ -lzmq \
+ -lczmq \
+ -lzookeeper_mt \
+ -lgtest
+if LMDB
+singatest_LDFLAGS += -llmdb
+endif
+
clean-local:
rm -rf $(PROTO_SRCS) $(PROTO_HDRS)
rm -rf $(PROTO_PYS)
[12/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/LICENSE.md
----------------------------------------------------------------------
diff --git a/LICENSE.md b/LICENSE.md
deleted file mode 100644
index 5f43a73..0000000
--- a/LICENSE.md
+++ /dev/null
@@ -1,864 +0,0 @@
-Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "{}"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright [yyyy] [name of copyright owner]
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-============================================================================
-The SINGA project contains subcomponents with separate copyright
-notices and license terms. Your use of the source code for the these
-subcomponents is subject to the terms and conditions of the following
-licenses.
-
-============================================================================
-For cafee (see comments in src/neuralnet/layer.cc)
-===========================================================================
-COPYRIGHT
-All contributions by the University of California:
-Copyright (c) 2014, The Regents of the University of California (Regents)
-All rights reserved.
-All other contributions:
-Copyright (c) 2014, the respective contributors
-All rights reserved.
-Caffe uses a shared copyright model: each contributor holds copyright over
-their contributions to Caffe. The project versioning records all such
-contribution and copyright details. If a contributor wants to further mark
-their specific copyright on a particular contribution, they should indicate
-their copyright solely in the commit message of the change when it is
-committed.
-LICENSE
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-1. Redistributions of source code must retain the above copyright notice, this
-list of conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright notice,
-this list of conditions and the following disclaimer in the documentation
-and/or other materials provided with the distribution.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
-ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
-(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
-ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-CONTRIBUTION AGREEMENT
-By contributing to the BVLC/caffe repository through pull-request, comment,
-or otherwise, the contributor releases their content to the
-license and copyright terms herein.
-
-=====================================================================
-For czmq
-=====================================================================
-Mozilla Public License Version 2.0
-==================================
-
-1. Definitions
---------------
-
-1.1. "Contributor"
- means each individual or legal entity that creates, contributes to
- the creation of, or owns Covered Software.
-
-1.2. "Contributor Version"
- means the combination of the Contributions of others (if any) used
- by a Contributor and that particular Contributor's Contribution.
-
-1.3. "Contribution"
- means Covered Software of a particular Contributor.
-
-1.4. "Covered Software"
- means Source Code Form to which the initial Contributor has attached
- the notice in Exhibit A, the Executable Form of such Source Code
- Form, and Modifications of such Source Code Form, in each case
- including portions thereof.
-
-1.5. "Incompatible With Secondary Licenses"
- means
-
- (a) that the initial Contributor has attached the notice described
- in Exhibit B to the Covered Software; or
-
- (b) that the Covered Software was made available under the terms of
- version 1.1 or earlier of the License, but not also under the
- terms of a Secondary License.
-
-1.6. "Executable Form"
- means any form of the work other than Source Code Form.
-
-1.7. "Larger Work"
- means a work that combines Covered Software with other material, in
- a separate file or files, that is not Covered Software.
-
-1.8. "License"
- means this document.
-
-1.9. "Licensable"
- means having the right to grant, to the maximum extent possible,
- whether at the time of the initial grant or subsequently, any and
- all of the rights conveyed by this License.
-
-1.10. "Modifications"
- means any of the following:
-
- (a) any file in Source Code Form that results from an addition to,
- deletion from, or modification of the contents of Covered
- Software; or
-
- (b) any new file in Source Code Form that contains any Covered
- Software.
-
-1.11. "Patent Claims" of a Contributor
- means any patent claim(s), including without limitation, method,
- process, and apparatus claims, in any patent Licensable by such
- Contributor that would be infringed, but for the grant of the
- License, by the making, using, selling, offering for sale, having
- made, import, or transfer of either its Contributions or its
- Contributor Version.
-
-1.12. "Secondary License"
- means either the GNU General Public License, Version 2.0, the GNU
- Lesser General Public License, Version 2.1, the GNU Affero General
- Public License, Version 3.0, or any later versions of those
- licenses.
-
-1.13. "Source Code Form"
- means the form of the work preferred for making modifications.
-
-1.14. "You" (or "Your")
- means an individual or a legal entity exercising rights under this
- License. For legal entities, "You" includes any entity that
- controls, is controlled by, or is under common control with You. For
- purposes of this definition, "control" means (a) the power, direct
- or indirect, to cause the direction or management of such entity,
- whether by contract or otherwise, or (b) ownership of more than
- fifty percent (50%) of the outstanding shares or beneficial
- ownership of such entity.
-
-2. License Grants and Conditions
---------------------------------
-
-2.1. Grants
-
-Each Contributor hereby grants You a world-wide, royalty-free,
-non-exclusive license:
-
-(a) under intellectual property rights (other than patent or trademark)
- Licensable by such Contributor to use, reproduce, make available,
- modify, display, perform, distribute, and otherwise exploit its
- Contributions, either on an unmodified basis, with Modifications, or
- as part of a Larger Work; and
-
-(b) under Patent Claims of such Contributor to make, use, sell, offer
- for sale, have made, import, and otherwise transfer either its
- Contributions or its Contributor Version.
-
-2.2. Effective Date
-
-The licenses granted in Section 2.1 with respect to any Contribution
-become effective for each Contribution on the date the Contributor first
-distributes such Contribution.
-
-2.3. Limitations on Grant Scope
-
-The licenses granted in this Section 2 are the only rights granted under
-this License. No additional rights or licenses will be implied from the
-distribution or licensing of Covered Software under this License.
-Notwithstanding Section 2.1(b) above, no patent license is granted by a
-Contributor:
-
-(a) for any code that a Contributor has removed from Covered Software;
- or
-
-(b) for infringements caused by: (i) Your and any other third party's
- modifications of Covered Software, or (ii) the combination of its
- Contributions with other software (except as part of its Contributor
- Version); or
-
-(c) under Patent Claims infringed by Covered Software in the absence of
- its Contributions.
-
-This License does not grant any rights in the trademarks, service marks,
-or logos of any Contributor (except as may be necessary to comply with
-the notice requirements in Section 3.4).
-
-2.4. Subsequent Licenses
-
-No Contributor makes additional grants as a result of Your choice to
-distribute the Covered Software under a subsequent version of this
-License (see Section 10.2) or under the terms of a Secondary License (if
-permitted under the terms of Section 3.3).
-
-2.5. Representation
-
-Each Contributor represents that the Contributor believes its
-Contributions are its original creation(s) or it has sufficient rights
-to grant the rights to its Contributions conveyed by this License.
-
-2.6. Fair Use
-
-This License is not intended to limit any rights You have under
-applicable copyright doctrines of fair use, fair dealing, or other
-equivalents.
-
-2.7. Conditions
-
-Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
-in Section 2.1.
-
-3. Responsibilities
--------------------
-
-3.1. Distribution of Source Form
-
-All distribution of Covered Software in Source Code Form, including any
-Modifications that You create or to which You contribute, must be under
-the terms of this License. You must inform recipients that the Source
-Code Form of the Covered Software is governed by the terms of this
-License, and how they can obtain a copy of this License. You may not
-attempt to alter or restrict the recipients' rights in the Source Code
-Form.
-
-3.2. Distribution of Executable Form
-
-If You distribute Covered Software in Executable Form then:
-
-(a) such Covered Software must also be made available in Source Code
- Form, as described in Section 3.1, and You must inform recipients of
- the Executable Form how they can obtain a copy of such Source Code
- Form by reasonable means in a timely manner, at a charge no more
- than the cost of distribution to the recipient; and
-
-(b) You may distribute such Executable Form under the terms of this
- License, or sublicense it under different terms, provided that the
- license for the Executable Form does not attempt to limit or alter
- the recipients' rights in the Source Code Form under this License.
-
-3.3. Distribution of a Larger Work
-
-You may create and distribute a Larger Work under terms of Your choice,
-provided that You also comply with the requirements of this License for
-the Covered Software. If the Larger Work is a combination of Covered
-Software with a work governed by one or more Secondary Licenses, and the
-Covered Software is not Incompatible With Secondary Licenses, this
-License permits You to additionally distribute such Covered Software
-under the terms of such Secondary License(s), so that the recipient of
-the Larger Work may, at their option, further distribute the Covered
-Software under the terms of either this License or such Secondary
-License(s).
-
-3.4. Notices
-
-You may not remove or alter the substance of any license notices
-(including copyright notices, patent notices, disclaimers of warranty,
-or limitations of liability) contained within the Source Code Form of
-the Covered Software, except that You may alter any license notices to
-the extent required to remedy known factual inaccuracies.
-
-3.5. Application of Additional Terms
-
-You may choose to offer, and to charge a fee for, warranty, support,
-indemnity or liability obligations to one or more recipients of Covered
-Software. However, You may do so only on Your own behalf, and not on
-behalf of any Contributor. You must make it absolutely clear that any
-such warranty, support, indemnity, or liability obligation is offered by
-You alone, and You hereby agree to indemnify every Contributor for any
-liability incurred by such Contributor as a result of warranty, support,
-indemnity or liability terms You offer. You may include additional
-disclaimers of warranty and limitations of liability specific to any
-jurisdiction.
-
-4. Inability to Comply Due to Statute or Regulation
----------------------------------------------------
-
-If it is impossible for You to comply with any of the terms of this
-License with respect to some or all of the Covered Software due to
-statute, judicial order, or regulation then You must: (a) comply with
-the terms of this License to the maximum extent possible; and (b)
-describe the limitations and the code they affect. Such description must
-be placed in a text file included with all distributions of the Covered
-Software under this License. Except to the extent prohibited by statute
-or regulation, such description must be sufficiently detailed for a
-recipient of ordinary skill to be able to understand it.
-
-5. Termination
---------------
-
-5.1. The rights granted under this License will terminate automatically
-if You fail to comply with any of its terms. However, if You become
-compliant, then the rights granted under this License from a particular
-Contributor are reinstated (a) provisionally, unless and until such
-Contributor explicitly and finally terminates Your grants, and (b) on an
-ongoing basis, if such Contributor fails to notify You of the
-non-compliance by some reasonable means prior to 60 days after You have
-come back into compliance. Moreover, Your grants from a particular
-Contributor are reinstated on an ongoing basis if such Contributor
-notifies You of the non-compliance by some reasonable means, this is the
-first time You have received notice of non-compliance with this License
-from such Contributor, and You become compliant prior to 30 days after
-Your receipt of the notice.
-
-5.2. If You initiate litigation against any entity by asserting a patent
-infringement claim (excluding declaratory judgment actions,
-counter-claims, and cross-claims) alleging that a Contributor Version
-directly or indirectly infringes any patent, then the rights granted to
-You by any and all Contributors for the Covered Software under Section
-2.1 of this License shall terminate.
-
-5.3. In the event of termination under Sections 5.1 or 5.2 above, all
-end user license agreements (excluding distributors and resellers) which
-have been validly granted by You or Your distributors under this License
-prior to termination shall survive termination.
-
-************************************************************************
-* *
-* 6. Disclaimer of Warranty *
-* ------------------------- *
-* *
-* Covered Software is provided under this License on an "as is" *
-* basis, without warranty of any kind, either expressed, implied, or *
-* statutory, including, without limitation, warranties that the *
-* Covered Software is free of defects, merchantable, fit for a *
-* particular purpose or non-infringing. The entire risk as to the *
-* quality and performance of the Covered Software is with You. *
-* Should any Covered Software prove defective in any respect, You *
-* (not any Contributor) assume the cost of any necessary servicing, *
-* repair, or correction. This disclaimer of warranty constitutes an *
-* essential part of this License. No use of any Covered Software is *
-* authorized under this License except under this disclaimer. *
-* *
-************************************************************************
-
-************************************************************************
-* *
-* 7. Limitation of Liability *
-* -------------------------- *
-* *
-* Under no circumstances and under no legal theory, whether tort *
-* (including negligence), contract, or otherwise, shall any *
-* Contributor, or anyone who distributes Covered Software as *
-* permitted above, be liable to You for any direct, indirect, *
-* special, incidental, or consequential damages of any character *
-* including, without limitation, damages for lost profits, loss of *
-* goodwill, work stoppage, computer failure or malfunction, or any *
-* and all other commercial damages or losses, even if such party *
-* shall have been informed of the possibility of such damages. This *
-* limitation of liability shall not apply to liability for death or *
-* personal injury resulting from such party's negligence to the *
-* extent applicable law prohibits such limitation. Some *
-* jurisdictions do not allow the exclusion or limitation of *
-* incidental or consequential damages, so this exclusion and *
-* limitation may not apply to You. *
-* *
-************************************************************************
-
-8. Litigation
--------------
-
-Any litigation relating to this License may be brought only in the
-courts of a jurisdiction where the defendant maintains its principal
-place of business and such litigation shall be governed by laws of that
-jurisdiction, without reference to its conflict-of-law provisions.
-Nothing in this Section shall prevent a party's ability to bring
-cross-claims or counter-claims.
-
-9. Miscellaneous
-----------------
-
-This License represents the complete agreement concerning the subject
-matter hereof. If any provision of this License is held to be
-unenforceable, such provision shall be reformed only to the extent
-necessary to make it enforceable. Any law or regulation which provides
-that the language of a contract shall be construed against the drafter
-shall not be used to construe this License against a Contributor.
-
-10. Versions of the License
----------------------------
-
-10.1. New Versions
-
-Mozilla Foundation is the license steward. Except as provided in Section
-10.3, no one other than the license steward has the right to modify or
-publish new versions of this License. Each version will be given a
-distinguishing version number.
-
-10.2. Effect of New Versions
-
-You may distribute the Covered Software under the terms of the version
-of the License under which You originally received the Covered Software,
-or under the terms of any subsequent version published by the license
-steward.
-
-10.3. Modified Versions
-
-If you create software not governed by this License, and you want to
-create a new license for such software, you may create and use a
-modified version of this License if you rename the license and remove
-any references to the name of the license steward (except to note that
-such modified license differs from this License).
-
-10.4. Distributing Source Code Form that is Incompatible With Secondary
-Licenses
-
-If You choose to distribute Source Code Form that is Incompatible With
-Secondary Licenses under the terms of this version of the License, the
-notice described in Exhibit B of this License must be attached.
-
-Exhibit A - Source Code Form License Notice
--------------------------------------------
-
- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/.
-
-If it is not possible or desirable to put the notice in a particular
-file, then You may include the notice in a location (such as a LICENSE
-file in a relevant directory) where a recipient would be likely to look
-for such a notice.
-
-You may add additional accurate notices of copyright ownership.
-
-Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------
-
- This Source Code Form is "Incompatible With Secondary Licenses", as
- defined by the Mozilla Public License, v. 2.0.
-
-======================================================================
-For gflags:
-======================================================================
-Copyright (c) 2006, Google Inc.
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above
- copyright notice, this list of conditions and the following disclaimer
- in the documentation and/or other materials provided with the
- distribution.
- * Neither the name of Google Inc. nor the names of its
- contributors may be used to endorse or promote products derived from this
- software without specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- POSSIBILITY OF SUCH DAMAGE.
-
-======================================================================
-For glog:
-======================================================================
-Copyright (c) 2008, Google Inc.
-All rights reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-* Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-* Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
-* Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-A function gettimeofday in utilities.cc is based on
-http://www.google.com/codesearch/p?hl=en#dR3YEbitojA/COPYING&q=GetSystemTimeAsFileTime%20license:bsd
-The license of this code is:
-Copyright (c) 2003-2008, Jouni Malinen <j...@w1.fi> and contributors
-All Rights Reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-1. Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright
-notice, this list of conditions and the following disclaimer in the
-documentation and/or other materials provided with the distribution.
-3. Neither the name(s) of the above-listed copyright holder(s) nor the
-names of its contributors may be used to endorse or promote products
-derived from this software without specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
-=====================================================================
-For lmdb
-=====================================================================
-The OpenLDAP Public License
- Version 2.8, 17 August 2003
-
- Redistribution and use of this software and associated documentation
- ("Software"), with or without modification, are permitted provided
- that the following conditions are met:
-
- 1. Redistributions in source form must retain copyright statements
- and notices,
-
- 2. Redistributions in binary form must reproduce applicable copyright
- statements and notices, this list of conditions, and the following disclaimer
- in the documentation and/or other materials provided with the distribution,
- and
-
- 3. Redistributions must contain a verbatim copy of this document.
-
- The OpenLDAP Foundation may revise this license from time to
- time.
- Each revision is distinguished by a version number. You may use
- this Software under terms of this license revision or under the
- terms of any subsequent revision of the license.
-
- THIS SOFTWARE IS PROVIDED BY THE OPENLDAP FOUNDATION AND ITS
- CONTRIBUTORS ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES,
- INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
- MERCHANTABILITY
- AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
- SHALL THE OPENLDAP FOUNDATION, ITS CONTRIBUTORS, OR THE AUTHOR(S)
- OR OWNER(S) OF THE SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT,
- INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- (INCLUDING,
- BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
- CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- STRICT
- LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
- ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- POSSIBILITY OF SUCH DAMAGE.
-
- The names of the authors and copyright holders must not be used
- in
- advertising or otherwise to promote the sale, use or other
- dealing
- in this Software without specific, written prior permission.
- Title
- to copyright in this Software shall at all times remain with
- copyright
- holders.
-
- OpenLDAP is a registered trademark of the OpenLDAP Foundation.
-
- Copyright 1999-2003 The OpenLDAP Foundation, Redwood City,
- California, USA. All Rights Reserved. Permission to copy and
- distribute verbatim copies of this document is granted.
-
-======================================================================
-For protobuf:
-=====================================================================
-Copyright 2008, Google Inc.
-All rights reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-* Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-* Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
-* Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-Code generated by the Protocol Buffer compiler is owned by the owner
-of the input file used when generating it. This code is not
-standalone and requires a support library to be linked with it. This
-support library is itself covered by the above license.
-
-========================================================================
-For OpenCV used in preprocessing images
-=======================================================================
-License Agreement
-For Open Source Computer Vision Library
-(3-clause BSD License)
-
-Redistribution and use in source and binary forms, with or without modification,
-are permitted provided that the following conditions are met:
-Redistributions of source code must retain the above copyright notice,
-this list of conditions and the following disclaimer.
-Redistributions in binary form must reproduce the above copyright notice,
-this list of conditions and the following disclaimer in the documentation
-and/or other materials provided with the distribution.
-Neither the names of the copyright holders nor the names of the contributors
-may be used to endorse or promote products derived from this software without
-specific prior written permission.
-
-This software is provided by the copyright holders and contributors “as is”
-and any express or implied warranties, including, but not limited to, the
-implied warranties of merchantability and fitness for a particular purpose are
-disclaimed. In no event shall copyright holders or contributors be liable for
-any direct, indirect, incidental, special, exemplary, or consequential damages
-(including, but not limited to, procurement of substitute goods or services;
-loss of use, data, or profits; or business interruption) however caused and on
-any theory of liability, whether in contract, strict liability, or tort
-(including negligence or otherwise) arising in any way out of the use of this
-software, even if advised of the possibility of such damage.
-
-
-========================================================================
-For Openblas
-=======================================================================
-Copyright (c) 2011-2014, The OpenBLAS Project
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- 1. Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
-
- 2. Redistributions in binary form must reproduce the above copyright notice,
- this list of conditions and the following disclaimer in the documentation
- and/or other materials provided with the distribution.
- 3. Neither the name of the OpenBLAS project nor the names of its contributors
- may be used to endorse or promote products derived from this software without
- specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- POSSIBILITY OF SUCH DAMAGE.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/Makefile.example
----------------------------------------------------------------------
diff --git a/Makefile.example b/Makefile.example
index 2c9d57d..76b8c0d 100644
--- a/Makefile.example
+++ b/Makefile.example
@@ -1,3 +1,23 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
###################User Config Varaibles #############################
# third-party library installation folder
HOME_DIR := /usr
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/NOTICE
----------------------------------------------------------------------
diff --git a/NOTICE b/NOTICE
new file mode 100644
index 0000000..67a6d78
--- /dev/null
+++ b/NOTICE
@@ -0,0 +1,12 @@
+Apache SINGA
+Copyright 2015 The Apache Software Foundation
+
+This product includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+Portions of this software were developed at the National University of
+Singapore.
+
+The software uses thridpart libaries including czmq, glog, google protobuf, OpenCV and Openblas.
+This software also contains code derived from the caffe project (https://github.com/BVLC/caffe)
+and tinydir from Cong Xu.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index f1f490b..513a1a9 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,4 @@
+TODO
##Apache SINGA
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/autogen.sh
----------------------------------------------------------------------
diff --git a/autogen.sh b/autogen.sh
index 2ab62ac..1960c15 100755
--- a/autogen.sh
+++ b/autogen.sh
@@ -1,3 +1,22 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
#!/bin/sh
mkdir -p ./config;
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/configure.ac
----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index fa54546..bc11c34 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1,3 +1,22 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
# -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/examples/cifar10/Makefile.example
----------------------------------------------------------------------
diff --git a/examples/cifar10/Makefile.example b/examples/cifar10/Makefile.example
index 744b256..9e65a58 100644
--- a/examples/cifar10/Makefile.example
+++ b/examples/cifar10/Makefile.example
@@ -1,3 +1,23 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
libs :=singa glog protobuf
.PHONY: all download create
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/examples/cifar10/create_shard.cc
----------------------------------------------------------------------
diff --git a/examples/cifar10/create_shard.cc b/examples/cifar10/create_shard.cc
index 5b8c245..0a00639 100644
--- a/examples/cifar10/create_shard.cc
+++ b/examples/cifar10/create_shard.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
//
// This code creates training and test DataShard for CIFAR dataset.
// It is adapted from the convert_cifar_data from Caffe
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/examples/cifar10/run.sh
----------------------------------------------------------------------
diff --git a/examples/cifar10/run.sh b/examples/cifar10/run.sh
index eb64047..53c81c1 100755
--- a/examples/cifar10/run.sh
+++ b/examples/cifar10/run.sh
@@ -1,3 +1,23 @@
+#
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
#!/bin/bash
if [ $# -ne 2 ];then
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/examples/mnist/Makefile.example
----------------------------------------------------------------------
diff --git a/examples/mnist/Makefile.example b/examples/mnist/Makefile.example
index 9016887..4df4edd 100644
--- a/examples/mnist/Makefile.example
+++ b/examples/mnist/Makefile.example
@@ -1,3 +1,24 @@
+#
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
libs :=singa glog protobuf
.PHONY: all download create
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/examples/mnist/create_shard.cc
----------------------------------------------------------------------
diff --git a/examples/mnist/create_shard.cc b/examples/mnist/create_shard.cc
index 635f901..3d7bd97 100644
--- a/examples/mnist/create_shard.cc
+++ b/examples/mnist/create_shard.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
//
// This code creates DataShard for MNIST dataset.
// It is adapted from the convert_mnist_data from Caffe
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/communication/msg.h
----------------------------------------------------------------------
diff --git a/include/communication/msg.h b/include/communication/msg.h
index 75c91a7..217d89a 100644
--- a/include/communication/msg.h
+++ b/include/communication/msg.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_COMMUNICATION_MSG_H_
#define SINGA_COMMUNICATION_MSG_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/communication/socket.h
----------------------------------------------------------------------
diff --git a/include/communication/socket.h b/include/communication/socket.h
index d76a163..3590577 100644
--- a/include/communication/socket.h
+++ b/include/communication/socket.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_COMMUNICATION_SOCKET_H_
#define SINGA_COMMUNICATION_SOCKET_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/driver.h
----------------------------------------------------------------------
diff --git a/include/driver.h b/include/driver.h
index b16cef3..7d15c98 100644
--- a/include/driver.h
+++ b/include/driver.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_DRIVER_H_
#define SINGA_DRIVER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/connection_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/connection_layer.h b/include/neuralnet/connection_layer.h
index 233714d..17de22d 100644
--- a/include/neuralnet/connection_layer.h
+++ b/include/neuralnet/connection_layer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_CONNECTION_LAYER_H_
#define SINGA_NEURALNET_CONNECTION_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/input_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/input_layer.h b/include/neuralnet/input_layer.h
index 67af2eb..42f0468 100644
--- a/include/neuralnet/input_layer.h
+++ b/include/neuralnet/input_layer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_INPUT_LAYER_H_
#define SINGA_NEURALNET_INPUT_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/layer.h b/include/neuralnet/layer.h
index 4f153d3..6c99cbd 100644
--- a/include/neuralnet/layer.h
+++ b/include/neuralnet/layer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_LAYER_H_
#define SINGA_NEURALNET_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/loss_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/loss_layer.h b/include/neuralnet/loss_layer.h
index c9f6681..3af0b46 100644
--- a/include/neuralnet/loss_layer.h
+++ b/include/neuralnet/loss_layer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_LOSS_LAYER_H_
#define SINGA_NEURALNET_LOSS_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/neuralnet.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/neuralnet.h b/include/neuralnet/neuralnet.h
index bcd102c..f3b091a 100644
--- a/include/neuralnet/neuralnet.h
+++ b/include/neuralnet/neuralnet.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_NEURALNET_H_
#define SINGA_NEURALNET_NEURALNET_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/neuron_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/neuron_layer.h b/include/neuralnet/neuron_layer.h
index 86b55a3..6c4647d 100644
--- a/include/neuralnet/neuron_layer.h
+++ b/include/neuralnet/neuron_layer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_NEURON_LAYER_H_
#define SINGA_NEURALNET_NEURON_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/neuralnet/output_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/output_layer.h b/include/neuralnet/output_layer.h
index ac83d00..d48d805 100644
--- a/include/neuralnet/output_layer.h
+++ b/include/neuralnet/output_layer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_NEURALNET_OUTPUT_LAYER_H_
#define SINGA_NEURALNET_OUTPUT_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/singa.h
----------------------------------------------------------------------
diff --git a/include/singa.h b/include/singa.h
index a98af2b..d4ee557 100644
--- a/include/singa.h
+++ b/include/singa.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_SINGA_H_
#define SINGA_SINGA_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/trainer/server.h
----------------------------------------------------------------------
diff --git a/include/trainer/server.h b/include/trainer/server.h
index 869d10a..3f1c12d 100644
--- a/include/trainer/server.h
+++ b/include/trainer/server.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef INCLUDE_TRAINER_SERVER_H_
#define INCLUDE_TRAINER_SERVER_H_
#include <memory>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/trainer/trainer.h
----------------------------------------------------------------------
diff --git a/include/trainer/trainer.h b/include/trainer/trainer.h
index ed50705..6630e51 100644
--- a/include/trainer/trainer.h
+++ b/include/trainer/trainer.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef INCLUDE_TRAINER_TRAINER_H_
#define INCLUDE_TRAINER_TRAINER_H_
#include <unordered_map>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/trainer/worker.h
----------------------------------------------------------------------
diff --git a/include/trainer/worker.h b/include/trainer/worker.h
index 35ce77e..0607385 100644
--- a/include/trainer/worker.h
+++ b/include/trainer/worker.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_TRAINER_WORKER_H_
#define SINGA_TRAINER_WORKER_H_
#include "neuralnet/neuralnet.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/blob.h
----------------------------------------------------------------------
diff --git a/include/utils/blob.h b/include/utils/blob.h
index 8769e34..91db095 100644
--- a/include/utils/blob.h
+++ b/include/utils/blob.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
/**
* The code is adapted from that of Caffe whose license is attached.
*
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/cluster.h
----------------------------------------------------------------------
diff --git a/include/utils/cluster.h b/include/utils/cluster.h
index 73474af..afeb947 100644
--- a/include/utils/cluster.h
+++ b/include/utils/cluster.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_CLUSTER_H_
#define SINGA_UTILS_CLUSTER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/cluster_rt.h
----------------------------------------------------------------------
diff --git a/include/utils/cluster_rt.h b/include/utils/cluster_rt.h
index 49f6e4d..c78bf75 100644
--- a/include/utils/cluster_rt.h
+++ b/include/utils/cluster_rt.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_CLUSTER_RT_H_
#define SINGA_UTILS_CLUSTER_RT_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/common.h
----------------------------------------------------------------------
diff --git a/include/utils/common.h b/include/utils/common.h
index 599424d..2be2715 100644
--- a/include/utils/common.h
+++ b/include/utils/common.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_COMMON_H_
#define SINGA_UTILS_COMMON_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/data_shard.h
----------------------------------------------------------------------
diff --git a/include/utils/data_shard.h b/include/utils/data_shard.h
index 7e86da1..7d69ae5 100644
--- a/include/utils/data_shard.h
+++ b/include/utils/data_shard.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_DATA_SHARD_H_
#define SINGA_UTILS_DATA_SHARD_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/factory.h
----------------------------------------------------------------------
diff --git a/include/utils/factory.h b/include/utils/factory.h
index e0cd6e3..3af25f0 100644
--- a/include/utils/factory.h
+++ b/include/utils/factory.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_FACTORY_H_
#define SINGA_UTILS_FACTORY_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/graph.h
----------------------------------------------------------------------
diff --git a/include/utils/graph.h b/include/utils/graph.h
index d2a5d38..bad7b19 100644
--- a/include/utils/graph.h
+++ b/include/utils/graph.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_GRAPH_H_
#define SINGA_UTILS_GRAPH_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/param.h
----------------------------------------------------------------------
diff --git a/include/utils/param.h b/include/utils/param.h
index 4909026..e6c8c7c 100644
--- a/include/utils/param.h
+++ b/include/utils/param.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_PARAM_H_
#define SINGA_UTILS_PARAM_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/singleton.h
----------------------------------------------------------------------
diff --git a/include/utils/singleton.h b/include/utils/singleton.h
index f02c595..4cf487e 100644
--- a/include/utils/singleton.h
+++ b/include/utils/singleton.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_SINGLETON_H_
#define SINGA_UTILS_SINGLETON_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/include/utils/updater.h
----------------------------------------------------------------------
diff --git a/include/utils/updater.h b/include/utils/updater.h
index 25fc31f..4afa32f 100644
--- a/include/utils/updater.h
+++ b/include/utils/updater.h
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#ifndef SINGA_UTILS_UPDATER_H_
#define SINGA_UTILS_UPDATER_H_
[14/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
SINGA-68 Preparation for Release -- License etc.
Add README.md.
Updated Readme.md with lincese, documentation, mail list description.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/9cded5e3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/9cded5e3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/9cded5e3
Branch: refs/heads/tutorial
Commit: 9cded5e3a98b073138525fab1f5b3589e4040ed0
Parents: e1e8838
Author: xiezl <xi...@comp.nus.edu.sg>
Authored: Tue Sep 15 11:34:28 2015 +0800
Committer: wang sheng <wa...@gmail.com>
Committed: Wed Sep 16 11:09:25 2015 +0800
----------------------------------------------------------------------
README.md | 61 ++++++++++++++++++++++++++++++++++++++++++++++------------
1 file changed, 49 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/9cded5e3/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 6ccaac4..0777c6b 100644
--- a/README.md
+++ b/README.md
@@ -1,20 +1,57 @@
-TODO
-##Apache SINGA
+#Apache SINGA
Distributed deep learning system
-[Project Website](http://singa.incubator.apache.org)
+##Project Website
-All the details can be found in project website.
+All the details can be found in [Project Website](http://singa.incubator.apache.org), including the following instructions.
-The current code depends on the following external libraries:
- * glog (New BSD)
- * google-protobuf (New BSD)
- * openblas (New BSD)
- * zeromq (LGPLv3 + static link exception)
- * czmq (Mozilla Public License Version 2.0)
- * zookeeper (Apache 2.0)
- * lmdb (OpenLDAP)
+##Mailing Lists
+* [Development Mailing List](mailto:dev-subscribe@singa.incubator.apache.org)([Archive](http://mail-archives.apache.org/mod_mbox/singa-dev/))
+* [Commits Mailing List](mailto:commits-subscribe@singa.incubator.apache.org)([Archive](http://mail-archives.apache.org/mod_mbox/singa-commits/))
+##Documentation
+
+Documentation is available in [Official Documentation](https://singa.incubator.apache.org/docs/overview.html#).
+
+##Building SINGA
+
+Just clone our github repo and execute following commands:
+
+ $ git clone git@github.com:apache/incubator-singa.git
+ $ cd incubator-singa
+ $ ./autogen.sh
+ $ ./configure
+ $ make
+
+You can also download the release package from the [Project Website](http://singa.incubator.apache.org) and follow the instructions on [Official Installation Guide](http://singa.incubator.apache.org/docs/installation.html).
+
+##Running an Example
+
+After installation, you may want to run an example to try SINGA.
+Let us train the [CNN model](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks) over the
+[CIFAR-10](http://www.cs.toronto.edu/~kriz/cifar.html) dataset without parallelism as an instance.
+The hyper-parameters are set following
+[cuda-convnet](https://code.google.com/p/cuda-convnet/). More details about this example are
+available at [CNN example](http://singa.incubator.apache.org/docs/cnn).
+
+We firstly download the dataset and create the data shard:
+
+ $ cd examples/cifar10/
+ $ make download
+ $ make create
+
+Then we start training:
+
+ $ cd ../../
+ $ ./bin/singa-run.sh -conf examples/cifar10/job.conf
+
+Now we just need to wait until it is done! About this part, you can also refer to the [Quick Start](http://singa.incubator.apache.org/docs/quick-start.html).
+
+##LICENSE
+
+Apache Singa is licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
+
+For additional information, see the LICENSE and NOTICE files.
[16/21] incubator-singa git commit: SINGA-10 Add Support for
Recurrent Neural Networks (RNN)
Posted by wa...@apache.org.
SINGA-10 Add Support for Recurrent Neural Networks (RNN)
Fix bugs from OutputLayer (wid-start).
Can reach 79 ppl, but need to optimize the training speed.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/78dcab92
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/78dcab92
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/78dcab92
Branch: refs/heads/tutorial
Commit: 78dcab92a9670dcb9053abcdcd49dffbae85ac73
Parents: cca37b9
Author: Wei Wang <wa...@comp.nus.edu.sg>
Authored: Wed Sep 16 00:46:50 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Wed Sep 16 11:39:16 2015 +0800
----------------------------------------------------------------------
examples/rnnlm/create_shard.cc | 49 ++++++++++++++++++++++++++++++-------
examples/rnnlm/job.conf | 6 ++---
examples/rnnlm/rnnlm.cc | 19 ++++++++------
examples/rnnlm/rnnlm.h | 4 ++-
4 files changed, 57 insertions(+), 21 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/78dcab92/examples/rnnlm/create_shard.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/create_shard.cc b/examples/rnnlm/create_shard.cc
index dd56a84..f337350 100644
--- a/examples/rnnlm/create_shard.cc
+++ b/examples/rnnlm/create_shard.cc
@@ -1,16 +1,45 @@
+/*
+ * This file include code from rnnlmlib-0.4 whose licence is as follows:
+Copyright (c) 2010-2012 Tomas Mikolov
+Copyright (c) 2013 Cantab Research Ltd
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+
+1. Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright
+notice, this list of conditions and the following disclaimer in the
+documentation and/or other materials provided with the distribution.
+
+3. Neither name of copyright holders nor the names of its contributors
+may be used to endorse or promote products derived from this software
+without specific prior written permission.
+
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+*/
//
// This code creates DataShard for RNNLM dataset.
-// It is adapted from the convert_mnist_data from Caffe
// The RNNLM dataset could be downloaded at
// http://www.rnnlm.org/
//
// Usage:
// create_shard.bin -train train_file -class_size [-debug] [-valid valid_file] [-test test_file]
-#include <gflags/gflags.h>
-#include <glog/logging.h>
-
-
#include "utils/data_shard.h"
#include "utils/common.h"
#include "proto/common.pb.h"
@@ -255,14 +284,15 @@ int init_class() {
return 0;
}
-int create_shard(char *input_file, char *output_file) {
+int create_shard(const char *input_file, const char *output_file) {
DataShard dataShard(output_file, DataShard::kCreate);
singa::WordRecord wordRecord;
- char word[MAX_STRING];
+ char word[MAX_STRING], str_buffer[32];
FILE *fin;
int a, i;
fin = fopen(input_file, "rb");
+ int wcnt = 0;
while (1) {
readWord(word, fin);
if (feof(fin)) break;
@@ -276,7 +306,8 @@ int create_shard(char *input_file, char *output_file) {
wordRecord.set_class_index(class_idx);
wordRecord.set_class_start(class_start[class_idx]);
wordRecord.set_class_end(class_end[class_idx]);
- dataShard.Insert(word, wordRecord);
+ int length = snprintf(str_buffer, 32, "%05d", wcnt++);
+ dataShard.Insert(string(str_buffer, length), wordRecord);
}
}
@@ -397,4 +428,4 @@ int main(int argc, char **argv) {
if (test_mode) create_shard(test_file, "test_shard");
return 0;
-}
\ No newline at end of file
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/78dcab92/examples/rnnlm/job.conf
----------------------------------------------------------------------
diff --git a/examples/rnnlm/job.conf b/examples/rnnlm/job.conf
index 81bdb94..98a4157 100644
--- a/examples/rnnlm/job.conf
+++ b/examples/rnnlm/job.conf
@@ -5,7 +5,7 @@ cluster {
nworkers_per_group: 1
nservers_per_procs: 1
nworkers_per_procs: 1
- workspace: "examples/rnnlm/"
+ workspace: "examples/rnnlm/"
}
name: "recurrent-neural-network-language-model"
@@ -15,7 +15,7 @@ train_steps:214050
test_steps:583
test_freq:7135
#disp_freq is specific to training
-disp_freq:7135
+disp_freq:1000
train_one_batch {
alg: kBP
@@ -115,7 +115,7 @@ layer{
vocab_size: 3720
}
param{
- name: "w3"
+ name: "w3"
init {
type: kUniform
low:-0.3
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/78dcab92/examples/rnnlm/rnnlm.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.cc b/examples/rnnlm/rnnlm.cc
index 4d39b5f..0ad29a6 100644
--- a/examples/rnnlm/rnnlm.cc
+++ b/examples/rnnlm/rnnlm.cc
@@ -47,17 +47,14 @@ void RnnDataLayer::ComputeFeature(int flag, Metric *perf) {
CHECK(records_.size() <= shard_->Count());
records_[0] = records_[window_];
window_ = max_window_;
- singa::WordRecord wr;
for (int i = 1; i <= max_window_; i++) {
string key;
if (shard_->Next(&key, &records_[i])) {
- wr = records_[i];
- if(wr.word_index() == 0) {
- window_ = i;
+ if(records_[i].word_index() == 0) {
+ window_ = i; // +1 ??
break;
}
- }
- else{
+ } else{
shard_->SeekToFirst();
CHECK(shard_->Next(&key, &records_[i]));
}
@@ -68,6 +65,7 @@ void RnnDataLayer::ComputeFeature(int flag, Metric *perf) {
void WordLayer::Setup(const LayerProto& proto, int npartitions) {
Layer::Setup(proto, npartitions);
CHECK_EQ(srclayers_.size(), 1);
+ LOG(ERROR) << srclayers_[0]->name();
int max_window = static_cast<RnnDataLayer*>(srclayers_[0])->max_window();
LOG(ERROR) << "clee " << max_window;
data_.Reshape(vector<int>{max_window});
@@ -213,7 +211,7 @@ void OutputLayer::Setup(const LayerProto& proto, int npartitions) {
int nclass = proto.GetExtension(output_conf).nclass();
word_weight_ = Param::Create(proto.param(0));
word_weight_->Setup(vector<int>{vocab_size, vdim});
- class_weight_ = Param::Create(proto.param(0));
+ class_weight_ = Param::Create(proto.param(1));
class_weight_->Setup(vector<int>{nclass, vdim});
pword_.resize(max_window);
@@ -234,6 +232,7 @@ void OutputLayer::ComputeFeature(int flag, Metric* perf) {
int end = static_cast<int>(label[t * 4 + 1]);
auto wordWeight = word_weight.Slice(start, end);
+ CHECK_GT(end, start);
pword_[t].Reshape(vector<int>{end-start});
auto pword = RTensor1(&pword_[t]);
pword = dot(src[t], wordWeight.T());
@@ -244,6 +243,8 @@ void OutputLayer::ComputeFeature(int flag, Metric* perf) {
int wid = static_cast<int>(label[t * 4 + 2]);
int cid = static_cast<int>(label[t * 4 + 3]);
+ CHECK_GT(end, wid);
+ CHECK_GE(wid, start);
loss += -log(std::max(pword[wid - start] * pclass[t][cid], FLT_MIN));
ppl += log10(std::max(pword[wid - start] * pclass[t][cid], FLT_MIN));
}
@@ -269,11 +270,13 @@ void OutputLayer::ComputeGradient(int flag, Metric* perf) {
int wid = static_cast<int>(label[t * 4 + 2]);
int cid = static_cast<int>(label[t * 4 + 3]);
auto pword = RTensor1(&pword_[t]);
+ CHECK_GT(end, wid);
+ CHECK_GE(wid, start);
// gL/gclass_act
pclass[t][cid] -= 1.0;
// gL/gword_act
- pword[wid] -= 1.0;
+ pword[wid - start] -= 1.0;
// gL/gword_weight
gword_weight.Slice(start, end) += dot(pword.FlatTo2D().T(), src[t].FlatTo2D());
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/78dcab92/examples/rnnlm/rnnlm.h
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.h b/examples/rnnlm/rnnlm.h
index e9b7c55..888ebe7 100644
--- a/examples/rnnlm/rnnlm.h
+++ b/examples/rnnlm/rnnlm.h
@@ -32,10 +32,12 @@ class RnnDataLayer : public RNNLayer {
void Setup(const LayerProto& proto, int npartitions) override;
void ComputeFeature(int flag, Metric *perf) override;
void ComputeGradient(int flag, Metric* perf) override {}
+ ConnectionType dst_layer_connection() const override {
+ return kOneToMany;
+ }
int max_window() const {
return max_window_;
}
-
const std::vector<singa::WordRecord>& records() const {
return records_;
}
[20/21] incubator-singa git commit: SINGA-10 Add Support for
Recurrent Neural Networks (RNN)
Posted by wa...@apache.org.
SINGA-10 Add Support for Recurrent Neural Networks (RNN)
Add user-defined records for word (word string, word id, class id,
startpos, endpos);
Implement RnnDataLayer, WordLayer and RnnLabelLayer;
Implement create_shard.cc for the sample dataset of rnnlmlib;
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/b4a8d2b2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/b4a8d2b2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/b4a8d2b2
Branch: refs/heads/tutorial
Commit: b4a8d2b2beb077e6488569791a1581add4a957bc
Parents: e53a23c
Author: kaiping <ka...@comp.nus.edu.sg>
Authored: Sun Sep 13 20:07:11 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Wed Sep 16 11:39:16 2015 +0800
----------------------------------------------------------------------
examples/rnnlm/Makefile.example | 26 +++
examples/rnnlm/create_shard.cc | 400 +++++++++++++++++++++++++++++++++++
examples/rnnlm/main.cc | 3 +
examples/rnnlm/rnnlm.cc | 77 +++++++
examples/rnnlm/rnnlm.h | 47 ++++
examples/rnnlm/rnnlm.proto | 15 ++
6 files changed, 568 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b4a8d2b2/examples/rnnlm/Makefile.example
----------------------------------------------------------------------
diff --git a/examples/rnnlm/Makefile.example b/examples/rnnlm/Makefile.example
index 5eeca78..b4505cf 100644
--- a/examples/rnnlm/Makefile.example
+++ b/examples/rnnlm/Makefile.example
@@ -1,5 +1,31 @@
MSHADOW_FLAGS :=-DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0
+libs :=singa glog protobuf
+filename = rnnlm-0.4b.tgz
+# note: filelink for rnnlm-0.4b may change
+filelink = https://f25ea9ccb7d3346ce6891573d543960492b92c30.googledrive.com/host/0ByxdPXuxLPS5RFM5dVNvWVhTd0U
+dirname = $(patsubst %.tgz,%, $(filename))
+numclass = 100
+dirshards = train_shard valid_shard test_shard
+
+
+.PHONY: all download create
+
+download: rnnlm
+
+rnnlm:
+ wget $(filelink)/$(filename)
+ tar zxf $(filename)
+ rm $(filename)
+
+create:
+ $(CXX) create_shard.cc -std=c++11 -lsinga -lprotobuf -lzookeeper_mt -lglog -I../../include \
+ -L../../.libs/ -L/usr/local/lib -Wl,-unresolved-symbols=ignore-in-shared-libs -Wl,-rpath=../../.libs/ \
+ -o create_shard.bin
+ for d in $(dirshards); do mkdir -p $${d}; done
+ ./create_shard.bin -train $(dirname)/train -class_size $(numclass) -test $(dirname)/test
+
+
all:
protoc --proto_path=../../src/proto --proto_path=. --cpp_out=. rnnlm.proto
$(CXX) main.cc rnnlm.cc rnnlm.pb.cc $(MSHADOW_FLAGS) -std=c++11 -lsinga -lglog -lprotobuf -lopenblas -I../../include\
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b4a8d2b2/examples/rnnlm/create_shard.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/create_shard.cc b/examples/rnnlm/create_shard.cc
new file mode 100644
index 0000000..dd56a84
--- /dev/null
+++ b/examples/rnnlm/create_shard.cc
@@ -0,0 +1,400 @@
+//
+// This code creates DataShard for RNNLM dataset.
+// It is adapted from the convert_mnist_data from Caffe
+// The RNNLM dataset could be downloaded at
+// http://www.rnnlm.org/
+//
+// Usage:
+// create_shard.bin -train train_file -class_size [-debug] [-valid valid_file] [-test test_file]
+
+#include <gflags/gflags.h>
+#include <glog/logging.h>
+
+
+#include "utils/data_shard.h"
+#include "utils/common.h"
+#include "proto/common.pb.h"
+#include "singa.h"
+#include "rnnlm.pb.h"
+
+#define MAX_STRING 100
+
+#include <cstring>
+#include <cstdlib>
+#include <cstdio>
+#include <cmath>
+#include <algorithm>
+#include <fstream>
+
+using namespace std;
+using singa::DataShard;
+
+struct vocab_word {
+ int cn;
+ char word[MAX_STRING];
+ int class_index;
+};
+
+struct vocab_word *vocab;
+int vocab_max_size;
+int vocab_size;
+int *vocab_hash;
+int vocab_hash_size;
+int debug_mode;
+int old_classes;
+int *class_start;
+int *class_end;
+int class_size;
+
+char train_file[MAX_STRING];
+char valid_file[MAX_STRING];
+char test_file[MAX_STRING];
+
+int valid_mode;
+int test_mode;
+
+unsigned int getWordHash(char *word) {
+ unsigned int hash, a;
+
+ hash = 0;
+ for (a = 0; a < strlen(word); a++) hash = hash * 237 + word[a];
+ hash = hash % vocab_hash_size;
+
+ return hash;
+}
+
+int searchVocab(char *word) {
+ int a;
+ unsigned int hash;
+
+ hash = getWordHash(word);
+
+ if (vocab_hash[hash] == -1) return -1;
+ if (!strcmp(word, vocab[vocab_hash[hash]].word)) return vocab_hash[hash];
+
+ for (a = 0; a < vocab_size; a++) { //search in vocabulary
+ if (!strcmp(word, vocab[a].word)) {
+ vocab_hash[hash] = a;
+ return a;
+ }
+ }
+
+ return -1; //return OOV if not found
+}
+
+int addWordToVocab(char *word) {
+ unsigned int hash;
+
+ strcpy(vocab[vocab_size].word, word);
+ vocab[vocab_size].cn = 0;
+ vocab_size++;
+
+ if (vocab_size + 2 >= vocab_max_size) { //reallocate memory if needed
+ vocab_max_size += 100;
+ vocab = (struct vocab_word *) realloc(vocab, vocab_max_size * sizeof(struct vocab_word));
+ }
+
+ hash = getWordHash(word);
+ vocab_hash[hash] = vocab_size - 1;
+
+ return vocab_size - 1;
+}
+
+void readWord(char *word, FILE *fin) {
+ int a = 0, ch;
+
+ while (!feof(fin)) {
+ ch = fgetc(fin);
+
+ if (ch == 13) continue;
+
+ if ((ch == ' ') || (ch == '\t') || (ch == '\n')) {
+ if (a > 0) {
+ if (ch == '\n') ungetc(ch, fin);
+ break;
+ }
+
+ if (ch == '\n') {
+ strcpy(word, (char *) "</s>");
+ return;
+ }
+ else continue;
+ }
+
+ word[a] = char(ch);
+ a++;
+
+ if (a >= MAX_STRING) {
+ //printf("Too long word found!\n"); //truncate too long words
+ a--;
+ }
+ }
+ word[a] = 0;
+}
+
+void sortVocab() {
+ int a, b, max;
+ vocab_word swap;
+
+ for (a = 1; a < vocab_size; a++) {
+ max = a;
+ for (b = a + 1; b < vocab_size; b++) if (vocab[max].cn < vocab[b].cn) max = b;
+
+ swap = vocab[max];
+ vocab[max] = vocab[a];
+ vocab[a] = swap;
+ }
+}
+
+int learnVocabFromTrainFile() {
+ char word[MAX_STRING];
+ FILE *fin;
+ int a, i, train_wcn;
+
+ for (a = 0; a < vocab_hash_size; a++) vocab_hash[a] = -1;
+
+ fin = fopen(train_file, "rb");
+
+ vocab_size = 0;
+
+ addWordToVocab((char *) "</s>");
+
+ train_wcn = 0;
+ while (1) {
+ readWord(word, fin);
+ if (feof(fin)) break;
+
+ train_wcn++;
+
+ i = searchVocab(word);
+ if (i == -1) {
+ a = addWordToVocab(word);
+ vocab[a].cn = 1;
+ } else vocab[i].cn++;
+ }
+
+ sortVocab();
+
+ if (debug_mode > 0) {
+ printf("Vocab size: %d\n", vocab_size);
+ printf("Words in train file: %d\n", train_wcn);
+ }
+
+ //train_words = train_wcn;
+
+ fclose(fin);
+ return 0;
+}
+
+int splitClasses() {
+ double df, dd;
+ int i, a, b;
+
+ df = 0;
+ dd = 0;
+ a = 0;
+ b = 0;
+
+ class_start = (int *) calloc(class_size, sizeof(int));
+ memset(class_start, 0x7f, sizeof(int) * class_size);
+ class_end = (int *) calloc(class_size, sizeof(int));
+ memset(class_end, 0, sizeof(int) * class_size);
+
+ if (old_classes) { // old classes
+ for (i = 0; i < vocab_size; i++) b += vocab[i].cn;
+ for (i = 0; i < vocab_size; i++) {
+ df += vocab[i].cn / (double) b;
+ if (df > 1) df = 1;
+ if (df > (a + 1) / (double) class_size) {
+ vocab[i].class_index = a;
+ if (a < class_size - 1) a++;
+ } else {
+ vocab[i].class_index = a;
+ }
+ }
+ } else { // new classes
+ for (i = 0; i < vocab_size; i++) b += vocab[i].cn;
+ for (i = 0; i < vocab_size; i++) dd += sqrt(vocab[i].cn / (double) b);
+ for (i = 0; i < vocab_size; i++) {
+ df += sqrt(vocab[i].cn / (double) b) / dd;
+ if (df > 1) df = 1;
+ if (df > (a + 1) / (double) class_size) {
+ vocab[i].class_index = a;
+ if (a < class_size - 1) a++;
+ } else {
+ vocab[i].class_index = a;
+ }
+ }
+ }
+
+ // after dividing classes, update class start and class end information
+ for(i = 0; i < vocab_size; i++) {
+ a = vocab[i].class_index;
+ class_start[a] = min(i, class_start[a]);
+ class_end[a] = max(i + 1, class_end[a]);
+ }
+ return 0;
+}
+
+int init_class() {
+ //debug_mode = 1;
+ debug_mode = 0;
+ vocab_max_size = 100; // largest length value for each word
+ vocab_size = 0;
+ vocab = (struct vocab_word *) calloc(vocab_max_size, sizeof(struct vocab_word));
+ vocab_hash_size = 100000000;
+ vocab_hash = (int *) calloc(vocab_hash_size, sizeof(int));
+ old_classes = 1;
+
+ // read vocab
+ learnVocabFromTrainFile();
+
+ // split classes
+ splitClasses();
+
+ return 0;
+}
+
+int create_shard(char *input_file, char *output_file) {
+ DataShard dataShard(output_file, DataShard::kCreate);
+ singa::WordRecord wordRecord;
+
+ char word[MAX_STRING];
+ FILE *fin;
+ int a, i;
+ fin = fopen(input_file, "rb");
+ while (1) {
+ readWord(word, fin);
+ if (feof(fin)) break;
+ i = searchVocab(word);
+ if (i == -1) {
+ if (debug_mode) printf("unknown word [%s] detected!", word);
+ } else {
+ wordRecord.set_word(string(word));
+ wordRecord.set_word_index(i);
+ int class_idx = vocab[i].class_index;
+ wordRecord.set_class_index(class_idx);
+ wordRecord.set_class_start(class_start[class_idx]);
+ wordRecord.set_class_end(class_end[class_idx]);
+ dataShard.Insert(word, wordRecord);
+ }
+ }
+
+ dataShard.Flush();
+ fclose(fin);
+ return 0;
+}
+
+int argPos(char *str, int argc, char **argv) {
+ int a;
+
+ for (a = 1; a < argc; a++) if (!strcmp(str, argv[a])) return a;
+
+ return -1;
+}
+
+int main(int argc, char **argv) {
+ int i;
+ FILE *f;
+
+ //set debug mode
+ i = argPos((char *) "-debug", argc, argv);
+ if (i > 0) {
+ debug_mode = 1;
+ if (debug_mode > 0)
+ printf("debug mode: %d\n", debug_mode);
+ }
+
+ //search for train file
+ i = argPos((char *) "-train", argc, argv);
+ if (i > 0) {
+ if (i + 1 == argc) {
+ printf("ERROR: training data file not specified!\n");
+ return 0;
+ }
+
+ strcpy(train_file, argv[i + 1]);
+
+ if (debug_mode > 0)
+ printf("train file: %s\n", train_file);
+
+ f = fopen(train_file, "rb");
+ if (f == NULL) {
+ printf("ERROR: training data file not found!\n");
+ return 0;
+ }
+ fclose(f);
+ } else {
+ printf("ERROR: training data must be set.\n");
+ }
+
+ //search for valid file
+ i = argPos((char *) "-valid", argc, argv);
+ if (i > 0) {
+ if (i + 1 == argc) {
+ printf("ERROR: validating data file not specified!\n");
+ return 0;
+ }
+
+ strcpy(valid_file, argv[i + 1]);
+
+ if (debug_mode > 0)
+ printf("valid file: %s\n", valid_file);
+
+ f = fopen(valid_file, "rb");
+ if (f == NULL) {
+ printf("ERROR: validating data file not found!\n");
+ return 0;
+ }
+ fclose(f);
+ valid_mode = 1;
+ }
+
+ //search for test file
+ i = argPos((char *) "-test", argc, argv);
+ if (i > 0) {
+ if (i + 1 == argc) {
+ printf("ERROR: testing data file not specified!\n");
+ return 0;
+ }
+
+ strcpy(test_file, argv[i + 1]);
+
+ if (debug_mode > 0)
+ printf("test file: %s\n", test_file);
+
+ f = fopen(test_file, "rb");
+ if (f == NULL) {
+ printf("ERROR: testing data file not found!\n");
+ return 0;
+ }
+ fclose(f);
+ test_mode = 1;
+ }
+
+ //search for class size
+ i = argPos((char *) "-class_size", argc, argv);
+ if (i > 0) {
+ if (i + 1 == argc) {
+ printf("ERROR: class size not specified!\n");
+ return 0;
+ }
+
+ class_size = atoi(argv[i + 1]);
+
+ if (debug_mode > 0)
+ printf("class size: %d\n", class_size);
+ }
+ if (class_size <= 0) {
+ printf("ERROR: no or invalid class size received!\n");
+ return 0;
+ }
+
+ init_class();
+
+ create_shard(train_file, "train_shard");
+ if (valid_mode) create_shard(valid_file, "valid_shard");
+ if (test_mode) create_shard(test_file, "test_shard");
+
+ return 0;
+}
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b4a8d2b2/examples/rnnlm/main.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/main.cc b/examples/rnnlm/main.cc
index 690c158..3cb59f1 100644
--- a/examples/rnnlm/main.cc
+++ b/examples/rnnlm/main.cc
@@ -15,6 +15,9 @@ int main(int argc, char **argv) {
driver.RegisterLayer<singa::EmbeddingLayer, std::string>("kEmbedding");
driver.RegisterLayer<singa::HiddenLayer, std::string>("kHidden");
driver.RegisterLayer<singa::OutputLayer, std::string>("kOutput");
+ driver.RegisterLayer<singa::RnnDataLayer, std::string>("kRnnData");
+ driver.RegisterLayer<singa::WordLayer, std::string>("kWord");
+ driver.RegisterLayer<singa::RnnLabelLayer, std::string>("kRnnLabel");
singa::JobProto jobConf = driver.job_conf();
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b4a8d2b2/examples/rnnlm/rnnlm.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.cc b/examples/rnnlm/rnnlm.cc
index ddb0f63..180300f 100644
--- a/examples/rnnlm/rnnlm.cc
+++ b/examples/rnnlm/rnnlm.cc
@@ -25,6 +25,83 @@ inline Tensor<cpu, 1> RTensor1(Blob<float>* blob) {
return tensor;
}
+
+/*******InputLayer**************/
+RnnDataLayer::~RnnDataLayer() {
+ if (shard_ != nullptr)
+ delete shard_;
+ shard_ = nullptr;
+}
+
+void RnnDataLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ shard_ = new DataShard(proto.GetExtension(input_conf).path(), DataShard::kRead);
+ string key;
+ max_window_ = proto.GetExtension(input_conf).max_window();
+ records_.resize(max_window_ + 1); // # of records in data layer is max_window_ + 1
+ window_ = 0;
+ shard_->Next(&key, &records_[window_]);
+}
+
+void RnnDataLayer::ComputeFeature(int flag, Metric *perf) {
+ CHECK(records_.size() <= shard_->Count());
+ records_[0] = records_[window_];
+ window_ = max_window_;
+ singa::WordRecord wr;
+ for (int i = 1; i <= max_window_; i++) {
+ string key;
+ if (shard_->Next(&key, &records_[i])) {
+ wr = records_[i];
+ if(wr.word_index() == 0) {
+ window_ = i;
+ break;
+ }
+ }
+ else{
+ shard_->SeekToFirst();
+ CHECK(shard_->Next(&key, &records_[i]));
+ }
+ }
+}
+
+/*******WordLayer**************/
+void WordLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 1);
+ int max_window = static_cast<RnnDataLayer*>(srclayers_[0])->max_window();
+ data_.Reshape(vector<int>{max_window});
+}
+
+void WordLayer::ComputeFeature(int flag, Metric *perf) {
+ auto records = static_cast<RnnDataLayer*>(srclayers_[0])->records();
+ float *word = data_.mutable_cpu_data();
+ window_ = static_cast<RNNLayer*>(srclayers_[0])->window();
+ for(int i = 0; i < window_; i++) {
+ word[i] = records[i].word_index();
+ }
+}
+
+
+/*******LabelLayer**************/
+void RnnLabelLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 1);
+ int max_window = static_cast<RnnDataLayer*>(srclayers_[0])->max_window();
+ data_.Reshape(vector<int>{max_window, 4});
+}
+
+void RnnLabelLayer::ComputeFeature(int flag, Metric *perf) {
+ auto records = static_cast<RnnDataLayer*>(srclayers_[0])->records();
+ float *label = data_.mutable_cpu_data();
+ window_ = static_cast<RNNLayer*>(srclayers_[0])->window();
+ for (int i = 0; i < window_; i++) {
+ label[4 * i + 0] = records[i + 1].class_start();
+ label[4 * i + 1] = records[i + 1].class_end();
+ label[4 * i + 2] = records[i + 1].word_index();
+ label[4 * i + 3] = records[i + 1].class_index();
+ }
+}
+
/*******EmbeddingLayer**************/
EmbeddingLayer::~EmbeddingLayer() {
delete embed_;
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b4a8d2b2/examples/rnnlm/rnnlm.h
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.h b/examples/rnnlm/rnnlm.h
index 14d947c..e9b7c55 100644
--- a/examples/rnnlm/rnnlm.h
+++ b/examples/rnnlm/rnnlm.h
@@ -1,4 +1,5 @@
#include "singa.h"
+#include "rnnlm.pb.h"
namespace singa {
/**
@@ -23,6 +24,52 @@ class RNNLayer : public NeuronLayer {
};
/**
+ * Input layer that get read records from data shard
+ */
+class RnnDataLayer : public RNNLayer {
+ public:
+ ~RnnDataLayer();
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override {}
+ int max_window() const {
+ return max_window_;
+ }
+
+ const std::vector<singa::WordRecord>& records() const {
+ return records_;
+ }
+
+ private:
+ int max_window_;
+ DataShard* shard_;
+ std::vector<singa::WordRecord> records_;
+};
+
+
+/**
+ * WordLayer that read records_[0] to records_[window_ - 1] from RnnDataLayer to offer data for computation
+ */
+class WordLayer : public RNNLayer {
+ public:
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override {}
+};
+
+
+/**
+ * LabelLayer that read records_[1] to records_[window_] from RnnDataLayer to offer label information
+ */
+class RnnLabelLayer : public RNNLayer {
+ public:
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override {}
+};
+
+
+/**
* Word embedding layer that get one row from the embedding matrix for each
* word based on the word index
*/
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b4a8d2b2/examples/rnnlm/rnnlm.proto
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.proto b/examples/rnnlm/rnnlm.proto
index 35b6bc2..65c34ec 100644
--- a/examples/rnnlm/rnnlm.proto
+++ b/examples/rnnlm/rnnlm.proto
@@ -1,5 +1,6 @@
package singa;
import "job.proto";
+import "common.proto";
message EmbeddingProto {
@@ -12,7 +13,21 @@ message OutputProto {
optional int32 vocab_size = 2;
}
+message InputProto {
+ required string path = 1;
+ optional int32 max_window = 2;
+}
+
extend LayerProto {
optional EmbeddingProto embedding_conf = 101;
optional OutputProto output_conf = 102;
+ optional InputProto input_conf = 103;
}
+
+message WordRecord {
+ optional string word = 1;
+ optional int32 word_index = 2;
+ optional int32 class_index = 3;
+ optional int32 class_start = 4;
+ optional int32 class_end = 5;
+}
\ No newline at end of file
[15/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
SINGA-68 Preparation for Release -- License etc.
Minor changes to LICENSE, NOTICE, RELEASE_NOTES to make them consistent.
Still need to complete README file.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/e1e88389
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/e1e88389
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/e1e88389
Branch: refs/heads/tutorial
Commit: e1e88389e8b65740752881ca2b694273cb5e8eb1
Parents: be035df
Author: Anh Dinh <ug...@gmail.com>
Authored: Sun Sep 13 20:07:58 2015 +0800
Committer: wang sheng <wa...@gmail.com>
Committed: Wed Sep 16 11:09:25 2015 +0800
----------------------------------------------------------------------
LICENSE | 632 ++---------------------------------------------------
NOTICE | 8 +-
README | 20 ++
README.md | 11 +
RELEASE_NOTES | 23 +-
5 files changed, 60 insertions(+), 634 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e1e88389/LICENSE
----------------------------------------------------------------------
diff --git a/LICENSE b/LICENSE
index 83a58c7..64f06dd 100644
--- a/LICENSE
+++ b/LICENSE
@@ -206,627 +206,29 @@ subcomponents is subject to the terms and conditions of the following
licenses.
============================================================================
-For cafee (see comments in src/neuralnet/layer.cc)
-===========================================================================
-COPYRIGHT
-All contributions by the University of California:
-Copyright (c) 2014, The Regents of the University of California (Regents)
-All rights reserved.
-All other contributions:
-Copyright (c) 2014, the respective contributors
-All rights reserved.
-Caffe uses a shared copyright model: each contributor holds copyright over
-their contributions to Caffe. The project versioning records all such
-contribution and copyright details. If a contributor wants to further mark
-their specific copyright on a particular contribution, they should indicate
-their copyright solely in the commit message of the change when it is
-committed.
-LICENSE
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are met:
-1. Redistributions of source code must retain the above copyright notice, this
-list of conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright notice,
-this list of conditions and the following disclaimer in the documentation
-and/or other materials provided with the distribution.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
-ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
-ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
-(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
-ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-CONTRIBUTION AGREEMENT
-By contributing to the BVLC/caffe repository through pull-request, comment,
-or otherwise, the contributor releases their content to the
-license and copyright terms herein.
+SINGA bundles the following under BSD 2-clause license: include/utils/blob.h, src/utils/blob.cc,
+include/utils/common.h, src/utils/common.cc
-=====================================================================
-For czmq
-=====================================================================
-Mozilla Public License Version 2.0
-==================================
-
-1. Definitions
---------------
-
-1.1. "Contributor"
- means each individual or legal entity that creates, contributes to
- the creation of, or owns Covered Software.
-
-1.2. "Contributor Version"
- means the combination of the Contributions of others (if any) used
- by a Contributor and that particular Contributor's Contribution.
-
-1.3. "Contribution"
- means Covered Software of a particular Contributor.
-
-1.4. "Covered Software"
- means Source Code Form to which the initial Contributor has attached
- the notice in Exhibit A, the Executable Form of such Source Code
- Form, and Modifications of such Source Code Form, in each case
- including portions thereof.
-
-1.5. "Incompatible With Secondary Licenses"
- means
-
- (a) that the initial Contributor has attached the notice described
- in Exhibit B to the Covered Software; or
-
- (b) that the Covered Software was made available under the terms of
- version 1.1 or earlier of the License, but not also under the
- terms of a Secondary License.
-
-1.6. "Executable Form"
- means any form of the work other than Source Code Form.
-
-1.7. "Larger Work"
- means a work that combines Covered Software with other material, in
- a separate file or files, that is not Covered Software.
-
-1.8. "License"
- means this document.
-
-1.9. "Licensable"
- means having the right to grant, to the maximum extent possible,
- whether at the time of the initial grant or subsequently, any and
- all of the rights conveyed by this License.
-
-1.10. "Modifications"
- means any of the following:
-
- (a) any file in Source Code Form that results from an addition to,
- deletion from, or modification of the contents of Covered
- Software; or
+Copyright (c) 2014, 2015, The Regents of the University of California (Regents)
+https://github.com/BVLC/caffe/blob/master/LICENSE
- (b) any new file in Source Code Form that contains any Covered
- Software.
-
-1.11. "Patent Claims" of a Contributor
- means any patent claim(s), including without limitation, method,
- process, and apparatus claims, in any patent Licensable by such
- Contributor that would be infringed, but for the grant of the
- License, by the making, using, selling, offering for sale, having
- made, import, or transfer of either its Contributions or its
- Contributor Version.
-
-1.12. "Secondary License"
- means either the GNU General Public License, Version 2.0, the GNU
- Lesser General Public License, Version 2.1, the GNU Affero General
- Public License, Version 3.0, or any later versions of those
- licenses.
-
-1.13. "Source Code Form"
- means the form of the work preferred for making modifications.
-
-1.14. "You" (or "Your")
- means an individual or a legal entity exercising rights under this
- License. For legal entities, "You" includes any entity that
- controls, is controlled by, or is under common control with You. For
- purposes of this definition, "control" means (a) the power, direct
- or indirect, to cause the direction or management of such entity,
- whether by contract or otherwise, or (b) ownership of more than
- fifty percent (50%) of the outstanding shares or beneficial
- ownership of such entity.
-
-2. License Grants and Conditions
---------------------------------
-
-2.1. Grants
-
-Each Contributor hereby grants You a world-wide, royalty-free,
-non-exclusive license:
+=====================================================================
+SINGA bundles the following under BSD 2-clause license: include/utils/tinydir.h
-(a) under intellectual property rights (other than patent or trademark)
- Licensable by such Contributor to use, reproduce, make available,
- modify, display, perform, distribute, and otherwise exploit its
- Contributions, either on an unmodified basis, with Modifications, or
- as part of a Larger Work; and
+Copyright (c) 2013, Cong Xu
+https://github.com/cxong/tinydir/blob/master/COPYING
-(b) under Patent Claims of such Contributor to make, use, sell, offer
- for sale, have made, import, and otherwise transfer either its
- Contributions or its Contributor Version.
+=====================================================================
+SINGA bundles the following under Apache v2.0 license: include/mshadow/*
-2.2. Effective Date
+Copyright (c) 2014 by Contributors
+https://github.com/dmlc/mshadow/blob/master/LICENSE
-The licenses granted in Section 2.1 with respect to any Contribution
-become effective for each Contribution on the date the Contributor first
-distributes such Contribution.
+=====================================================================
+SINGA bundles the following under New BSD license: include/mshadow/*
-2.3. Limitations on Grant Scope
+Copyright 2008, Google Inc.
+https://code.google.com/p/googletest/source/browse/trunk/LICENSE
+=====================================================================
-The licenses granted in this Section 2 are the only rights granted under
-this License. No additional rights or licenses will be implied from the
-distribution or licensing of Covered Software under this License.
-Notwithstanding Section 2.1(b) above, no patent license is granted by a
-Contributor:
-
-(a) for any code that a Contributor has removed from Covered Software;
- or
-
-(b) for infringements caused by: (i) Your and any other third party's
- modifications of Covered Software, or (ii) the combination of its
- Contributions with other software (except as part of its Contributor
- Version); or
-
-(c) under Patent Claims infringed by Covered Software in the absence of
- its Contributions.
-
-This License does not grant any rights in the trademarks, service marks,
-or logos of any Contributor (except as may be necessary to comply with
-the notice requirements in Section 3.4).
-
-2.4. Subsequent Licenses
-
-No Contributor makes additional grants as a result of Your choice to
-distribute the Covered Software under a subsequent version of this
-License (see Section 10.2) or under the terms of a Secondary License (if
-permitted under the terms of Section 3.3).
-
-2.5. Representation
-
-Each Contributor represents that the Contributor believes its
-Contributions are its original creation(s) or it has sufficient rights
-to grant the rights to its Contributions conveyed by this License.
-
-2.6. Fair Use
-
-This License is not intended to limit any rights You have under
-applicable copyright doctrines of fair use, fair dealing, or other
-equivalents.
-
-2.7. Conditions
-
-Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
-in Section 2.1.
-
-3. Responsibilities
--------------------
-
-3.1. Distribution of Source Form
-
-All distribution of Covered Software in Source Code Form, including any
-Modifications that You create or to which You contribute, must be under
-the terms of this License. You must inform recipients that the Source
-Code Form of the Covered Software is governed by the terms of this
-License, and how they can obtain a copy of this License. You may not
-attempt to alter or restrict the recipients' rights in the Source Code
-Form.
-
-3.2. Distribution of Executable Form
-
-If You distribute Covered Software in Executable Form then:
-
-(a) such Covered Software must also be made available in Source Code
- Form, as described in Section 3.1, and You must inform recipients of
- the Executable Form how they can obtain a copy of such Source Code
- Form by reasonable means in a timely manner, at a charge no more
- than the cost of distribution to the recipient; and
-
-(b) You may distribute such Executable Form under the terms of this
- License, or sublicense it under different terms, provided that the
- license for the Executable Form does not attempt to limit or alter
- the recipients' rights in the Source Code Form under this License.
-
-3.3. Distribution of a Larger Work
-
-You may create and distribute a Larger Work under terms of Your choice,
-provided that You also comply with the requirements of this License for
-the Covered Software. If the Larger Work is a combination of Covered
-Software with a work governed by one or more Secondary Licenses, and the
-Covered Software is not Incompatible With Secondary Licenses, this
-License permits You to additionally distribute such Covered Software
-under the terms of such Secondary License(s), so that the recipient of
-the Larger Work may, at their option, further distribute the Covered
-Software under the terms of either this License or such Secondary
-License(s).
-
-3.4. Notices
-
-You may not remove or alter the substance of any license notices
-(including copyright notices, patent notices, disclaimers of warranty,
-or limitations of liability) contained within the Source Code Form of
-the Covered Software, except that You may alter any license notices to
-the extent required to remedy known factual inaccuracies.
-
-3.5. Application of Additional Terms
-
-You may choose to offer, and to charge a fee for, warranty, support,
-indemnity or liability obligations to one or more recipients of Covered
-Software. However, You may do so only on Your own behalf, and not on
-behalf of any Contributor. You must make it absolutely clear that any
-such warranty, support, indemnity, or liability obligation is offered by
-You alone, and You hereby agree to indemnify every Contributor for any
-liability incurred by such Contributor as a result of warranty, support,
-indemnity or liability terms You offer. You may include additional
-disclaimers of warranty and limitations of liability specific to any
-jurisdiction.
-
-4. Inability to Comply Due to Statute or Regulation
----------------------------------------------------
-
-If it is impossible for You to comply with any of the terms of this
-License with respect to some or all of the Covered Software due to
-statute, judicial order, or regulation then You must: (a) comply with
-the terms of this License to the maximum extent possible; and (b)
-describe the limitations and the code they affect. Such description must
-be placed in a text file included with all distributions of the Covered
-Software under this License. Except to the extent prohibited by statute
-or regulation, such description must be sufficiently detailed for a
-recipient of ordinary skill to be able to understand it.
-
-5. Termination
---------------
-
-5.1. The rights granted under this License will terminate automatically
-if You fail to comply with any of its terms. However, if You become
-compliant, then the rights granted under this License from a particular
-Contributor are reinstated (a) provisionally, unless and until such
-Contributor explicitly and finally terminates Your grants, and (b) on an
-ongoing basis, if such Contributor fails to notify You of the
-non-compliance by some reasonable means prior to 60 days after You have
-come back into compliance. Moreover, Your grants from a particular
-Contributor are reinstated on an ongoing basis if such Contributor
-notifies You of the non-compliance by some reasonable means, this is the
-first time You have received notice of non-compliance with this License
-from such Contributor, and You become compliant prior to 30 days after
-Your receipt of the notice.
-
-5.2. If You initiate litigation against any entity by asserting a patent
-infringement claim (excluding declaratory judgment actions,
-counter-claims, and cross-claims) alleging that a Contributor Version
-directly or indirectly infringes any patent, then the rights granted to
-You by any and all Contributors for the Covered Software under Section
-2.1 of this License shall terminate.
-
-5.3. In the event of termination under Sections 5.1 or 5.2 above, all
-end user license agreements (excluding distributors and resellers) which
-have been validly granted by You or Your distributors under this License
-prior to termination shall survive termination.
-
-************************************************************************
-* *
-* 6. Disclaimer of Warranty *
-* ------------------------- *
-* *
-* Covered Software is provided under this License on an "as is" *
-* basis, without warranty of any kind, either expressed, implied, or *
-* statutory, including, without limitation, warranties that the *
-* Covered Software is free of defects, merchantable, fit for a *
-* particular purpose or non-infringing. The entire risk as to the *
-* quality and performance of the Covered Software is with You. *
-* Should any Covered Software prove defective in any respect, You *
-* (not any Contributor) assume the cost of any necessary servicing, *
-* repair, or correction. This disclaimer of warranty constitutes an *
-* essential part of this License. No use of any Covered Software is *
-* authorized under this License except under this disclaimer. *
-* *
-************************************************************************
-
-************************************************************************
-* *
-* 7. Limitation of Liability *
-* -------------------------- *
-* *
-* Under no circumstances and under no legal theory, whether tort *
-* (including negligence), contract, or otherwise, shall any *
-* Contributor, or anyone who distributes Covered Software as *
-* permitted above, be liable to You for any direct, indirect, *
-* special, incidental, or consequential damages of any character *
-* including, without limitation, damages for lost profits, loss of *
-* goodwill, work stoppage, computer failure or malfunction, or any *
-* and all other commercial damages or losses, even if such party *
-* shall have been informed of the possibility of such damages. This *
-* limitation of liability shall not apply to liability for death or *
-* personal injury resulting from such party's negligence to the *
-* extent applicable law prohibits such limitation. Some *
-* jurisdictions do not allow the exclusion or limitation of *
-* incidental or consequential damages, so this exclusion and *
-* limitation may not apply to You. *
-* *
-************************************************************************
-
-8. Litigation
--------------
-
-Any litigation relating to this License may be brought only in the
-courts of a jurisdiction where the defendant maintains its principal
-place of business and such litigation shall be governed by laws of that
-jurisdiction, without reference to its conflict-of-law provisions.
-Nothing in this Section shall prevent a party's ability to bring
-cross-claims or counter-claims.
-
-9. Miscellaneous
-----------------
-
-This License represents the complete agreement concerning the subject
-matter hereof. If any provision of this License is held to be
-unenforceable, such provision shall be reformed only to the extent
-necessary to make it enforceable. Any law or regulation which provides
-that the language of a contract shall be construed against the drafter
-shall not be used to construe this License against a Contributor.
-
-10. Versions of the License
----------------------------
-
-10.1. New Versions
-
-Mozilla Foundation is the license steward. Except as provided in Section
-10.3, no one other than the license steward has the right to modify or
-publish new versions of this License. Each version will be given a
-distinguishing version number.
-
-10.2. Effect of New Versions
-
-You may distribute the Covered Software under the terms of the version
-of the License under which You originally received the Covered Software,
-or under the terms of any subsequent version published by the license
-steward.
-
-10.3. Modified Versions
-
-If you create software not governed by this License, and you want to
-create a new license for such software, you may create and use a
-modified version of this License if you rename the license and remove
-any references to the name of the license steward (except to note that
-such modified license differs from this License).
-
-10.4. Distributing Source Code Form that is Incompatible With Secondary
-Licenses
-
-If You choose to distribute Source Code Form that is Incompatible With
-Secondary Licenses under the terms of this version of the License, the
-notice described in Exhibit B of this License must be attached.
-
-Exhibit A - Source Code Form License Notice
--------------------------------------------
-
- This Source Code Form is subject to the terms of the Mozilla Public
- License, v. 2.0. If a copy of the MPL was not distributed with this
- file, You can obtain one at http://mozilla.org/MPL/2.0/.
-
-If it is not possible or desirable to put the notice in a particular
-file, then You may include the notice in a location (such as a LICENSE
-file in a relevant directory) where a recipient would be likely to look
-for such a notice.
-
-You may add additional accurate notices of copyright ownership.
-
-Exhibit B - "Incompatible With Secondary Licenses" Notice
----------------------------------------------------------
-
- This Source Code Form is "Incompatible With Secondary Licenses", as
- defined by the Mozilla Public License, v. 2.0.
-
-======================================================================
-For glog:
-======================================================================
-Copyright (c) 2008, Google Inc.
-All rights reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-* Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-* Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
-* Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-A function gettimeofday in utilities.cc is based on
-http://www.google.com/codesearch/p?hl=en#dR3YEbitojA/COPYING&q=GetSystemTimeAsFileTime%20license:bsd
-The license of this code is:
-Copyright (c) 2003-2008, Jouni Malinen <j...@w1.fi> and contributors
-All Rights Reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-1. Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-2. Redistributions in binary form must reproduce the above copyright
-notice, this list of conditions and the following disclaimer in the
-documentation and/or other materials provided with the distribution.
-3. Neither the name(s) of the above-listed copyright holder(s) nor the
-names of its contributors may be used to endorse or promote products
-derived from this software without specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-=====================================================================
-For lmdb
-=====================================================================
-The OpenLDAP Public License
- Version 2.8, 17 August 2003
-
- Redistribution and use of this software and associated documentation
- ("Software"), with or without modification, are permitted provided
- that the following conditions are met:
-
- 1. Redistributions in source form must retain copyright statements
- and notices,
-
- 2. Redistributions in binary form must reproduce applicable copyright
- statements and notices, this list of conditions, and the following disclaimer
- in the documentation and/or other materials provided with the distribution,
- and
-
- 3. Redistributions must contain a verbatim copy of this document.
-
- The OpenLDAP Foundation may revise this license from time to
- time.
- Each revision is distinguished by a version number. You may use
- this Software under terms of this license revision or under the
- terms of any subsequent revision of the license.
-
- THIS SOFTWARE IS PROVIDED BY THE OPENLDAP FOUNDATION AND ITS
- CONTRIBUTORS ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES,
- INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
- MERCHANTABILITY
- AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
- SHALL THE OPENLDAP FOUNDATION, ITS CONTRIBUTORS, OR THE AUTHOR(S)
- OR OWNER(S) OF THE SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT,
- INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- (INCLUDING,
- BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
- CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
- STRICT
- LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
- ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- POSSIBILITY OF SUCH DAMAGE.
-
- The names of the authors and copyright holders must not be used
- in
- advertising or otherwise to promote the sale, use or other
- dealing
- in this Software without specific, written prior permission.
- Title
- to copyright in this Software shall at all times remain with
- copyright
- holders.
-
- OpenLDAP is a registered trademark of the OpenLDAP Foundation.
-
- Copyright 1999-2003 The OpenLDAP Foundation, Redwood City,
- California, USA. All Rights Reserved. Permission to copy and
- distribute verbatim copies of this document is granted.
-
-======================================================================
-For protobuf:
-=====================================================================
-Copyright 2008, Google Inc.
-All rights reserved.
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-* Redistributions of source code must retain the above copyright
-notice, this list of conditions and the following disclaimer.
-* Redistributions in binary form must reproduce the above
-copyright notice, this list of conditions and the following disclaimer
-in the documentation and/or other materials provided with the
-distribution.
-* Neither the name of Google Inc. nor the names of its
-contributors may be used to endorse or promote products derived from
-this software without specific prior written permission.
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
-A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
-OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
-SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
-LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
-DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
-THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
-OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-Code generated by the Protocol Buffer compiler is owned by the owner
-of the input file used when generating it. This code is not
-standalone and requires a support library to be linked with it. This
-support library is itself covered by the above license.
-
-========================================================================
-For OpenCV used in preprocessing images
-=======================================================================
-License Agreement
-For Open Source Computer Vision Library
-(3-clause BSD License)
-
-Redistribution and use in source and binary forms, with or without modification,
-are permitted provided that the following conditions are met:
-Redistributions of source code must retain the above copyright notice,
-this list of conditions and the following disclaimer.
-Redistributions in binary form must reproduce the above copyright notice,
-this list of conditions and the following disclaimer in the documentation
-and/or other materials provided with the distribution.
-Neither the names of the copyright holders nor the names of the contributors
-may be used to endorse or promote products derived from this software without
-specific prior written permission.
-
-This software is provided by the copyright holders and contributors “as is”
-and any express or implied warranties, including, but not limited to, the
-implied warranties of merchantability and fitness for a particular purpose are
-disclaimed. In no event shall copyright holders or contributors be liable for
-any direct, indirect, incidental, special, exemplary, or consequential damages
-(including, but not limited to, procurement of substitute goods or services;
-loss of use, data, or profits; or business interruption) however caused and on
-any theory of liability, whether in contract, strict liability, or tort
-(including negligence or otherwise) arising in any way out of the use of this
-software, even if advised of the possibility of such damage.
-
-
-========================================================================
-For Openblas
-=======================================================================
-Copyright (c) 2011-2014, The OpenBLAS Project
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without
-modification, are permitted provided that the following conditions are
-met:
-
- 1. Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
-
- 2. Redistributions in binary form must reproduce the above copyright notice,
- this list of conditions and the following disclaimer in the documentation
- and/or other materials provided with the distribution.
- 3. Neither the name of the OpenBLAS project nor the names of its contributors
- may be used to endorse or promote products derived from this software without
- specific prior written permission.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- POSSIBILITY OF SUCH DAMAGE.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e1e88389/NOTICE
----------------------------------------------------------------------
diff --git a/NOTICE b/NOTICE
index 67a6d78..860314f 100644
--- a/NOTICE
+++ b/NOTICE
@@ -7,6 +7,8 @@ The Apache Software Foundation (http://www.apache.org/).
Portions of this software were developed at the National University of
Singapore.
-The software uses thridpart libaries including czmq, glog, google protobuf, OpenCV and Openblas.
-This software also contains code derived from the caffe project (https://github.com/BVLC/caffe)
-and tinydir from Cong Xu.
+This software contains codes from the other projects: Caffe (https://github.com/BVLC/caffe), tinydir
+(https://github.com/cxong/tinydir), mshadow (https://github.com/dmlc/mshadow) and gtest
+(https://code.google.com/p/googletest/).
+
+The software uses external libraries including czmq, glog, google protobuf, OpenCV and Openblas.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e1e88389/README
----------------------------------------------------------------------
diff --git a/README b/README
new file mode 100644
index 0000000..6ccaac4
--- /dev/null
+++ b/README
@@ -0,0 +1,20 @@
+TODO
+
+##Apache SINGA
+
+Distributed deep learning system
+
+[Project Website](http://singa.incubator.apache.org)
+
+All the details can be found in project website.
+
+The current code depends on the following external libraries:
+ * glog (New BSD)
+ * google-protobuf (New BSD)
+ * openblas (New BSD)
+ * zeromq (LGPLv3 + static link exception)
+ * czmq (Mozilla Public License Version 2.0)
+ * zookeeper (Apache 2.0)
+ * lmdb (OpenLDAP)
+
+
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e1e88389/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 513a1a9..6ccaac4 100644
--- a/README.md
+++ b/README.md
@@ -7,3 +7,14 @@ Distributed deep learning system
[Project Website](http://singa.incubator.apache.org)
All the details can be found in project website.
+
+The current code depends on the following external libraries:
+ * glog (New BSD)
+ * google-protobuf (New BSD)
+ * openblas (New BSD)
+ * zeromq (LGPLv3 + static link exception)
+ * czmq (Mozilla Public License Version 2.0)
+ * zookeeper (Apache 2.0)
+ * lmdb (OpenLDAP)
+
+
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e1e88389/RELEASE_NOTES
----------------------------------------------------------------------
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 2a114b5..2425cd0 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -1,6 +1,11 @@
Release Notes - SINGA - Version singa-incubating-0.1-rc0
-----------------------------------------
-This release includes these features:
+
+SINGA is a general distributed deep learning platform for training big deep learning models over large datasets. It is
+designed with an intuitive programming model based on the layer abstraction. SINGA supports a wide variety of popular
+deep learning models.
+
+This release includes following features:
* Job management
* [SINGA-3] - Use Zookeeper to check stopping (finish) time of the system
@@ -70,24 +75,10 @@ Some bugs are fixed during the development of this release
* [SINGA-49] Fix a bug in HandlePutMsg func that sets param fields to invalid values
* [SINGA-66] Fix bugs in Worker::RunOneBatch function and ClusterProto
-Some tasks should be done in the next release
+Features planned for the next release
* [SINGA-11] Start SINGA using Mesos
* [SINGA-31] Extend Blob to support xpu (cpu or gpu)
* [SINGA-35] Add random number generators
* [SINGA-40] Support sparse Param update
* [SINGA-41] Support single node single GPU training
-The current code depends on the following external libraries:
- * glog (New BSD)
- * google-protobuf (New BSD)
- * openblas (New BSD)
- * zeromq (LGPLv3 + static link exception)
- * czmq (Mozilla Public License Version 2.0)
- * zookeeper (Apache 2.0)
- * lmdb (OpenLDAP)
-
-Some source files/folders include third-party code:
- * include/gtest (New BSD)
- * include/mshadow (Apache 2.0)
- * include/utils/tinydir.h (BSD 2-clause)
- * caffe <include/utils/blob.h, src/utils/blob.cc, include/utils/common.h, src/utils/common.cc> (BSD 2-clause)
[19/21] incubator-singa git commit: SINGA-65 Add an example of
implementing user's own layers
Posted by wa...@apache.org.
SINGA-65 Add an example of implementing user's own layers
The folder examples/mlp includes files for a user-defined layer, i.e.,
HiddenLayer class. The main.cc register this class. The myproto.proto defines
the configuration of this layer and extends it to the base LayerProto.
Makefile.example contains instructions for compiling and linking
user code.
job.conf specifies a simple model with only one hidden layer.
deep.conf configures a deep model with 6 hidden layers, which is slower
per iteration, but converges faster than the simple model.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/4820d5e7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/4820d5e7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/4820d5e7
Branch: refs/heads/tutorial
Commit: 4820d5e757395915e0f4064ff6a0df304b4fea45
Parents: 78dcab9
Author: wangwei <wa...@comp.nus.edu.sg>
Authored: Thu Sep 3 09:57:03 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Wed Sep 16 11:39:16 2015 +0800
----------------------------------------------------------------------
examples/cifar10/job.conf | 25 +++--
examples/mlp/Makefile.example | 8 ++
examples/mlp/deep.conf | 221 +++++++++++++++++++++++++++++++++++++
examples/mlp/hidden_layer.cc | 72 ++++++++++++
examples/mlp/hidden_layer.h | 22 ++++
examples/mlp/job.conf | 95 ++++++++++++++++
examples/mlp/main.cc | 25 +++++
examples/mlp/myproto.proto | 11 ++
src/utils/tool.cc | 1 +
9 files changed, 468 insertions(+), 12 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/cifar10/job.conf
----------------------------------------------------------------------
diff --git a/examples/cifar10/job.conf b/examples/cifar10/job.conf
index 343d969..3f09c3f 100644
--- a/examples/cifar10/job.conf
+++ b/examples/cifar10/job.conf
@@ -2,25 +2,26 @@ name: "cifar10-convnet"
train_steps: 1000
test_steps: 100
test_freq:300
-disp_freq: 30
-train_one_batch {
+disp_freq:30
+train_one_batch
+{
alg: kBP
}
-updater{
+updater {
type: kSGD
weight_decay:0.004
learning_rate {
- type: kFixedStep
- fixedstep_conf:{
- step:0
- step:60000
- step:65000
- step_lr:0.001
- step_lr:0.0001
- step_lr:0.00001
- }
+ type: kFixedStep
+ fixedstep_conf:{
+ step:0
+ step:60000
+ step:65000
+ step_lr:0.001
+ step_lr:0.0001
+ step_lr:0.00001
}
}
+}
neuralnet {
layer{
name: "data"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/Makefile.example
----------------------------------------------------------------------
diff --git a/examples/mlp/Makefile.example b/examples/mlp/Makefile.example
new file mode 100644
index 0000000..71edd71
--- /dev/null
+++ b/examples/mlp/Makefile.example
@@ -0,0 +1,8 @@
+# refer to examples/mnist/ for data creation
+MSHADOW_FLAGS :=-DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0
+
+all:
+ protoc --proto_path=../../src/proto --proto_path=. --cpp_out=. myproto.proto
+ $(CXX) main.cc hidden_layer.cc myproto.pb.cc $(MSHADOW_FLAGS) -std=c++11 -lsinga -lglog -lprotobuf -lopenblas -I../../include\
+ -I../../include/proto/ -L../../.libs/ -L/usr/local -Wl,-unresolved-symbols=ignore-in-shared-libs -Wl,-rpath=../../.libs/\
+ -o mlp.bin
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/deep.conf
----------------------------------------------------------------------
diff --git a/examples/mlp/deep.conf b/examples/mlp/deep.conf
new file mode 100644
index 0000000..588fefa
--- /dev/null
+++ b/examples/mlp/deep.conf
@@ -0,0 +1,221 @@
+name: "mlp"
+train_steps: 1200
+test_steps:10
+test_freq:60
+disp_freq:10
+train_one_batch {
+ alg: kBP
+}
+updater{
+ type: kSGD
+ learning_rate{
+ type : kStep
+ base_lr: 0.001
+ step_conf{
+ change_freq: 60
+ gamma: 0.997
+ }
+ }
+}
+neuralnet {
+ layer {
+ name: "data"
+ type: kShardData
+ sharddata_conf {
+ path: "examples/mnist/mnist_train_shard"
+ batchsize: 1000
+ }
+ exclude: kTest
+ }
+
+ layer {
+ name: "data"
+ type: kShardData
+ sharddata_conf {
+ path: "examples/mnist/mnist_test_shard"
+ batchsize: 1000
+ }
+ exclude: kTrain
+ }
+
+ layer{
+ name:"mnist"
+ type: kMnist
+ srclayers: "data"
+ mnist_conf {
+ norm_a: 127.5
+ norm_b: 1
+ }
+ }
+ layer{
+ name: "label"
+ type: kLabel
+ srclayers: "data"
+ }
+
+ layer{
+ name: "hid1"
+ user_type: "kHidden"
+ srclayers:"mnist"
+ [singa.hidden_conf] {
+ num_output: 2500
+ }
+ param{
+ name: "w1"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b1"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+
+ layer{
+ name: "hid2"
+ user_type: "kHidden"
+ srclayers:"hid1"
+ [singa.hidden_conf] {
+ num_output: 2000
+ }
+ param{
+ name: "w2"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b2"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+
+ layer{
+ name: "hid3"
+ user_type: "kHidden"
+ srclayers:"hid2"
+ [singa.hidden_conf] {
+ num_output: 1500
+ }
+ param{
+ name: "w3"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b3"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+
+ layer{
+ name: "hid4"
+ user_type: "kHidden"
+ srclayers:"hid3"
+ [singa.hidden_conf] {
+ num_output: 1000
+ }
+ param{
+ name: "w4"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b4"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+
+
+ layer{
+ name: "hid5"
+ user_type: "kHidden"
+ srclayers:"hid4"
+ [singa.hidden_conf] {
+ num_output: 500
+ }
+ param{
+ name: "w5"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b5"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+
+
+ layer{
+ name: "hid6"
+ user_type: "kHidden"
+ srclayers:"hid5"
+ [singa.hidden_conf] {
+ num_output: 10
+ }
+ param{
+ name: "w6"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b6"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+ layer{
+ name: "loss"
+ type:kSoftmaxLoss
+ softmaxloss_conf{
+ topk:1
+ }
+ srclayers:"hid6"
+ srclayers:"label"
+ }
+}
+cluster {
+ nworker_groups: 1
+ nserver_groups: 1
+ workspace: "examples/mnist"
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/hidden_layer.cc
----------------------------------------------------------------------
diff --git a/examples/mlp/hidden_layer.cc b/examples/mlp/hidden_layer.cc
new file mode 100644
index 0000000..f4dacf2
--- /dev/null
+++ b/examples/mlp/hidden_layer.cc
@@ -0,0 +1,72 @@
+#include "hidden_layer.h"
+
+#include "mshadow/tensor.h"
+#include "mshadow/cxxnet_op.h"
+
+namespace singa {
+using namespace mshadow;
+using mshadow::cpu;
+using mshadow::Shape1;
+using mshadow::Shape2;
+
+inline Tensor<cpu, 2> NewTensor2(Blob<float>* blob) {
+ const vector<int>& shape = blob->shape();
+ Tensor<cpu, 2> tensor(blob->mutable_cpu_data(),
+ Shape2(shape[0], blob->count() / shape[0]));
+ return tensor;
+}
+
+inline Tensor<cpu, 1> NewTensor1(Blob<float>* blob) {
+ Tensor<cpu, 1> tensor(blob->mutable_cpu_data(), Shape1(blob->count()));
+ return tensor;
+}
+
+
+HiddenLayer::~HiddenLayer() {
+ delete weight_;
+ delete bias_;
+}
+
+void HiddenLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 1);
+ const auto& src = srclayers_[0]->data(this);
+ batchsize_ = src.shape()[0];
+ vdim_ = src.count() / batchsize_;
+ hdim_ = layer_proto_.GetExtension(hidden_conf).num_output();
+ data_.Reshape(vector<int>{batchsize_, hdim_});
+ grad_.ReshapeLike(data_);
+ weight_ = Param::Create(proto.param(0));
+ bias_ = Param::Create(proto.param(1));
+ weight_->Setup(vector<int>{hdim_, vdim_});
+ bias_->Setup(vector<int>{hdim_});
+}
+
+void HiddenLayer::ComputeFeature(int flag, Metric* perf) {
+ auto data = NewTensor2(&data_);
+ auto src = NewTensor2(srclayers_[0]->mutable_data(this));
+ auto weight = NewTensor2(weight_->mutable_data());
+ auto bias = NewTensor1(bias_->mutable_data());
+ data = dot(src, weight.T());
+ // repmat: repeat bias vector into batchsize rows
+ data += expr::repmat(bias, batchsize_);
+ data = expr::F<op::stanh>(data);
+}
+
+void HiddenLayer::ComputeGradient(int flag, Metric* perf) {
+ auto data = NewTensor2(&data_);
+ auto src = NewTensor2(srclayers_[0]->mutable_data(this));
+ auto grad = NewTensor2(&grad_);
+ auto weight = NewTensor2(weight_->mutable_data());
+ auto gweight = NewTensor2(weight_->mutable_grad());
+ auto gbias = NewTensor1(bias_->mutable_grad());
+
+ grad = expr::F<op::stanh_grad>(data) * grad;
+ gbias = expr::sum_rows(grad);
+ gweight = dot(grad.T(), src);
+ if (srclayers_[0]->mutable_grad(this) != nullptr) {
+ auto gsrc = NewTensor2(srclayers_[0]->mutable_grad(this));
+ gsrc = dot(grad, weight);
+ }
+}
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/hidden_layer.h
----------------------------------------------------------------------
diff --git a/examples/mlp/hidden_layer.h b/examples/mlp/hidden_layer.h
new file mode 100644
index 0000000..c305261
--- /dev/null
+++ b/examples/mlp/hidden_layer.h
@@ -0,0 +1,22 @@
+#include "neuralnet/layer.h"
+#include "myproto.pb.h"
+
+namespace singa {
+class HiddenLayer : public NeuronLayer {
+ public:
+ ~HiddenLayer();
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric* perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+ const std::vector<Param*> GetParams() const override {
+ std::vector<Param*> params{weight_, bias_};
+ return params;
+ }
+
+ private:
+ int batchsize_;
+ int vdim_, hdim_;
+ bool transpose_;
+ Param *weight_, *bias_;
+};
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/job.conf
----------------------------------------------------------------------
diff --git a/examples/mlp/job.conf b/examples/mlp/job.conf
new file mode 100644
index 0000000..163bc19
--- /dev/null
+++ b/examples/mlp/job.conf
@@ -0,0 +1,95 @@
+name: "mlp"
+train_steps: 1200
+test_steps:10
+test_freq:60
+disp_freq:10
+train_one_batch {
+ alg: kBP
+}
+updater{
+ type: kSGD
+ learning_rate{
+ type : kStep
+ base_lr: 0.001
+ step_conf{
+ change_freq: 60
+ gamma: 0.997
+ }
+ }
+}
+neuralnet {
+ layer {
+ name: "data"
+ type: kShardData
+ sharddata_conf {
+ path: "examples/mnist/mnist_train_shard"
+ batchsize: 1000
+ }
+ exclude: kTest
+ }
+
+ layer {
+ name: "data"
+ type: kShardData
+ sharddata_conf {
+ path: "examples/mnist/mnist_test_shard"
+ batchsize: 1000
+ }
+ exclude: kTrain
+ }
+
+ layer{
+ name:"mnist"
+ type: kMnist
+ srclayers: "data"
+ mnist_conf {
+ norm_a: 127.5
+ norm_b: 1
+ }
+ }
+ layer{
+ name: "label"
+ type: kLabel
+ srclayers: "data"
+ }
+
+
+ layer{
+ name: "hid1"
+ user_type: "kHidden"
+ srclayers:"mnist"
+ [singa.hidden_conf] {
+ num_output: 10
+ }
+ param{
+ name: "w1"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b1"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+ layer{
+ name: "loss"
+ type:kSoftmaxLoss
+ softmaxloss_conf{
+ topk:1
+ }
+ srclayers:"hid1"
+ srclayers:"label"
+ }
+}
+cluster {
+ nworker_groups: 1
+ nserver_groups: 1
+ workspace: "examples/mnist"
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/main.cc
----------------------------------------------------------------------
diff --git a/examples/mlp/main.cc b/examples/mlp/main.cc
new file mode 100644
index 0000000..c27f38d
--- /dev/null
+++ b/examples/mlp/main.cc
@@ -0,0 +1,25 @@
+#include <string>
+#include "singa.h"
+#include "hidden_layer.h"
+#include "myproto.pb.h"
+#include "utils/common.h"
+
+int main(int argc, char **argv) {
+ // must create driver at the beginning and call its Init method.
+ singa::Driver driver;
+ driver.Init(argc, argv);
+
+ // if -resume in argument list, set resume to true; otherwise false
+ int resume_pos = singa::ArgPos(argc, argv, "-resume");
+ bool resume = (resume_pos != -1);
+
+ // users can register new subclasses of layer, updater, etc.
+ driver.RegisterLayer<singa::HiddenLayer, std::string>("kHidden");
+
+ // get the job conf, and custmize it if need
+ singa::JobProto jobConf = driver.job_conf();
+
+ // submit the job
+ driver.Submit(resume, jobConf);
+ return 0;
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/examples/mlp/myproto.proto
----------------------------------------------------------------------
diff --git a/examples/mlp/myproto.proto b/examples/mlp/myproto.proto
new file mode 100644
index 0000000..ba62ac2
--- /dev/null
+++ b/examples/mlp/myproto.proto
@@ -0,0 +1,11 @@
+package singa;
+import "job.proto";
+
+message HiddenProto {
+ required int32 num_output = 1;
+ optional bool use_stanh = 2 [default = true];
+}
+
+extend LayerProto {
+ optional HiddenProto hidden_conf = 102;
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/4820d5e7/src/utils/tool.cc
----------------------------------------------------------------------
diff --git a/src/utils/tool.cc b/src/utils/tool.cc
index 305cbe7..d2bf631 100644
--- a/src/utils/tool.cc
+++ b/src/utils/tool.cc
@@ -24,6 +24,7 @@
#include <algorithm>
#include <fstream>
#include <iostream>
+#include <google/protobuf/text_format.h>
#include "proto/job.pb.h"
#include "proto/singa.pb.h"
#include "utils/cluster_rt.h"
[04/21] incubator-singa git commit: SINGA-51 Improve the convolution
and pooling operations
Posted by wa...@apache.org.
SINGA-51 Improve the convolution and pooling operations
Caffe's im2col is adopted to speed up the convolution operation.
The max pooling operation is accelerated by book-keeping the max neuron
position like Caffe.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/6d59eecf
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/6d59eecf
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/6d59eecf
Branch: refs/heads/tutorial
Commit: 6d59eecf0502a0f3575770cb166be94775cf83f2
Parents: 50deedd
Author: Wei Wang <wa...@comp.nus.edu.sg>
Authored: Sun Sep 13 20:00:02 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Sun Sep 13 20:00:02 2015 +0800
----------------------------------------------------------------------
examples/cifar10/job.conf | 20 +-
examples/mnist/conv.conf | 8 +-
include/neuralnet/neuron_layer.h | 21 +++
include/utils/common.h | 42 ++++-
src/driver.cc | 2 +
src/neuralnet/neuron_layer.cc | 82 +++++++-
src/proto/job.proto | 5 +-
src/trainer/trainer.cc | 1 -
src/utils/blob.cc | 2 +-
src/utils/common.cc | 344 +++++++++++++++++++++++++++++-----
10 files changed, 451 insertions(+), 76 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/examples/cifar10/job.conf
----------------------------------------------------------------------
diff --git a/examples/cifar10/job.conf b/examples/cifar10/job.conf
index 0fdd244..b36c45a 100644
--- a/examples/cifar10/job.conf
+++ b/examples/cifar10/job.conf
@@ -27,7 +27,7 @@ neuralnet {
type: kShardData
sharddata_conf {
path: "examples/cifar10/cifar10_train_shard"
- batchsize: 16
+ batchsize: 64
random_skip: 5000
}
exclude: kTest
@@ -57,7 +57,7 @@ neuralnet {
layer {
name: "conv1"
- type: kConvolution
+ type: kCConvolution
srclayers: "rgb"
convolution_conf {
num_filters: 32
@@ -84,7 +84,7 @@ neuralnet {
layer {
name: "pool1"
- type: kPooling
+ type: kCPooling
srclayers: "conv1"
pooling_conf {
pool: MAX
@@ -109,7 +109,7 @@ neuralnet {
}
layer {
name: "conv2"
- type: kConvolution
+ type: kCConvolution
srclayers: "norm1"
convolution_conf {
num_filters: 32
@@ -140,10 +140,10 @@ neuralnet {
}
layer {
name: "pool2"
- type: kPooling
+ type: kCPooling
srclayers: "relu2"
pooling_conf {
- pool: AVE
+ pool: AVG
kernel: 3
stride: 2
}
@@ -160,7 +160,7 @@ neuralnet {
}
layer {
name: "conv3"
- type: kConvolution
+ type: kCConvolution
srclayers: "norm2"
convolution_conf {
num_filters: 64
@@ -190,10 +190,10 @@ neuralnet {
}
layer {
name: "pool3"
- type: kPooling
+ type: kCPooling
srclayers: "relu3"
pooling_conf {
- pool: AVE
+ pool: AVG
kernel: 3
stride: 2
}
@@ -237,5 +237,7 @@ neuralnet {
cluster {
nworker_groups: 1
nserver_groups: 1
+ nworkers_per_group: 1
+ nworkers_per_procs: 1
workspace: "examples/cifar10"
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/examples/mnist/conv.conf
----------------------------------------------------------------------
diff --git a/examples/mnist/conv.conf b/examples/mnist/conv.conf
index aaf34f2..7f7a158 100644
--- a/examples/mnist/conv.conf
+++ b/examples/mnist/conv.conf
@@ -57,7 +57,7 @@ neuralnet {
}
layer {
name: "conv1"
- type: kConvolution
+ type: kCConvolution
srclayers: "mnist"
convolution_conf {
num_filters: 20
@@ -81,7 +81,7 @@ neuralnet {
}
layer {
name: "pool1"
- type: kPooling
+ type: kCPooling
srclayers: "conv1"
pooling_conf {
pool: MAX
@@ -91,7 +91,7 @@ neuralnet {
}
layer {
name: "conv2"
- type: kConvolution
+ type: kCConvolution
srclayers: "pool1"
convolution_conf {
num_filters: 50
@@ -115,7 +115,7 @@ neuralnet {
}
layer {
name: "pool2"
- type: kPooling
+ type: kCPooling
srclayers: "conv2"
pooling_conf {
pool: MAX
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/include/neuralnet/neuron_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/neuron_layer.h b/include/neuralnet/neuron_layer.h
index e5663d8..dd45eec 100644
--- a/include/neuralnet/neuron_layer.h
+++ b/include/neuralnet/neuron_layer.h
@@ -36,6 +36,15 @@ class ConvolutionLayer : public NeuronLayer {
Blob<float> col_data_, col_grad_;
};
+/**
+ * Use im2col from Caffe
+ */
+class CConvolutionLayer : public ConvolutionLayer {
+ public:
+ void ComputeFeature(int flag, Metric* perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+};
+
class DropoutLayer : public NeuronLayer {
public:
void Setup(const LayerProto& proto, int npartitions) override;
@@ -85,6 +94,18 @@ class PoolingLayer : public NeuronLayer {
PoolingProto_PoolMethod pool_;
};
+/**
+ * Use book-keeping for BP following Caffe's pooling implementation
+ */
+class CPoolingLayer : public PoolingLayer {
+ public:
+ void Setup(const LayerProto& proto, int npartitions);
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+ private:
+ Blob<float> mask_;
+};
+
class ReLULayer : public NeuronLayer {
public:
void Setup(const LayerProto& proto, int npartitions) override;
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/include/utils/common.h
----------------------------------------------------------------------
diff --git a/include/utils/common.h b/include/utils/common.h
index 868047a..599424d 100644
--- a/include/utils/common.h
+++ b/include/utils/common.h
@@ -14,14 +14,7 @@ namespace singa {
std::string IntVecToString(const std::vector<int>& vec);
std::string VStringPrintf(std::string fmt, va_list l);
std::string StringPrintf(std::string fmt, ...);
-void ReadProtoFromTextFile(const char* filename,
- google::protobuf::Message* proto);
-void WriteProtoToTextFile(const google::protobuf::Message& proto,
- const char* filename);
-void ReadProtoFromBinaryFile(const char* filename,
- google::protobuf::Message* proto);
-void WriteProtoToBinaryFile(const google::protobuf::Message& proto,
- const char* filename);
+
/**
* Locate the position of the arg in arglist.
*
@@ -102,6 +95,39 @@ class Metric {
std::unordered_map<std::string, std::pair<int, float>> entry_;
};
+using google::protobuf::Message;
+void Im2col(const float* data_im, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* data_col);
+void Col2im(const float* data_col, const int channels,
+ const int height, const int width, const int patch_h, const int patch_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* data_im);
+void ForwardMaxPooling(const float* bottom, const int num, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* top, float* mask);
+void BackwardMaxPooling(const float* top, const float* mask, const int num,
+ const int channels, const int height, const int width,
+ const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,
+ const int stride_h, const int stride_w,
+ float* bottom);
+void ForwardAvgPooling(const float* bottom, const int num, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* top);
+void BackwardAvgPooling(const float* top, const int num, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* bottom);
+
+void ReadProtoFromTextFile(const char* filename, Message* proto);
+void WriteProtoToTextFile(const Message& proto, const char* filename);
+void ReadProtoFromBinaryFile(const char* filename, Message* proto);
+void WriteProtoToBinaryFile(const Message& proto, const char* filename);
+
+
} // namespace singa
#endif // SINGA_UTILS_COMMON_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/src/driver.cc
----------------------------------------------------------------------
diff --git a/src/driver.cc b/src/driver.cc
index f017f45..a891a08 100644
--- a/src/driver.cc
+++ b/src/driver.cc
@@ -32,6 +32,8 @@ void Driver::Init(int argc, char **argv) {
RegisterLayer<BridgeDstLayer, int>(kBridgeDst);
RegisterLayer<BridgeSrcLayer, int>(kBridgeSrc);
RegisterLayer<ConvolutionLayer, int>(kConvolution);
+ RegisterLayer<CConvolutionLayer, int>(kCConvolution);
+ RegisterLayer<CPoolingLayer, int>(kCPooling);
RegisterLayer<ConcateLayer, int>(kConcate);
RegisterLayer<DropoutLayer, int>(kDropout);
RegisterLayer<EuclideanLossLayer, int>(kEuclideanLoss);
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/src/neuralnet/neuron_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/neuron_layer.cc b/src/neuralnet/neuron_layer.cc
index b86d7da..edfa022 100644
--- a/src/neuralnet/neuron_layer.cc
+++ b/src/neuralnet/neuron_layer.cc
@@ -134,6 +134,49 @@ void ConvolutionLayer::ComputeGradient(int flag, Metric* perf) {
}
}
+/******************* Implementation for CConvolutionLayer *********/
+void CConvolutionLayer::ComputeFeature(int flag, Metric* perf) {
+ auto src = Tensor4(srclayers_[0]->mutable_data(this));
+ auto data = Tensor3(&data_);
+ auto col = Tensor2(&col_data_);
+ auto weight = Tensor2(weight_->mutable_data());
+ auto bias = Tensor1(bias_->mutable_data());
+
+ for (int n = 0; n < batchsize_; n++) {
+ Im2col(src[n].dptr, channels_, height_, width_,
+ kernel_, kernel_, pad_, pad_, stride_, stride_, col.dptr);
+ data[n] = dot(weight, col);
+ }
+ data += expr::broadcast<1>(bias, data.shape);
+}
+
+void CConvolutionLayer::ComputeGradient(int flag, Metric* perf) {
+ auto src = Tensor4(srclayers_[0]->mutable_data(this));
+ auto col = Tensor2(&col_data_);
+ auto weight = Tensor2(weight_->mutable_data());
+
+ auto grad = Tensor3(&grad_);
+ auto gcol = Tensor2(&col_grad_);
+ auto gweight = Tensor2(weight_->mutable_grad());
+ auto gbias = Tensor1(bias_->mutable_grad());
+ gweight = 0.f;
+ Blob<float>* gsrcblob = srclayers_[0]->mutable_grad(this);
+ Tensor<cpu, 4> gsrc(nullptr, Shape4(batchsize_, channels_, height_, width_));
+ if (gsrcblob != nullptr)
+ gsrc.dptr = gsrcblob->mutable_cpu_data();
+ gbias = expr::sumall_except_dim<1>(grad);
+ for(int n = 0; n < batchsize_; n++) {
+ Im2col(src[n].dptr, channels_, height_, width_,
+ kernel_, kernel_, pad_, pad_, stride_, stride_, col.dptr);
+ gweight += dot(grad[n], col.T());
+ if (gsrcblob != nullptr) {
+ gcol = dot(weight.T(), grad[n]);
+ Col2im(gcol.dptr, channels_, height_, width_,
+ kernel_, kernel_, pad_, pad_, stride_, stride_, gsrc[n].dptr);
+ }
+ }
+}
+
/****************** Implementation for DropoutLayer ***********************/
void DropoutLayer::Setup(const LayerProto& proto, int npartitions) {
Layer::Setup(proto, npartitions);
@@ -430,7 +473,7 @@ void PoolingLayer::Setup(const LayerProto& proto, int npartitions) {
stride_ = pool_conf.stride();
CHECK_LT(pad_, kernel_);
pool_ = proto.pooling_conf().pool();
- CHECK(pool_ == PoolingProto_PoolMethod_AVE
+ CHECK(pool_ == PoolingProto_PoolMethod_AVG
|| pool_ == PoolingProto_PoolMethod_MAX)
<< "Padding implemented only for average and max pooling.";
const auto& srcshape = srclayers_[0]->data(this).shape();
@@ -455,7 +498,7 @@ void PoolingLayer::ComputeFeature(int flag, Metric* perf) {
auto data = Tensor4(&data_);
if (pool_ == PoolingProto_PoolMethod_MAX)
data = expr::pool<red::maximum>(src, kernel_, stride_);
- else if (pool_ == PoolingProto_PoolMethod_AVE)
+ else if (pool_ == PoolingProto_PoolMethod_AVG)
data = expr::pool<red::sum>(src, kernel_, stride_)
* (1.0f / (kernel_ * kernel_));
}
@@ -471,11 +514,44 @@ void PoolingLayer::ComputeGradient(int flag, Metric* perf) {
auto grad = Tensor4(&grad_);
if (pool_ == PoolingProto_PoolMethod_MAX)
gsrc = expr::unpool<red::maximum>(src, data, grad, kernel_, stride_);
- else if (pool_ == PoolingProto_PoolMethod_AVE)
+ else if (pool_ == PoolingProto_PoolMethod_AVG)
gsrc = expr::unpool<red::sum>(src, data, grad, kernel_, stride_)
* (1.0f / (kernel_ * kernel_));
}
+/***************** Implementation of CPoolingLayer ***************/
+
+void CPoolingLayer::Setup(const LayerProto& proto, int npartitions) {
+ PoolingLayer::Setup(proto, npartitions);
+ if(pool_ == PoolingProto_PoolMethod_MAX)
+ mask_.ReshapeLike(data_);
+}
+void CPoolingLayer::ComputeFeature(int flag, Metric* perf) {
+ if(pool_ == PoolingProto_PoolMethod_MAX)
+ ForwardMaxPooling(srclayers_[0]->mutable_data(this)->mutable_cpu_data(),
+ batchsize_, channels_, height_, width_, kernel_, kernel_, pad_, pad_,
+ stride_, stride_, data_.mutable_cpu_data(), mask_.mutable_cpu_data());
+ else if(pool_ == PoolingProto_PoolMethod_AVG)
+ ForwardAvgPooling(srclayers_[0]->mutable_data(this)->mutable_cpu_data(),
+ batchsize_, channels_, height_, width_, kernel_, kernel_, pad_, pad_,
+ stride_, stride_, data_.mutable_cpu_data());
+ else
+ LOG(FATAL) << "unknow pooling method";
+}
+
+void CPoolingLayer::ComputeGradient(int flag, Metric* perf) {
+ if(pool_ == PoolingProto_PoolMethod_MAX)
+ BackwardMaxPooling(grad_.cpu_data(), mask_.cpu_data(), batchsize_,
+ channels_, height_, width_, kernel_, kernel_, pad_, pad_,
+ stride_, stride_,srclayers_[0]->mutable_grad(this)->mutable_cpu_data());
+ else if(pool_ == PoolingProto_PoolMethod_AVG)
+ BackwardAvgPooling(grad_.cpu_data(), batchsize_,
+ channels_, height_, width_, kernel_, kernel_, pad_, pad_,
+ stride_, stride_,srclayers_[0]->mutable_grad(this)->mutable_cpu_data());
+ else
+ LOG(FATAL) << "unknow pooling method";
+}
+
/***************** Implementation for ReLULayer *****************************/
void ReLULayer::Setup(const LayerProto& proto, int npartitions) {
Layer::Setup(proto, npartitions);
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/src/proto/job.proto
----------------------------------------------------------------------
diff --git a/src/proto/job.proto b/src/proto/job.proto
index 9adae6d..7861eae 100644
--- a/src/proto/job.proto
+++ b/src/proto/job.proto
@@ -389,7 +389,7 @@ message PoolingProto {
required int32 kernel= 1;
enum PoolMethod {
MAX = 0;
- AVE = 1;
+ AVG = 1;
}
// The pooling method
optional PoolMethod pool = 30 [default = MAX];
@@ -514,6 +514,8 @@ enum LayerType {
// Neuron layers
// - Feature transformation
kConvolution = 1;
+ kCConvolution = 27;
+ kCPooling = 28;
kDropout = 4;
kInnerProduct = 5;
kLRN = 6;
@@ -535,6 +537,7 @@ enum LayerType {
kSlice = 12;
kSplit = 13;
+
// Indicate the user defined layer. Users should configure user_type
kUserLayer = 102;
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/src/trainer/trainer.cc
----------------------------------------------------------------------
diff --git a/src/trainer/trainer.cc b/src/trainer/trainer.cc
index 1d78c37..b6dc729 100644
--- a/src/trainer/trainer.cc
+++ b/src/trainer/trainer.cc
@@ -481,7 +481,6 @@ const vector<Msg*> Trainer::HandleUpdate(ParamEntry *entry, Msg** msg) {
mshadow::Tensor<mshadow::cpu,1> grad((*it)->mutable_cpu_grad(), shape);
sum += grad;
}
- sum /= entry->num_total;
}
int step = (*msg)->trgt_version();
GenMsgs(kUpdate, step, entry, *msg, &ret);
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/src/utils/blob.cc
----------------------------------------------------------------------
diff --git a/src/utils/blob.cc b/src/utils/blob.cc
index fd402a8..3df1aef 100644
--- a/src/utils/blob.cc
+++ b/src/utils/blob.cc
@@ -1,5 +1,5 @@
/**
- * The code is adapted from that of Caffe whose license is attached.
+ * The code is adapted from Caffe whose license is attached.
*
* COPYRIGHT
* All contributions by the University of California:
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/6d59eecf/src/utils/common.cc
----------------------------------------------------------------------
diff --git a/src/utils/common.cc b/src/utils/common.cc
index d13faea..3c3dc39 100644
--- a/src/utils/common.cc
+++ b/src/utils/common.cc
@@ -1,32 +1,70 @@
+/**
+ * The some functions in this file are adapted from Caffe whose license
+ * is attached.
+ *
+ * COPYRIGHT
+ * All contributions by the University of California:
+ * Copyright (c) 2014, The Regents of the University of California (Regents)
+ * All rights reserved.
+ * All other contributions:
+ * Copyright (c) 2014, the respective contributors
+ * All rights reserved.
+ * Caffe uses a shared copyright model: each contributor holds copyright over
+ * their contributions to Caffe. The project versioning records all such
+ * contribution and copyright details. If a contributor wants to further mark
+ * their specific copyright on a particular contribution, they should indicate
+ * their copyright solely in the commit message of the change when it is
+ * committed.
+ * LICENSE
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ * 1. Redistributions of source code must retain the above copyright notice, this
+ * list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright notice,
+ * this list of conditions and the following disclaimer in the documentation
+ * and/or other materials provided with the distribution.
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+ * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+ * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ * CONTRIBUTION AGREEMENT
+ * By contributing to the BVLC/caffe repository through pull-request, comment,
+ * or otherwise, the contributor releases their content to the
+ * license and copyright terms herein.
+ */
#include "utils/common.h"
-#include <arpa/inet.h>
-#include <fcntl.h>
-#include <glog/logging.h>
-#include <google/protobuf/io/coded_stream.h>
-#include <google/protobuf/io/zero_copy_stream_impl.h>
-#include <google/protobuf/text_format.h>
-#include <stdarg.h>
-#include <stdio.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <sys/stat.h>
#include <sys/types.h>
+
#include <netinet/in.h>
#include <net/if.h>
+#include <arpa/inet.h>
+
+#include <stdarg.h>
+#include <stdio.h>
#include <time.h>
#include <unistd.h>
+#include <fcntl.h>
+#include <cfloat>
+
+#include <glog/logging.h>
+#include <google/protobuf/io/coded_stream.h>
+#include <google/protobuf/io/zero_copy_stream_impl.h>
+#include <google/protobuf/text_format.h>
namespace singa {
using std::string;
using std::vector;
-using google::protobuf::io::CodedInputStream;
-using google::protobuf::io::FileInputStream;
-using google::protobuf::io::FileOutputStream;
-using google::protobuf::io::ZeroCopyInputStream;
-using google::protobuf::Message;
-
const int kBufLen = 1024;
string IntVecToString(const vector<int>& vec) {
@@ -56,42 +94,7 @@ string StringPrintf(string fmt, ...) {
return result;
}
-// the proto related functions are from Caffe.
-void ReadProtoFromTextFile(const char* filename, Message* proto) {
- int fd = open(filename, O_RDONLY);
- CHECK_NE(fd, -1) << "File not found: " << filename;
- FileInputStream* input = new FileInputStream(fd);
- CHECK(google::protobuf::TextFormat::Parse(input, proto));
- delete input;
- close(fd);
-}
-void WriteProtoToTextFile(const Message& proto, const char* filename) {
- int fd = open(filename, O_WRONLY | O_CREAT, 0644);
- FileOutputStream* output = new FileOutputStream(fd);
- CHECK(google::protobuf::TextFormat::Print(proto, output));
- delete output;
- close(fd);
-}
-
-void ReadProtoFromBinaryFile(const char* filename, Message* proto) {
- int fd = open(filename, O_RDONLY);
- CHECK_NE(fd, -1) << "File not found: " << filename;
- ZeroCopyInputStream* raw_input = new FileInputStream(fd);
- CodedInputStream* coded_input = new CodedInputStream(raw_input);
- // upper limit 512MB, warning threshold 256MB
- coded_input->SetTotalBytesLimit(536870912, 268435456);
- CHECK(proto->ParseFromCodedStream(coded_input));
- delete coded_input;
- delete raw_input;
- close(fd);
-}
-
-void WriteProtoToBinaryFile(const Message& proto, const char* filename) {
- int fd = open(filename, O_CREAT|O_WRONLY|O_TRUNC, 0644);
- CHECK_NE(fd, -1) << "File cannot open: " << filename;
- CHECK(proto.SerializeToFileDescriptor(fd));
-}
int ArgPos(int argc, char** arglist, const char* arg) {
for (int i = 0; i < argc; i++) {
@@ -293,4 +296,247 @@ void Metric::ParseFrom(const string& msg) {
}
}
+
+/*************Below functions are adapted from Caffe ************/
+using google::protobuf::io::CodedInputStream;
+using google::protobuf::io::FileInputStream;
+using google::protobuf::io::FileOutputStream;
+using google::protobuf::io::ZeroCopyInputStream;
+
+
+void Im2col(const float* data_im, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* data_col) {
+ int height_col = (height + 2 * pad_h - kernel_h) / stride_h + 1;
+ int width_col = (width + 2 * pad_w - kernel_w) / stride_w + 1;
+ int channels_col = channels * kernel_h * kernel_w;
+ for (int c = 0; c < channels_col; ++c) {
+ int w_offset = c % kernel_w;
+ int h_offset = (c / kernel_w) % kernel_h;
+ int c_im = c / kernel_h / kernel_w;
+ for (int h = 0; h < height_col; ++h) {
+ for (int w = 0; w < width_col; ++w) {
+ int h_pad = h * stride_h - pad_h + h_offset;
+ int w_pad = w * stride_w - pad_w + w_offset;
+ if (h_pad >= 0 && h_pad < height && w_pad >= 0 && w_pad < width)
+ data_col[(c * height_col + h) * width_col + w] =
+ data_im[(c_im * height + h_pad) * width + w_pad];
+ else
+ data_col[(c * height_col + h) * width_col + w] = 0;
+ }
+ }
+ }
+}
+
+void Col2im(const float* data_col, const int channels,
+ const int height, const int width, const int patch_h, const int patch_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* data_im) {
+ memset(data_im, 0, height * width * channels * sizeof(float));
+ int height_col = (height + 2 * pad_h - patch_h) / stride_h + 1;
+ int width_col = (width + 2 * pad_w - patch_w) / stride_w + 1;
+ int channels_col = channels * patch_h * patch_w;
+ for (int c = 0; c < channels_col; ++c) {
+ int w_offset = c % patch_w;
+ int h_offset = (c / patch_w) % patch_h;
+ int c_im = c / patch_h / patch_w;
+ for (int h = 0; h < height_col; ++h) {
+ for (int w = 0; w < width_col; ++w) {
+ int h_pad = h * stride_h - pad_h + h_offset;
+ int w_pad = w * stride_w - pad_w + w_offset;
+ if (h_pad >= 0 && h_pad < height && w_pad >= 0 && w_pad < width)
+ data_im[(c_im * height + h_pad) * width + w_pad] +=
+ data_col[(c * height_col + h) * width_col + w];
+ }
+ }
+ }
+}
+
+void ForwardMaxPooling(const float* bottom, const int num, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* top, float* mask) {
+ int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ int top_count = num * top_height * top_width * channels;
+ for (int i = 0; i < top_count; i++) {
+ mask[i] = -1;
+ top[i] = -FLT_MAX;
+ }
+ const int bottom_offset = height * width;
+ const int top_offset = top_height * top_width;
+ // The main loop
+ for (int n = 0; n < num; ++n) {
+ for (int c = 0; c < channels; ++c) {
+ for (int ph = 0; ph < top_height; ++ph) {
+ for (int pw = 0; pw < top_width; ++pw) {
+ int hstart = ph * stride_h - pad_h;
+ int wstart = pw * stride_w - pad_w;
+ int hend = std::min(hstart + kernel_h, height);
+ int wend = std::min(wstart + kernel_w, width);
+ hstart = std::max(hstart, 0);
+ wstart = std::max(wstart, 0);
+ const int top_index = ph * top_width + pw;
+ for (int h = hstart; h < hend; ++h) {
+ for (int w = wstart; w < wend; ++w) {
+ const int index = h * width + w;
+ if (bottom[index] > top[top_index]) {
+ top[top_index] = bottom[index];
+ mask[top_index] = index;
+ }
+ }
+ }
+ }
+ }
+ // compute offset
+ bottom += bottom_offset;
+ top += top_offset;
+ mask += top_offset;
+ }
+ }
+}
+
+void BackwardMaxPooling(const float* top, const float* mask, const int num,
+ const int channels, const int height, const int width,
+ const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,
+ const int stride_h, const int stride_w,
+ float* bottom) {
+ int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ const int top_offset = top_height * top_width;
+ const int bottom_offset = height * width;
+ memset(bottom, 0, sizeof(float) * num * channels * bottom_offset);
+ for (int n = 0; n < num; ++n) {
+ for (int c = 0; c < channels; ++c) {
+ for (int ph = 0; ph < top_height; ++ph) {
+ for (int pw = 0; pw < top_width; ++pw) {
+ const int top_idx = ph * top_width + pw;
+ const int bottom_idx = static_cast<int>(mask[top_idx]);
+ bottom[bottom_idx] += top[top_idx];
+ }
+ }
+ top += top_offset;
+ mask += top_offset;
+ bottom += bottom_offset;
+ }
+ }
+}
+
+void ForwardAvgPooling(const float* bottom, const int num, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* top) {
+ int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ int top_count = num * top_height * top_width * channels;
+ for (int i = 0; i < top_count; i++) {
+ top[i] = 0;
+ }
+ const int bottom_offset = height * width;
+ const int top_offset = top_height * top_width;
+ // The main loop
+ for (int n = 0; n < num; ++n) {
+ for (int c = 0; c < channels; ++c) {
+ for (int ph = 0; ph < top_height; ++ph) {
+ for (int pw = 0; pw < top_width; ++pw) {
+ int hstart = ph * stride_h - pad_h;
+ int wstart = pw * stride_w - pad_w;
+ int hend = std::min(hstart + kernel_h, height+pad_h);
+ int wend = std::min(wstart + kernel_w, width+pad_w);
+ int pool_size = (hend-hstart) * (wend-wstart);
+ hstart = std::max(hstart, 0);
+ wstart = std::max(wstart, 0);
+ hend = std::min(hend, height);
+ wend = std::min(wend, width);
+ const int top_index = ph * top_width + pw;
+ for (int h = hstart; h < hend; ++h) {
+ for (int w = wstart; w < wend; ++w) {
+ const int index = h * width + w;
+ top[top_index] += bottom[index];
+ }
+ }
+ top[top_index] /= pool_size;
+ }
+ }
+ // compute offset
+ bottom += bottom_offset;
+ top += top_offset;
+ }
+ }
+}
+
+void BackwardAvgPooling(const float* top, const int num, const int channels,
+ const int height, const int width, const int kernel_h, const int kernel_w,
+ const int pad_h, const int pad_w, const int stride_h, const int stride_w,
+ float* bottom) {
+ int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ const int top_offset = top_height * top_width;
+ const int bottom_offset = height * width;
+ memset(bottom, 0, sizeof(float) * num * channels * bottom_offset);
+ for (int n = 0; n < num; ++n) {
+ for (int c = 0; c < channels; ++c) {
+ for (int ph = 0; ph < top_height; ++ph) {
+ for (int pw = 0; pw < top_width; ++pw) {
+ int hstart = ph * stride_h - pad_h;
+ int wstart = pw * stride_w - pad_w;
+ int hend = std::min(hstart + kernel_h, height+pad_h);
+ int wend = std::min(wstart + kernel_w, width+pad_w);
+ int pool_size = (hend-hstart) * (wend-wstart);
+ hstart = std::max(hstart, 0);
+ wstart = std::max(wstart, 0);
+ hend = std::min(hend, height);
+ wend = std::min(wend, width);
+ const int top_index = ph * top_width + pw;
+ for (int h = hstart; h < hend; ++h) {
+ for (int w = wstart; w < wend; ++w) {
+ const int index = h * width + w;
+ bottom[index] += top[top_index] / pool_size;
+ }
+ }
+
+ }
+ }
+ top += top_offset;
+ bottom += bottom_offset;
+ }
+ }
+}
+
+void ReadProtoFromTextFile(const char* filename, Message* proto) {
+ int fd = open(filename, O_RDONLY);
+ CHECK_NE(fd, -1) << "File not found: " << filename;
+ FileInputStream* input = new FileInputStream(fd);
+ CHECK(google::protobuf::TextFormat::Parse(input, proto));
+ delete input;
+ close(fd);
+}
+
+void WriteProtoToTextFile(const Message& proto, const char* filename) {
+ int fd = open(filename, O_WRONLY | O_CREAT, 0644);
+ FileOutputStream* output = new FileOutputStream(fd);
+ CHECK(google::protobuf::TextFormat::Print(proto, output));
+ delete output;
+ close(fd);
+}
+
+void ReadProtoFromBinaryFile(const char* filename, Message* proto) {
+ int fd = open(filename, O_RDONLY);
+ CHECK_NE(fd, -1) << "File not found: " << filename;
+ ZeroCopyInputStream* raw_input = new FileInputStream(fd);
+ CodedInputStream* coded_input = new CodedInputStream(raw_input);
+ // upper limit 512MB, warning threshold 256MB
+ coded_input->SetTotalBytesLimit(536870912, 268435456);
+ CHECK(proto->ParseFromCodedStream(coded_input));
+ delete coded_input;
+ delete raw_input;
+ close(fd);
+}
+
+void WriteProtoToBinaryFile(const Message& proto, const char* filename) {
+ int fd = open(filename, O_CREAT|O_WRONLY|O_TRUNC, 0644);
+ CHECK_NE(fd, -1) << "File cannot open: " << filename;
+ CHECK(proto.SerializeToFileDescriptor(fd));
+}
} // namespace singa
[11/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/communication/msg.cc
----------------------------------------------------------------------
diff --git a/src/communication/msg.cc b/src/communication/msg.cc
index 21d11bb..6042057 100644
--- a/src/communication/msg.cc
+++ b/src/communication/msg.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "communication/msg.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/communication/socket.cc
----------------------------------------------------------------------
diff --git a/src/communication/socket.cc b/src/communication/socket.cc
index f21edc1..60e1cc1 100644
--- a/src/communication/socket.cc
+++ b/src/communication/socket.cc
@@ -1,3 +1,23 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
#include "communication/socket.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/driver.cc
----------------------------------------------------------------------
diff --git a/src/driver.cc b/src/driver.cc
index 88bf4aa..28d21c2 100644
--- a/src/driver.cc
+++ b/src/driver.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "driver.h"
#include <cblas.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/main.cc
----------------------------------------------------------------------
diff --git a/src/main.cc b/src/main.cc
index 1f3f4a4..5e94de4 100644
--- a/src/main.cc
+++ b/src/main.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "singa.h"
/**
* \file main.cc provides an example main func.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/connection_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/connection_layer.cc b/src/neuralnet/connection_layer.cc
index e247161..1ba2d95 100644
--- a/src/neuralnet/connection_layer.cc
+++ b/src/neuralnet/connection_layer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "neuralnet/connection_layer.h"
namespace singa {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/input_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/input_layer.cc b/src/neuralnet/input_layer.cc
index f7167da..cf88c15 100644
--- a/src/neuralnet/input_layer.cc
+++ b/src/neuralnet/input_layer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "neuralnet/input_layer.h"
#include "mshadow/tensor.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/layer.cc b/src/neuralnet/layer.cc
index f38d592..d818533 100644
--- a/src/neuralnet/layer.cc
+++ b/src/neuralnet/layer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "neuralnet/layer.h"
#include <cblas.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/loss_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/loss_layer.cc b/src/neuralnet/loss_layer.cc
index f9b80a9..d8fd92b 100644
--- a/src/neuralnet/loss_layer.cc
+++ b/src/neuralnet/loss_layer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "neuralnet/loss_layer.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/neuralnet.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/neuralnet.cc b/src/neuralnet/neuralnet.cc
index 5f04372..2f50ec5 100644
--- a/src/neuralnet/neuralnet.cc
+++ b/src/neuralnet/neuralnet.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include <algorithm>
#include <queue>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/neuron_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/neuron_layer.cc b/src/neuralnet/neuron_layer.cc
index a98b40d..9e7831a 100644
--- a/src/neuralnet/neuron_layer.cc
+++ b/src/neuralnet/neuron_layer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "neuralnet/neuron_layer.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/neuralnet/output_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/output_layer.cc b/src/neuralnet/output_layer.cc
index 535480e..01ca0b6 100644
--- a/src/neuralnet/output_layer.cc
+++ b/src/neuralnet/output_layer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "neuralnet/output_layer.h"
namespace singa {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/proto/common.proto
----------------------------------------------------------------------
diff --git a/src/proto/common.proto b/src/proto/common.proto
index e166299..05e593f 100644
--- a/src/proto/common.proto
+++ b/src/proto/common.proto
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
package singa;
enum MsgType {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/proto/job.proto
----------------------------------------------------------------------
diff --git a/src/proto/job.proto b/src/proto/job.proto
index 80998e1..dc202d9 100644
--- a/src/proto/job.proto
+++ b/src/proto/job.proto
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
package singa;
// To start a training job, all we need is a JobProto object.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/proto/singa.proto
----------------------------------------------------------------------
diff --git a/src/proto/singa.proto b/src/proto/singa.proto
index 193c8b7..6e12d25 100644
--- a/src/proto/singa.proto
+++ b/src/proto/singa.proto
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
package singa;
message SingaProto {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/test/test_cluster.cc
----------------------------------------------------------------------
diff --git a/src/test/test_cluster.cc b/src/test/test_cluster.cc
index cfda3b4..5a91f21 100644
--- a/src/test/test_cluster.cc
+++ b/src/test/test_cluster.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "gtest/gtest.h"
#include "utils/cluster.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/test/test_msg.cc
----------------------------------------------------------------------
diff --git a/src/test/test_msg.cc b/src/test/test_msg.cc
index 61263d9..e8b2801 100644
--- a/src/test/test_msg.cc
+++ b/src/test/test_msg.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "gtest/gtest.h"
#include "communication/msg.h"
using namespace singa;
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/test/test_neuralnet.cc
----------------------------------------------------------------------
diff --git a/src/test/test_neuralnet.cc b/src/test/test_neuralnet.cc
index 9317ce0..d7787ee 100644
--- a/src/test/test_neuralnet.cc
+++ b/src/test/test_neuralnet.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "gtest/gtest.h"
#include "neuralnet/neuralnet.h"
using namespace singa;
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/test/test_paramslicer.cc
----------------------------------------------------------------------
diff --git a/src/test/test_paramslicer.cc b/src/test/test_paramslicer.cc
index 759c18b..abfaecd 100644
--- a/src/test/test_paramslicer.cc
+++ b/src/test/test_paramslicer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/param.h"
#include "gtest/gtest.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/test/test_shard.cc
----------------------------------------------------------------------
diff --git a/src/test/test_shard.cc b/src/test/test_shard.cc
index dcc3026..d36a40b 100644
--- a/src/test/test_shard.cc
+++ b/src/test/test_shard.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include <sys/stat.h>
#include "gtest/gtest.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/trainer/server.cc
----------------------------------------------------------------------
diff --git a/src/trainer/server.cc b/src/trainer/server.cc
index 601a837..18fe7d2 100644
--- a/src/trainer/server.cc
+++ b/src/trainer/server.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include <thread>
#include <chrono>
#include "mshadow/tensor.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/trainer/trainer.cc
----------------------------------------------------------------------
diff --git a/src/trainer/trainer.cc b/src/trainer/trainer.cc
index b02ef3e..4a4c183 100644
--- a/src/trainer/trainer.cc
+++ b/src/trainer/trainer.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include <thread>
#include <vector>
#include <map>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/trainer/worker.cc
----------------------------------------------------------------------
diff --git a/src/trainer/worker.cc b/src/trainer/worker.cc
index bcdde55..0e284e1 100644
--- a/src/trainer/worker.cc
+++ b/src/trainer/worker.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include <glog/logging.h>
#include <thread>
#include <chrono>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/blob.cc
----------------------------------------------------------------------
diff --git a/src/utils/blob.cc b/src/utils/blob.cc
index 3df1aef..24c7f54 100644
--- a/src/utils/blob.cc
+++ b/src/utils/blob.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
/**
* The code is adapted from Caffe whose license is attached.
*
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/cluster.cc
----------------------------------------------------------------------
diff --git a/src/utils/cluster.cc b/src/utils/cluster.cc
index a1716a1..3b09417 100644
--- a/src/utils/cluster.cc
+++ b/src/utils/cluster.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/cluster.h"
#include <sys/stat.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/cluster_rt.cc
----------------------------------------------------------------------
diff --git a/src/utils/cluster_rt.cc b/src/utils/cluster_rt.cc
index 5cb670e..265e80b 100644
--- a/src/utils/cluster_rt.cc
+++ b/src/utils/cluster_rt.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/cluster_rt.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/common.cc
----------------------------------------------------------------------
diff --git a/src/utils/common.cc b/src/utils/common.cc
index 4cf9a89..75974d1 100644
--- a/src/utils/common.cc
+++ b/src/utils/common.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
/**
* The some functions in this file are adapted from Caffe whose license
* is attached.
@@ -38,6 +59,7 @@
* or otherwise, the contributor releases their content to the
* license and copyright terms herein.
*/
+
#include "utils/common.h"
#include <sys/ioctl.h>
@@ -94,8 +116,6 @@ string StringPrintf(string fmt, ...) {
return result;
}
-
-
int ArgPos(int argc, char** arglist, const char* arg) {
for (int i = 0; i < argc; i++) {
if (strcmp(arglist[i], arg) == 0) {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/data_shard.cc
----------------------------------------------------------------------
diff --git a/src/utils/data_shard.cc b/src/utils/data_shard.cc
index 1dc61d2..77e45fa 100644
--- a/src/utils/data_shard.cc
+++ b/src/utils/data_shard.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/data_shard.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/graph.cc
----------------------------------------------------------------------
diff --git a/src/utils/graph.cc b/src/utils/graph.cc
index 192ac3c..27680b7 100644
--- a/src/utils/graph.cc
+++ b/src/utils/graph.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/graph.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/param.cc
----------------------------------------------------------------------
diff --git a/src/utils/param.cc b/src/utils/param.cc
index f2c6dd6..07c238c 100644
--- a/src/utils/param.cc
+++ b/src/utils/param.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/param.h"
#include <glog/logging.h>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/tool.cc
----------------------------------------------------------------------
diff --git a/src/utils/tool.cc b/src/utils/tool.cc
index 084ebea..ba453b2 100644
--- a/src/utils/tool.cc
+++ b/src/utils/tool.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include <glog/logging.h>
#include <algorithm>
#include <fstream>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/src/utils/updater.cc
----------------------------------------------------------------------
diff --git a/src/utils/updater.cc b/src/utils/updater.cc
index 2c4b6c3..a1742c4 100644
--- a/src/utils/updater.cc
+++ b/src/utils/updater.cc
@@ -1,3 +1,24 @@
+/************************************************************
+*
+* Licensed to the Apache Software Foundation (ASF) under one
+* or more contributor license agreements. See the NOTICE file
+* distributed with this work for additional information
+* regarding copyright ownership. The ASF licenses this file
+* to you under the Apache License, Version 2.0 (the
+* "License"); you may not use this file except in compliance
+* with the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing,
+* software distributed under the License is distributed on an
+* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+* KIND, either express or implied. See the License for the
+* specific language governing permissions and limitations
+* under the License.
+*
+*************************************************************/
+
#include "utils/updater.h"
#include "mshadow/cxxnet_op.h"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/thirdparty/install.sh
----------------------------------------------------------------------
diff --git a/thirdparty/install.sh b/thirdparty/install.sh
index a70b964..7bce5af 100755
--- a/thirdparty/install.sh
+++ b/thirdparty/install.sh
@@ -1,3 +1,22 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
#!/usr/bin/env bash
function install_cmake()
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/tool/graph.py
----------------------------------------------------------------------
diff --git a/tool/graph.py b/tool/graph.py
index 17aaea7..74ad98f 100644
--- a/tool/graph.py
+++ b/tool/graph.py
@@ -1,3 +1,23 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
+
import sys
import pygraphviz
import networkx as nx
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/tool/node.sh
----------------------------------------------------------------------
diff --git a/tool/node.sh b/tool/node.sh
index 74e0d8a..808ea03 100755
--- a/tool/node.sh
+++ b/tool/node.sh
@@ -1,3 +1,22 @@
+#/**
+# * Copyright 2015 The Apache Software Foundation
+# *
+# * Licensed to the Apache Software Foundation (ASF) under one
+# * or more contributor license agreements. See the NOTICE file
+# * distributed with this work for additional information
+# * regarding copyright ownership. The ASF licenses this file
+# * to you under the Apache License, Version 2.0 (the
+# * "License"); you may not use this file except in compliance
+# * with the License. You may obtain a copy of the License at
+# *
+# * http://www.apache.org/licenses/LICENSE-2.0
+# *
+# * Unless required by applicable law or agreed to in writing, software
+# * distributed under the License is distributed on an "AS IS" BASIS,
+# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# * See the License for the specific language governing permissions and
+# * limitations under the License.
+# */
#!/bin/bash
if [[ $# < 2 || ! -f $2 ]]
then
[17/21] incubator-singa git commit: SINGA-10 Add Support for
Recurrent Neural Networks (RNN)
Posted by wa...@apache.org.
SINGA-10 Add Support for Recurrent Neural Networks (RNN)
Draft upper layers for rnnlm;
Compile using Makefile.example;
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/e53a23c6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/e53a23c6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/e53a23c6
Branch: refs/heads/tutorial
Commit: e53a23c6ba0c22ba0b840fb4ce53fe42a35e43ac
Parents: 1791442
Author: Wei Wang <wa...@comp.nus.edu.sg>
Authored: Sun Sep 13 14:07:41 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Wed Sep 16 11:39:16 2015 +0800
----------------------------------------------------------------------
examples/rnnlm/Makefile.example | 7 ++
examples/rnnlm/main.cc | 23 ++++
examples/rnnlm/rnnlm.cc | 209 +++++++++++++++++++++++++++++++++++
examples/rnnlm/rnnlm.h | 89 +++++++++++++++
examples/rnnlm/rnnlm.proto | 18 +++
include/utils/common.h | 1 +
src/utils/common.cc | 5 +-
7 files changed, 351 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/examples/rnnlm/Makefile.example
----------------------------------------------------------------------
diff --git a/examples/rnnlm/Makefile.example b/examples/rnnlm/Makefile.example
new file mode 100644
index 0000000..5eeca78
--- /dev/null
+++ b/examples/rnnlm/Makefile.example
@@ -0,0 +1,7 @@
+MSHADOW_FLAGS :=-DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0
+
+all:
+ protoc --proto_path=../../src/proto --proto_path=. --cpp_out=. rnnlm.proto
+ $(CXX) main.cc rnnlm.cc rnnlm.pb.cc $(MSHADOW_FLAGS) -std=c++11 -lsinga -lglog -lprotobuf -lopenblas -I../../include\
+ -I../../include/proto/ -L../../.libs/ -L/usr/local -Wl,-unresolved-symbols=ignore-in-shared-libs -Wl,-rpath=../../.libs/\
+ -o rnnlm.bin
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/examples/rnnlm/main.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/main.cc b/examples/rnnlm/main.cc
new file mode 100644
index 0000000..690c158
--- /dev/null
+++ b/examples/rnnlm/main.cc
@@ -0,0 +1,23 @@
+#include <string>
+#include "singa.h"
+#include "rnnlm.h"
+#include "rnnlm.pb.h"
+
+int main(int argc, char **argv) {
+ singa::Driver driver;
+ driver.Init(argc, argv);
+
+ //if -resume in argument list, set resume to true; otherwise false
+ int resume_pos = singa::ArgPos(argc, argv, "-resume");
+ bool resume = (resume_pos != -1);
+
+ // register all layers for rnnlm
+ driver.RegisterLayer<singa::EmbeddingLayer, std::string>("kEmbedding");
+ driver.RegisterLayer<singa::HiddenLayer, std::string>("kHidden");
+ driver.RegisterLayer<singa::OutputLayer, std::string>("kOutput");
+
+ singa::JobProto jobConf = driver.job_conf();
+
+ driver.Submit(resume, jobConf);
+ return 0;
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/examples/rnnlm/rnnlm.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.cc b/examples/rnnlm/rnnlm.cc
new file mode 100644
index 0000000..ddb0f63
--- /dev/null
+++ b/examples/rnnlm/rnnlm.cc
@@ -0,0 +1,209 @@
+#include "rnnlm.h"
+#include "rnnlm.pb.h"
+#include "mshadow/tensor.h"
+#include "mshadow/cxxnet_op.h"
+
+namespace singa {
+using namespace mshadow;
+using mshadow::cpu;
+
+using mshadow::Shape;
+using mshadow::Shape1;
+using mshadow::Shape2;
+using mshadow::Tensor;
+
+
+inline Tensor<cpu, 2> RTensor2(Blob<float>* blob) {
+ const vector<int>& shape = blob->shape();
+ Tensor<cpu, 2> tensor(blob->mutable_cpu_data(),
+ Shape2(shape[0], blob->count() / shape[0]));
+ return tensor;
+}
+
+inline Tensor<cpu, 1> RTensor1(Blob<float>* blob) {
+ Tensor<cpu, 1> tensor(blob->mutable_cpu_data(), Shape1(blob->count()));
+ return tensor;
+}
+
+/*******EmbeddingLayer**************/
+EmbeddingLayer::~EmbeddingLayer() {
+ delete embed_;
+}
+
+void EmbeddingLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 1);
+ int max_window = srclayers_[0]->data(this).shape()[0];
+ word_dim_ = proto.GetExtension(embedding_conf).word_dim();
+ data_.Reshape(vector<int>{max_window, word_dim_});
+ grad_.ReshapeLike(data_);
+ vocab_size_ = proto.GetExtension(embedding_conf).vocab_size();
+ embed_ = Param::Create(proto.param(0));
+ embed_->Setup(vector<int>{vocab_size_, word_dim_});
+}
+
+void EmbeddingLayer::ComputeFeature(int flag, Metric* perf) {
+ window_ = static_cast<RNNLayer*>(srclayers_[0])->window();
+ auto words = RTensor2(&data_);
+ auto embed = RTensor2(embed_->mutable_data());
+ auto word_idx = RTensor1(srclayers_[0]->mutable_data(this));
+
+ for (int t = 0; t < window_; t++) {
+ int idx = static_cast<int>(word_idx[t]);
+ CHECK_GE(idx, 0);
+ CHECK_LT(idx, vocab_size_);
+ Copy(words[t], embed[idx]);
+ }
+}
+
+void EmbeddingLayer::ComputeGradient(int flag, Metric* perf) {
+ auto grad = RTensor2(&grad_);
+ auto gembed = RTensor2(embed_->mutable_grad());
+ auto word_idx = RTensor1(srclayers_[0]->mutable_data(this));
+ gembed = 0;
+ for (int t = 0; t < window_; t++) {
+ int idx = static_cast<int>(word_idx[t]);
+ Copy(gembed[idx], grad[t]);
+ }
+}
+/***********HiddenLayer**********/
+HiddenLayer::~HiddenLayer() {
+ delete weight_;
+}
+
+void HiddenLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 1);
+ const auto& innerproductData = srclayers_[0]->data(this);
+ data_.ReshapeLike(srclayers_[0]->data(this));
+ grad_.ReshapeLike(srclayers_[0]->grad(this));
+ int word_dim = data_.shape()[1];
+ weight_ = Param::Create(proto.param(0));
+ weight_->Setup(std::vector<int>{word_dim, word_dim});
+}
+
+// hid[t] = sigmoid(hid[t-1] * W + src[t])
+void HiddenLayer::ComputeFeature(int flag, Metric* perf) {
+ window_ = static_cast<RNNLayer*>(srclayers_[0])->window();
+ auto data = RTensor2(&data_);
+ auto src = RTensor2(srclayers_[0]->mutable_data(this));
+ auto weight = RTensor2(weight_->mutable_data());
+ for (int t = 0; t < window_; t++) { // Skip the 1st component
+ if (t == 0) {
+ data[t] = expr::F<op::sigmoid>(src[t]);
+ } else {
+ data[t] = dot(data[t - 1], weight);
+ data[t] += src[t];
+ data[t] = expr::F<op::sigmoid>(data[t]);
+ }
+ }
+}
+
+void HiddenLayer::ComputeGradient(int flag, Metric* perf) {
+ auto data = RTensor2(&data_);
+ auto grad = RTensor2(&grad_);
+ auto weight = RTensor2(weight_->mutable_data());
+ auto gweight = RTensor2(weight_->mutable_grad());
+ auto gsrc = RTensor2(srclayers_[0]->mutable_grad(this));
+ gweight = 0;
+ TensorContainer<cpu, 1> tmp(Shape1(data_.shape()[1]));
+ // Check!!
+ for (int t = window_ - 1; t >= 0; t--) {
+ if (t < window_ - 1) {
+ tmp = dot(grad[t + 1], weight.T());
+ grad[t] += tmp;
+ }
+ grad[t] = expr::F<op::sigmoid_grad>(data[t])* grad[t];
+ }
+ gweight = dot(data.Slice(0, window_-1).T(), grad.Slice(1, window_));
+ Copy(gsrc, grad);
+}
+
+/*********** 1-Implementation for OutputLayer **********/
+OutputLayer::~OutputLayer() {
+ delete word_weight_;
+ delete class_weight_;
+}
+
+void OutputLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 2);
+ const auto& src = srclayers_[0]->data(this);
+ int max_window = src.shape()[0];
+ int vdim = src.count() / max_window; // Dimension of input
+ int vocab_size = proto.GetExtension(output_conf).vocab_size();
+ int nclass = proto.GetExtension(output_conf).nclass();
+ word_weight_ = Param::Create(proto.param(0));
+ word_weight_->Setup(vector<int>{vocab_size, vdim});
+ class_weight_ = Param::Create(proto.param(0));
+ class_weight_->Setup(vector<int>{nclass, vdim});
+
+ pword_.resize(max_window);
+ pclass_.Reshape(vector<int>{max_window, nclass});
+}
+
+void OutputLayer::ComputeFeature(int flag, Metric* perf) {
+ window_ = static_cast<RNNLayer*>(srclayers_[0])->window();
+ auto pclass = RTensor2(&pclass_);
+ auto src = RTensor2(srclayers_[0]->mutable_data(this));
+ auto word_weight = RTensor2(word_weight_->mutable_data());
+ auto class_weight = RTensor2(class_weight_->mutable_data());
+ const float * label = srclayers_[1]->data(this).cpu_data();
+
+ float loss = 0.f, ppl =0.f;
+ for (int t = 0; t < window_; t++) {
+ int start = static_cast<int>(label[t * 4 + 0]);
+ int end = static_cast<int>(label[t * 4 + 1]);
+
+ auto wordWeight = word_weight.Slice(start, end);
+ pword_[t].Reshape(vector<int>{end-start});
+ auto pword = RTensor1(&pword_[t]);
+ pword = dot(src[t], wordWeight.T());
+ Softmax(pword, pword);
+
+ pclass[t] = dot(src[t], class_weight.T());
+ Softmax(pclass[t], pclass[t]);
+
+ int wid = static_cast<int>(label[t * 4 + 2]);
+ int cid = static_cast<int>(label[t * 4 + 3]);
+ loss += -log(std::max(pword[wid - start] * pclass[t][cid], FLT_MIN));
+ ppl += log10(std::max(pword[wid - start] * pclass[t][cid], FLT_MIN));
+ }
+
+ perf->Add("loss", loss, window_);
+ perf->Add("ppl before exp", ppl, window_);
+}
+
+void OutputLayer::ComputeGradient(int flag, Metric* perf) {
+ auto pclass = RTensor2(&pclass_);
+ auto src = RTensor2(srclayers_[0]->mutable_data(this));
+ auto gsrc = RTensor2(srclayers_[0]->mutable_grad(this));
+ auto word_weight = RTensor2(word_weight_->mutable_data());
+ auto gword_weight = RTensor2(word_weight_->mutable_grad());
+ auto class_weight = RTensor2(class_weight_->mutable_data());
+ auto gclass_weight = RTensor2(class_weight_->mutable_grad());
+ const float * label = srclayers_[1]->data(this).cpu_data();
+ gclass_weight = 0;
+ gword_weight = 0;
+ for (int t = 0; t < window_; t++) {
+ int start = static_cast<int>(label[t * 4 + 0]);
+ int end = static_cast<int>(label[t * 4 + 1]);
+ int wid = static_cast<int>(label[t * 4 + 2]);
+ int cid = static_cast<int>(label[t * 4 + 3]);
+ auto pword = RTensor1(&pword_[t]);
+
+ // gL/gclass_act
+ pclass[t][cid] -= 1.0;
+ // gL/gword_act
+ pword[wid] -= 1.0;
+
+ // gL/gword_weight
+ gword_weight.Slice(start, end) += dot(pword.FlatTo2D().T(), src[t].FlatTo2D());
+ // gL/gclass_weight
+ gclass_weight += dot(pclass[t].FlatTo2D().T(), src[t].FlatTo2D());
+
+ gsrc[t] = dot(pword, word_weight.Slice(start, end));
+ gsrc[t] += dot(pclass[t], class_weight);
+ }
+}
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/examples/rnnlm/rnnlm.h
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.h b/examples/rnnlm/rnnlm.h
new file mode 100644
index 0000000..14d947c
--- /dev/null
+++ b/examples/rnnlm/rnnlm.h
@@ -0,0 +1,89 @@
+#include "singa.h"
+namespace singa {
+
+/**
+ * Base RNN layer. May make it a base layer of SINGA.
+ */
+class RNNLayer : public NeuronLayer {
+ public:
+ /**
+ * The recurrent layers may be unrolled different times for different
+ * iterations, depending on the applications. For example, the ending word
+ * of a sentence may stop the unrolling; unrolling also stops when the max
+ * window size is reached. Every layer must reset window_ in its
+ * ComputeFeature function.
+ *
+ * @return the effective BPTT length, which is <= max_window.
+ */
+ inline int window() { return window_; }
+
+ protected:
+ //!< effect window size for BPTT
+ int window_;
+};
+
+/**
+ * Word embedding layer that get one row from the embedding matrix for each
+ * word based on the word index
+ */
+class EmbeddingLayer : public RNNLayer {
+ public:
+ ~EmbeddingLayer();
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+ const std::vector<Param*> GetParams() const override {
+ std::vector<Param*> params{embed_};
+ return params;
+ }
+
+
+ private:
+ int word_dim_;
+ int vocab_size_;
+ //!< word embedding matrix of size vocab_size_ x word_dim_
+ Param* embed_;
+};
+
+
+/**
+ * hid[t] = sigmoid(hid[t-1] * W + src[t])
+ */
+class HiddenLayer : public RNNLayer {
+ public:
+ ~HiddenLayer();
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+ const std::vector<Param*> GetParams() const override {
+ std::vector<Param*> params{weight_};
+ return params;
+ }
+
+
+ private:
+ Param* weight_;
+};
+
+/**
+ * p(word at t+1 is from class c) = softmax(src[t]*Wc)[c]
+ * p(w|c) = softmax(src[t]*Ww[Start(c):End(c)])
+ * p(word at t+1 is w)=p(word at t+1 is from class c)*p(w|c)
+ */
+class OutputLayer : public RNNLayer {
+ public:
+ ~OutputLayer();
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric *perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+ const std::vector<Param*> GetParams() const override {
+ std::vector<Param*> params{word_weight_, class_weight_};
+ return params;
+ }
+
+ private:
+ vector<Blob<float>> pword_;
+ Blob<float> pclass_;
+ Param* word_weight_, *class_weight_;
+};
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/examples/rnnlm/rnnlm.proto
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.proto b/examples/rnnlm/rnnlm.proto
new file mode 100644
index 0000000..35b6bc2
--- /dev/null
+++ b/examples/rnnlm/rnnlm.proto
@@ -0,0 +1,18 @@
+package singa;
+import "job.proto";
+
+
+message EmbeddingProto {
+ optional int32 word_dim = 1;
+ optional int32 vocab_size = 2;
+}
+
+message OutputProto {
+ optional int32 nclass = 1;
+ optional int32 vocab_size = 2;
+}
+
+extend LayerProto {
+ optional EmbeddingProto embedding_conf = 101;
+ optional OutputProto output_conf = 102;
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/include/utils/common.h
----------------------------------------------------------------------
diff --git a/include/utils/common.h b/include/utils/common.h
index 2be2715..3eb0bbd 100644
--- a/include/utils/common.h
+++ b/include/utils/common.h
@@ -95,6 +95,7 @@ class Metric {
* @param value metric value
*/
void Add(const std::string& name, float value);
+ void Add(const std::string& name, float value, int count);
/**
* reset all metric counter and value to 0
*/
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e53a23c6/src/utils/common.cc
----------------------------------------------------------------------
diff --git a/src/utils/common.cc b/src/utils/common.cc
index 75974d1..6dd40c8 100644
--- a/src/utils/common.cc
+++ b/src/utils/common.cc
@@ -267,11 +267,14 @@ Metric::Metric(const string& str) {
}
void Metric::Add(const string& name, float value) {
+ Add( name, value, 1);
+}
+void Metric::Add(const string& name, float value, int count) {
if (entry_.find(name) == entry_.end()) {
entry_[name] = std::make_pair(1, value);
} else {
auto& e = entry_.at(name);
- e.first += 1;
+ e.first += count;
e.second += value;
}
}
[05/21] incubator-singa git commit: SINGA-64 Add the test module for
utils/common
Posted by wa...@apache.org.
SINGA-64 Add the test module for utils/common
add test code for functions in utils/common
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/077d3804
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/077d3804
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/077d3804
Branch: refs/heads/tutorial
Commit: 077d3804fd0bf2d710b13b13ae76f03e439438b5
Parents: 5d7d53f
Author: ijingo <ij...@gmail.com>
Authored: Thu Aug 27 21:23:55 2015 +0800
Committer: ijingo <ij...@gmail.com>
Committed: Sun Sep 13 21:47:51 2015 +0800
----------------------------------------------------------------------
Makefile.am | 1 +
src/test/test_common.cc | 113 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 114 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/077d3804/Makefile.am
----------------------------------------------------------------------
diff --git a/Makefile.am b/Makefile.am
index baff947..6f3be4c 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -85,6 +85,7 @@ GTEST_SRCS := include/gtest/gtest-all.cc
GTEST_HRDS := include/gtest/gtest.h
TEST_SRCS := include/gtest/gtest_main.cc \
src/test/test_cluster.cc \
+ src/test/test_common.cc \
src/test/test_msg.cc \
src/test/test_neuralnet.cc \
src/test/test_paramslicer.cc \
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/077d3804/src/test/test_common.cc
----------------------------------------------------------------------
diff --git a/src/test/test_common.cc b/src/test/test_common.cc
new file mode 100644
index 0000000..b84e860
--- /dev/null
+++ b/src/test/test_common.cc
@@ -0,0 +1,113 @@
+#include "gtest/gtest.h"
+#include "utils/common.h"
+#include <unordered_map>
+#include <string>
+#include <vector>
+
+using std::string;
+using std::vector;
+using namespace singa;
+
+TEST(CommonTest, TestIntVecToString) {
+
+ vector<int> num_vec {2, 3, 5, 7, 11};
+ string str = "(2, 3, 5, 7, 11, )";
+ ASSERT_EQ(str, IntVecToString(num_vec));
+}
+
+TEST(CommonTest, TestStringPrintf) {
+ const char* str_a = "abc";
+ const char* str_b = "edfgh";
+ const char* str_c = " !@#";
+ const char* str_d = "1";
+ const char* str_e = "2";
+ const char* str_f = "3";
+
+ string fmt_a = "%s%s%s";
+ string fmt_b = "[%s] [%s] [%s] ";
+
+ string str_d_a = "abcedfgh !@#";
+ string str_d_b = "[1] [2] [3] ";
+
+ ASSERT_EQ(str_d_a, StringPrintf(fmt_a, str_a, str_b, str_c));
+ ASSERT_EQ(str_d_b, StringPrintf(fmt_b, str_d, str_e, str_f));
+}
+
+TEST(CommonTest, TestGCDLCM) {
+ int a = 2, b = 5, c = 10, d = 15;
+
+ ASSERT_EQ(1, gcd(a, b));
+ ASSERT_EQ(5, gcd(c, d));
+ ASSERT_EQ(10, LeastCommonMultiple(b, c));
+ ASSERT_EQ(30, LeastCommonMultiple(c, d));
+}
+
+TEST(CommonTest, TestMetric) {
+ string str, msg;
+ Metric metric;
+ metric.Add("a", 0.5);
+ metric.Add("b", 0.5);
+ metric.Add("a", 1.5);
+ str = metric.ToLogString();
+ msg = metric.ToString();
+ metric.Reset();
+ metric.ParseFrom(msg);
+ ASSERT_EQ(str, metric.ToLogString());
+}
+
+TEST(CommonTest, TestSlice) {
+ vector<vector<int>> slices_0;
+ vector<int> sizes {14112, 96, 256, 884736, 384};
+ ASSERT_EQ(slices_0, Slice(0, sizes));
+
+ vector<vector<int>> slices_1 {
+ { 14112 },
+ { 96 },
+ { 256 },
+ { 884736 },
+ { 384 },
+ };
+
+ vector<vector<int>> slices_2 {
+ { 14112 },
+ { 96 },
+ { 256 },
+ { 435328, 449408 },
+ { 384 },
+ };
+
+ vector<vector<int>> slices_4 {
+ { 14112 },
+ { 96 },
+ { 256 },
+ { 210432,224896,224896,224512 },
+ { 384 },
+ };
+
+ vector<vector<int>> slices_8 {
+ { 14112 },
+ { 96 },
+ { 256 },
+ { 97984,112448,112448,112448,112448,112448,112448,112064 },
+ { 384 },
+ };
+
+ ASSERT_EQ(slices_1, Slice(1, sizes));
+ ASSERT_EQ(slices_2, Slice(2, sizes));
+ ASSERT_EQ(slices_4, Slice(4, sizes));
+ ASSERT_EQ(slices_8, Slice(8, sizes));
+}
+
+TEST(CommonTest, TestPartitionSlices) {
+ vector<int> slices {
+ 97984,112448,112448,112448,112448,112448,112448,112064
+ };
+ vector<int> box_1 { 0, 0, 0, 0, 0, 0, 0, 0 };
+ vector<int> box_2 { 0, 0, 0, 0, 1, 1, 1, 1 };
+ vector<int> box_4 { 0, 0, 1, 1, 2, 2, 3, 3 };
+ vector<int> box_8 { 0, 1, 2, 3, 4, 5, 6, 7 };
+ ASSERT_EQ(box_1, PartitionSlices(1, slices));
+ ASSERT_EQ(box_2, PartitionSlices(2, slices));
+ ASSERT_EQ(box_4, PartitionSlices(4, slices));
+ ASSERT_EQ(box_8, PartitionSlices(8, slices));
+}
[13/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
SINGA-68 Preparation for Release -- License etc.
Added ASF License to all source files.
Removed gflags from LICENSE
Prepared NOTICE and DISCLAIMER file.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/74866478
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/74866478
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/74866478
Branch: refs/heads/tutorial
Commit: 748664780cb612d5e9324e4f3b45560e20b424be
Parents: ed9e373
Author: jinyangturbo <pk...@gmail.com>
Authored: Thu Sep 10 00:37:01 2015 -0700
Committer: wang sheng <wa...@gmail.com>
Committed: Wed Sep 16 11:09:25 2015 +0800
----------------------------------------------------------------------
DISCLAIMER | 10 +
LICENSE | 832 ++++++++++++++++++++++++++++
LICENSE.md | 864 ------------------------------
Makefile.example | 20 +
NOTICE | 12 +
README.md | 1 +
autogen.sh | 19 +
configure.ac | 19 +
examples/cifar10/Makefile.example | 20 +
examples/cifar10/create_shard.cc | 21 +
examples/cifar10/run.sh | 20 +
examples/mnist/Makefile.example | 21 +
examples/mnist/create_shard.cc | 21 +
include/communication/msg.h | 21 +
include/communication/socket.h | 21 +
include/driver.h | 21 +
include/neuralnet/connection_layer.h | 21 +
include/neuralnet/input_layer.h | 21 +
include/neuralnet/layer.h | 21 +
include/neuralnet/loss_layer.h | 21 +
include/neuralnet/neuralnet.h | 21 +
include/neuralnet/neuron_layer.h | 21 +
include/neuralnet/output_layer.h | 21 +
include/singa.h | 21 +
include/trainer/server.h | 21 +
include/trainer/trainer.h | 21 +
include/trainer/worker.h | 21 +
include/utils/blob.h | 21 +
include/utils/cluster.h | 21 +
include/utils/cluster_rt.h | 21 +
include/utils/common.h | 21 +
include/utils/data_shard.h | 21 +
include/utils/factory.h | 21 +
include/utils/graph.h | 21 +
include/utils/param.h | 21 +
include/utils/singleton.h | 21 +
include/utils/updater.h | 21 +
src/communication/msg.cc | 21 +
src/communication/socket.cc | 20 +
src/driver.cc | 21 +
src/main.cc | 21 +
src/neuralnet/connection_layer.cc | 21 +
src/neuralnet/input_layer.cc | 21 +
src/neuralnet/layer.cc | 21 +
src/neuralnet/loss_layer.cc | 21 +
src/neuralnet/neuralnet.cc | 21 +
src/neuralnet/neuron_layer.cc | 21 +
src/neuralnet/output_layer.cc | 21 +
src/proto/common.proto | 21 +
src/proto/job.proto | 21 +
src/proto/singa.proto | 21 +
src/test/test_cluster.cc | 21 +
src/test/test_msg.cc | 21 +
src/test/test_neuralnet.cc | 21 +
src/test/test_paramslicer.cc | 21 +
src/test/test_shard.cc | 21 +
src/trainer/server.cc | 21 +
src/trainer/trainer.cc | 21 +
src/trainer/worker.cc | 21 +
src/utils/blob.cc | 21 +
src/utils/cluster.cc | 21 +
src/utils/cluster_rt.cc | 21 +
src/utils/common.cc | 24 +-
src/utils/data_shard.cc | 21 +
src/utils/graph.cc | 21 +
src/utils/param.cc | 21 +
src/utils/tool.cc | 21 +
src/utils/updater.cc | 21 +
thirdparty/install.sh | 19 +
tool/graph.py | 20 +
tool/node.sh | 19 +
71 files changed, 2229 insertions(+), 866 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/DISCLAIMER
----------------------------------------------------------------------
diff --git a/DISCLAIMER b/DISCLAIMER
new file mode 100644
index 0000000..b8f2f12
--- /dev/null
+++ b/DISCLAIMER
@@ -0,0 +1,10 @@
+Apache SINGA is an effort undergoing incubation at the Apache Software
+Foundation (ASF), sponsored by the Apache Incubator PMC.
+
+Incubation is required of all newly accepted projects until a further review
+indicates that the infrastructure, communications, and decision making process
+have stabilized in a manner consistent with other successful ASF projects.
+
+While incubation status is not necessarily a reflection of the completeness
+or stability of the code, it does indicate that the project has yet to be
+fully endorsed by the ASF.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/74866478/LICENSE
----------------------------------------------------------------------
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 0000000..83a58c7
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,832 @@
+Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "{}"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+============================================================================
+The SINGA project contains subcomponents with separate copyright
+notices and license terms. Your use of the source code for the these
+subcomponents is subject to the terms and conditions of the following
+licenses.
+
+============================================================================
+For cafee (see comments in src/neuralnet/layer.cc)
+===========================================================================
+COPYRIGHT
+All contributions by the University of California:
+Copyright (c) 2014, The Regents of the University of California (Regents)
+All rights reserved.
+All other contributions:
+Copyright (c) 2014, the respective contributors
+All rights reserved.
+Caffe uses a shared copyright model: each contributor holds copyright over
+their contributions to Caffe. The project versioning records all such
+contribution and copyright details. If a contributor wants to further mark
+their specific copyright on a particular contribution, they should indicate
+their copyright solely in the commit message of the change when it is
+committed.
+LICENSE
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+1. Redistributions of source code must retain the above copyright notice, this
+list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice,
+this list of conditions and the following disclaimer in the documentation
+and/or other materials provided with the distribution.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+CONTRIBUTION AGREEMENT
+By contributing to the BVLC/caffe repository through pull-request, comment,
+or otherwise, the contributor releases their content to the
+license and copyright terms herein.
+
+=====================================================================
+For czmq
+=====================================================================
+Mozilla Public License Version 2.0
+==================================
+
+1. Definitions
+--------------
+
+1.1. "Contributor"
+ means each individual or legal entity that creates, contributes to
+ the creation of, or owns Covered Software.
+
+1.2. "Contributor Version"
+ means the combination of the Contributions of others (if any) used
+ by a Contributor and that particular Contributor's Contribution.
+
+1.3. "Contribution"
+ means Covered Software of a particular Contributor.
+
+1.4. "Covered Software"
+ means Source Code Form to which the initial Contributor has attached
+ the notice in Exhibit A, the Executable Form of such Source Code
+ Form, and Modifications of such Source Code Form, in each case
+ including portions thereof.
+
+1.5. "Incompatible With Secondary Licenses"
+ means
+
+ (a) that the initial Contributor has attached the notice described
+ in Exhibit B to the Covered Software; or
+
+ (b) that the Covered Software was made available under the terms of
+ version 1.1 or earlier of the License, but not also under the
+ terms of a Secondary License.
+
+1.6. "Executable Form"
+ means any form of the work other than Source Code Form.
+
+1.7. "Larger Work"
+ means a work that combines Covered Software with other material, in
+ a separate file or files, that is not Covered Software.
+
+1.8. "License"
+ means this document.
+
+1.9. "Licensable"
+ means having the right to grant, to the maximum extent possible,
+ whether at the time of the initial grant or subsequently, any and
+ all of the rights conveyed by this License.
+
+1.10. "Modifications"
+ means any of the following:
+
+ (a) any file in Source Code Form that results from an addition to,
+ deletion from, or modification of the contents of Covered
+ Software; or
+
+ (b) any new file in Source Code Form that contains any Covered
+ Software.
+
+1.11. "Patent Claims" of a Contributor
+ means any patent claim(s), including without limitation, method,
+ process, and apparatus claims, in any patent Licensable by such
+ Contributor that would be infringed, but for the grant of the
+ License, by the making, using, selling, offering for sale, having
+ made, import, or transfer of either its Contributions or its
+ Contributor Version.
+
+1.12. "Secondary License"
+ means either the GNU General Public License, Version 2.0, the GNU
+ Lesser General Public License, Version 2.1, the GNU Affero General
+ Public License, Version 3.0, or any later versions of those
+ licenses.
+
+1.13. "Source Code Form"
+ means the form of the work preferred for making modifications.
+
+1.14. "You" (or "Your")
+ means an individual or a legal entity exercising rights under this
+ License. For legal entities, "You" includes any entity that
+ controls, is controlled by, or is under common control with You. For
+ purposes of this definition, "control" means (a) the power, direct
+ or indirect, to cause the direction or management of such entity,
+ whether by contract or otherwise, or (b) ownership of more than
+ fifty percent (50%) of the outstanding shares or beneficial
+ ownership of such entity.
+
+2. License Grants and Conditions
+--------------------------------
+
+2.1. Grants
+
+Each Contributor hereby grants You a world-wide, royalty-free,
+non-exclusive license:
+
+(a) under intellectual property rights (other than patent or trademark)
+ Licensable by such Contributor to use, reproduce, make available,
+ modify, display, perform, distribute, and otherwise exploit its
+ Contributions, either on an unmodified basis, with Modifications, or
+ as part of a Larger Work; and
+
+(b) under Patent Claims of such Contributor to make, use, sell, offer
+ for sale, have made, import, and otherwise transfer either its
+ Contributions or its Contributor Version.
+
+2.2. Effective Date
+
+The licenses granted in Section 2.1 with respect to any Contribution
+become effective for each Contribution on the date the Contributor first
+distributes such Contribution.
+
+2.3. Limitations on Grant Scope
+
+The licenses granted in this Section 2 are the only rights granted under
+this License. No additional rights or licenses will be implied from the
+distribution or licensing of Covered Software under this License.
+Notwithstanding Section 2.1(b) above, no patent license is granted by a
+Contributor:
+
+(a) for any code that a Contributor has removed from Covered Software;
+ or
+
+(b) for infringements caused by: (i) Your and any other third party's
+ modifications of Covered Software, or (ii) the combination of its
+ Contributions with other software (except as part of its Contributor
+ Version); or
+
+(c) under Patent Claims infringed by Covered Software in the absence of
+ its Contributions.
+
+This License does not grant any rights in the trademarks, service marks,
+or logos of any Contributor (except as may be necessary to comply with
+the notice requirements in Section 3.4).
+
+2.4. Subsequent Licenses
+
+No Contributor makes additional grants as a result of Your choice to
+distribute the Covered Software under a subsequent version of this
+License (see Section 10.2) or under the terms of a Secondary License (if
+permitted under the terms of Section 3.3).
+
+2.5. Representation
+
+Each Contributor represents that the Contributor believes its
+Contributions are its original creation(s) or it has sufficient rights
+to grant the rights to its Contributions conveyed by this License.
+
+2.6. Fair Use
+
+This License is not intended to limit any rights You have under
+applicable copyright doctrines of fair use, fair dealing, or other
+equivalents.
+
+2.7. Conditions
+
+Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
+in Section 2.1.
+
+3. Responsibilities
+-------------------
+
+3.1. Distribution of Source Form
+
+All distribution of Covered Software in Source Code Form, including any
+Modifications that You create or to which You contribute, must be under
+the terms of this License. You must inform recipients that the Source
+Code Form of the Covered Software is governed by the terms of this
+License, and how they can obtain a copy of this License. You may not
+attempt to alter or restrict the recipients' rights in the Source Code
+Form.
+
+3.2. Distribution of Executable Form
+
+If You distribute Covered Software in Executable Form then:
+
+(a) such Covered Software must also be made available in Source Code
+ Form, as described in Section 3.1, and You must inform recipients of
+ the Executable Form how they can obtain a copy of such Source Code
+ Form by reasonable means in a timely manner, at a charge no more
+ than the cost of distribution to the recipient; and
+
+(b) You may distribute such Executable Form under the terms of this
+ License, or sublicense it under different terms, provided that the
+ license for the Executable Form does not attempt to limit or alter
+ the recipients' rights in the Source Code Form under this License.
+
+3.3. Distribution of a Larger Work
+
+You may create and distribute a Larger Work under terms of Your choice,
+provided that You also comply with the requirements of this License for
+the Covered Software. If the Larger Work is a combination of Covered
+Software with a work governed by one or more Secondary Licenses, and the
+Covered Software is not Incompatible With Secondary Licenses, this
+License permits You to additionally distribute such Covered Software
+under the terms of such Secondary License(s), so that the recipient of
+the Larger Work may, at their option, further distribute the Covered
+Software under the terms of either this License or such Secondary
+License(s).
+
+3.4. Notices
+
+You may not remove or alter the substance of any license notices
+(including copyright notices, patent notices, disclaimers of warranty,
+or limitations of liability) contained within the Source Code Form of
+the Covered Software, except that You may alter any license notices to
+the extent required to remedy known factual inaccuracies.
+
+3.5. Application of Additional Terms
+
+You may choose to offer, and to charge a fee for, warranty, support,
+indemnity or liability obligations to one or more recipients of Covered
+Software. However, You may do so only on Your own behalf, and not on
+behalf of any Contributor. You must make it absolutely clear that any
+such warranty, support, indemnity, or liability obligation is offered by
+You alone, and You hereby agree to indemnify every Contributor for any
+liability incurred by such Contributor as a result of warranty, support,
+indemnity or liability terms You offer. You may include additional
+disclaimers of warranty and limitations of liability specific to any
+jurisdiction.
+
+4. Inability to Comply Due to Statute or Regulation
+---------------------------------------------------
+
+If it is impossible for You to comply with any of the terms of this
+License with respect to some or all of the Covered Software due to
+statute, judicial order, or regulation then You must: (a) comply with
+the terms of this License to the maximum extent possible; and (b)
+describe the limitations and the code they affect. Such description must
+be placed in a text file included with all distributions of the Covered
+Software under this License. Except to the extent prohibited by statute
+or regulation, such description must be sufficiently detailed for a
+recipient of ordinary skill to be able to understand it.
+
+5. Termination
+--------------
+
+5.1. The rights granted under this License will terminate automatically
+if You fail to comply with any of its terms. However, if You become
+compliant, then the rights granted under this License from a particular
+Contributor are reinstated (a) provisionally, unless and until such
+Contributor explicitly and finally terminates Your grants, and (b) on an
+ongoing basis, if such Contributor fails to notify You of the
+non-compliance by some reasonable means prior to 60 days after You have
+come back into compliance. Moreover, Your grants from a particular
+Contributor are reinstated on an ongoing basis if such Contributor
+notifies You of the non-compliance by some reasonable means, this is the
+first time You have received notice of non-compliance with this License
+from such Contributor, and You become compliant prior to 30 days after
+Your receipt of the notice.
+
+5.2. If You initiate litigation against any entity by asserting a patent
+infringement claim (excluding declaratory judgment actions,
+counter-claims, and cross-claims) alleging that a Contributor Version
+directly or indirectly infringes any patent, then the rights granted to
+You by any and all Contributors for the Covered Software under Section
+2.1 of this License shall terminate.
+
+5.3. In the event of termination under Sections 5.1 or 5.2 above, all
+end user license agreements (excluding distributors and resellers) which
+have been validly granted by You or Your distributors under this License
+prior to termination shall survive termination.
+
+************************************************************************
+* *
+* 6. Disclaimer of Warranty *
+* ------------------------- *
+* *
+* Covered Software is provided under this License on an "as is" *
+* basis, without warranty of any kind, either expressed, implied, or *
+* statutory, including, without limitation, warranties that the *
+* Covered Software is free of defects, merchantable, fit for a *
+* particular purpose or non-infringing. The entire risk as to the *
+* quality and performance of the Covered Software is with You. *
+* Should any Covered Software prove defective in any respect, You *
+* (not any Contributor) assume the cost of any necessary servicing, *
+* repair, or correction. This disclaimer of warranty constitutes an *
+* essential part of this License. No use of any Covered Software is *
+* authorized under this License except under this disclaimer. *
+* *
+************************************************************************
+
+************************************************************************
+* *
+* 7. Limitation of Liability *
+* -------------------------- *
+* *
+* Under no circumstances and under no legal theory, whether tort *
+* (including negligence), contract, or otherwise, shall any *
+* Contributor, or anyone who distributes Covered Software as *
+* permitted above, be liable to You for any direct, indirect, *
+* special, incidental, or consequential damages of any character *
+* including, without limitation, damages for lost profits, loss of *
+* goodwill, work stoppage, computer failure or malfunction, or any *
+* and all other commercial damages or losses, even if such party *
+* shall have been informed of the possibility of such damages. This *
+* limitation of liability shall not apply to liability for death or *
+* personal injury resulting from such party's negligence to the *
+* extent applicable law prohibits such limitation. Some *
+* jurisdictions do not allow the exclusion or limitation of *
+* incidental or consequential damages, so this exclusion and *
+* limitation may not apply to You. *
+* *
+************************************************************************
+
+8. Litigation
+-------------
+
+Any litigation relating to this License may be brought only in the
+courts of a jurisdiction where the defendant maintains its principal
+place of business and such litigation shall be governed by laws of that
+jurisdiction, without reference to its conflict-of-law provisions.
+Nothing in this Section shall prevent a party's ability to bring
+cross-claims or counter-claims.
+
+9. Miscellaneous
+----------------
+
+This License represents the complete agreement concerning the subject
+matter hereof. If any provision of this License is held to be
+unenforceable, such provision shall be reformed only to the extent
+necessary to make it enforceable. Any law or regulation which provides
+that the language of a contract shall be construed against the drafter
+shall not be used to construe this License against a Contributor.
+
+10. Versions of the License
+---------------------------
+
+10.1. New Versions
+
+Mozilla Foundation is the license steward. Except as provided in Section
+10.3, no one other than the license steward has the right to modify or
+publish new versions of this License. Each version will be given a
+distinguishing version number.
+
+10.2. Effect of New Versions
+
+You may distribute the Covered Software under the terms of the version
+of the License under which You originally received the Covered Software,
+or under the terms of any subsequent version published by the license
+steward.
+
+10.3. Modified Versions
+
+If you create software not governed by this License, and you want to
+create a new license for such software, you may create and use a
+modified version of this License if you rename the license and remove
+any references to the name of the license steward (except to note that
+such modified license differs from this License).
+
+10.4. Distributing Source Code Form that is Incompatible With Secondary
+Licenses
+
+If You choose to distribute Source Code Form that is Incompatible With
+Secondary Licenses under the terms of this version of the License, the
+notice described in Exhibit B of this License must be attached.
+
+Exhibit A - Source Code Form License Notice
+-------------------------------------------
+
+ This Source Code Form is subject to the terms of the Mozilla Public
+ License, v. 2.0. If a copy of the MPL was not distributed with this
+ file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+If it is not possible or desirable to put the notice in a particular
+file, then You may include the notice in a location (such as a LICENSE
+file in a relevant directory) where a recipient would be likely to look
+for such a notice.
+
+You may add additional accurate notices of copyright ownership.
+
+Exhibit B - "Incompatible With Secondary Licenses" Notice
+---------------------------------------------------------
+
+ This Source Code Form is "Incompatible With Secondary Licenses", as
+ defined by the Mozilla Public License, v. 2.0.
+
+======================================================================
+For glog:
+======================================================================
+Copyright (c) 2008, Google Inc.
+All rights reserved.
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+* Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+* Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+A function gettimeofday in utilities.cc is based on
+http://www.google.com/codesearch/p?hl=en#dR3YEbitojA/COPYING&q=GetSystemTimeAsFileTime%20license:bsd
+The license of this code is:
+Copyright (c) 2003-2008, Jouni Malinen <j...@w1.fi> and contributors
+All Rights Reserved.
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+1. Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright
+notice, this list of conditions and the following disclaimer in the
+documentation and/or other materials provided with the distribution.
+3. Neither the name(s) of the above-listed copyright holder(s) nor the
+names of its contributors may be used to endorse or promote products
+derived from this software without specific prior written permission.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+=====================================================================
+For lmdb
+=====================================================================
+The OpenLDAP Public License
+ Version 2.8, 17 August 2003
+
+ Redistribution and use of this software and associated documentation
+ ("Software"), with or without modification, are permitted provided
+ that the following conditions are met:
+
+ 1. Redistributions in source form must retain copyright statements
+ and notices,
+
+ 2. Redistributions in binary form must reproduce applicable copyright
+ statements and notices, this list of conditions, and the following disclaimer
+ in the documentation and/or other materials provided with the distribution,
+ and
+
+ 3. Redistributions must contain a verbatim copy of this document.
+
+ The OpenLDAP Foundation may revise this license from time to
+ time.
+ Each revision is distinguished by a version number. You may use
+ this Software under terms of this license revision or under the
+ terms of any subsequent revision of the license.
+
+ THIS SOFTWARE IS PROVIDED BY THE OPENLDAP FOUNDATION AND ITS
+ CONTRIBUTORS ``AS IS'' AND ANY EXPRESSED OR IMPLIED WARRANTIES,
+ INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+ MERCHANTABILITY
+ AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
+ SHALL THE OPENLDAP FOUNDATION, ITS CONTRIBUTORS, OR THE AUTHOR(S)
+ OR OWNER(S) OF THE SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT,
+ INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+ (INCLUDING,
+ BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+ LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+ CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT,
+ STRICT
+ LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
+ ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ POSSIBILITY OF SUCH DAMAGE.
+
+ The names of the authors and copyright holders must not be used
+ in
+ advertising or otherwise to promote the sale, use or other
+ dealing
+ in this Software without specific, written prior permission.
+ Title
+ to copyright in this Software shall at all times remain with
+ copyright
+ holders.
+
+ OpenLDAP is a registered trademark of the OpenLDAP Foundation.
+
+ Copyright 1999-2003 The OpenLDAP Foundation, Redwood City,
+ California, USA. All Rights Reserved. Permission to copy and
+ distribute verbatim copies of this document is granted.
+
+======================================================================
+For protobuf:
+=====================================================================
+Copyright 2008, Google Inc.
+All rights reserved.
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+* Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+* Neither the name of Google Inc. nor the names of its
+contributors may be used to endorse or promote products derived from
+this software without specific prior written permission.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+Code generated by the Protocol Buffer compiler is owned by the owner
+of the input file used when generating it. This code is not
+standalone and requires a support library to be linked with it. This
+support library is itself covered by the above license.
+
+========================================================================
+For OpenCV used in preprocessing images
+=======================================================================
+License Agreement
+For Open Source Computer Vision Library
+(3-clause BSD License)
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+Redistributions of source code must retain the above copyright notice,
+this list of conditions and the following disclaimer.
+Redistributions in binary form must reproduce the above copyright notice,
+this list of conditions and the following disclaimer in the documentation
+and/or other materials provided with the distribution.
+Neither the names of the copyright holders nor the names of the contributors
+may be used to endorse or promote products derived from this software without
+specific prior written permission.
+
+This software is provided by the copyright holders and contributors “as is”
+and any express or implied warranties, including, but not limited to, the
+implied warranties of merchantability and fitness for a particular purpose are
+disclaimed. In no event shall copyright holders or contributors be liable for
+any direct, indirect, incidental, special, exemplary, or consequential damages
+(including, but not limited to, procurement of substitute goods or services;
+loss of use, data, or profits; or business interruption) however caused and on
+any theory of liability, whether in contract, strict liability, or tort
+(including negligence or otherwise) arising in any way out of the use of this
+software, even if advised of the possibility of such damage.
+
+
+========================================================================
+For Openblas
+=======================================================================
+Copyright (c) 2011-2014, The OpenBLAS Project
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+ 1. Redistributions of source code must retain the above copyright
+ notice, this list of conditions and the following disclaimer.
+
+ 2. Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+ 3. Neither the name of the OpenBLAS project nor the names of its contributors
+ may be used to endorse or promote products derived from this software without
+ specific prior written permission.
+
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ POSSIBILITY OF SUCH DAMAGE.
[18/21] incubator-singa git commit: SINGA-65 Add an example of
implementing user's own layers
Posted by wa...@apache.org.
SINGA-65 Add an example of implementing user's own layers
Move the complete code into example/mlp/full foder.
Remove some code in the source files to let users fill in.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/5a8a1a31
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/5a8a1a31
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/5a8a1a31
Branch: refs/heads/tutorial
Commit: 5a8a1a31648e7ab6d705bcca455adf3b584ea8d0
Parents: 4820d5e
Author: wangwei <wa...@comp.nus.edu.sg>
Authored: Fri Sep 4 18:39:25 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Wed Sep 16 11:39:16 2015 +0800
----------------------------------------------------------------------
examples/mlp/full/hidden_layer.cc | 72 ++++++++++++++++++++++++++
examples/mlp/full/hidden_layer.h | 22 ++++++++
examples/mlp/full/job.conf | 95 ++++++++++++++++++++++++++++++++++
examples/mlp/full/main.cc | 25 +++++++++
examples/mlp/full/myproto.proto | 11 ++++
examples/mlp/hidden_layer.cc | 20 ++-----
examples/mlp/hidden_layer.h | 11 +---
examples/mlp/job.conf | 42 ++-------------
examples/mlp/main.cc | 3 +-
examples/mlp/myproto.proto | 9 +---
10 files changed, 237 insertions(+), 73 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/full/hidden_layer.cc
----------------------------------------------------------------------
diff --git a/examples/mlp/full/hidden_layer.cc b/examples/mlp/full/hidden_layer.cc
new file mode 100644
index 0000000..f4dacf2
--- /dev/null
+++ b/examples/mlp/full/hidden_layer.cc
@@ -0,0 +1,72 @@
+#include "hidden_layer.h"
+
+#include "mshadow/tensor.h"
+#include "mshadow/cxxnet_op.h"
+
+namespace singa {
+using namespace mshadow;
+using mshadow::cpu;
+using mshadow::Shape1;
+using mshadow::Shape2;
+
+inline Tensor<cpu, 2> NewTensor2(Blob<float>* blob) {
+ const vector<int>& shape = blob->shape();
+ Tensor<cpu, 2> tensor(blob->mutable_cpu_data(),
+ Shape2(shape[0], blob->count() / shape[0]));
+ return tensor;
+}
+
+inline Tensor<cpu, 1> NewTensor1(Blob<float>* blob) {
+ Tensor<cpu, 1> tensor(blob->mutable_cpu_data(), Shape1(blob->count()));
+ return tensor;
+}
+
+
+HiddenLayer::~HiddenLayer() {
+ delete weight_;
+ delete bias_;
+}
+
+void HiddenLayer::Setup(const LayerProto& proto, int npartitions) {
+ Layer::Setup(proto, npartitions);
+ CHECK_EQ(srclayers_.size(), 1);
+ const auto& src = srclayers_[0]->data(this);
+ batchsize_ = src.shape()[0];
+ vdim_ = src.count() / batchsize_;
+ hdim_ = layer_proto_.GetExtension(hidden_conf).num_output();
+ data_.Reshape(vector<int>{batchsize_, hdim_});
+ grad_.ReshapeLike(data_);
+ weight_ = Param::Create(proto.param(0));
+ bias_ = Param::Create(proto.param(1));
+ weight_->Setup(vector<int>{hdim_, vdim_});
+ bias_->Setup(vector<int>{hdim_});
+}
+
+void HiddenLayer::ComputeFeature(int flag, Metric* perf) {
+ auto data = NewTensor2(&data_);
+ auto src = NewTensor2(srclayers_[0]->mutable_data(this));
+ auto weight = NewTensor2(weight_->mutable_data());
+ auto bias = NewTensor1(bias_->mutable_data());
+ data = dot(src, weight.T());
+ // repmat: repeat bias vector into batchsize rows
+ data += expr::repmat(bias, batchsize_);
+ data = expr::F<op::stanh>(data);
+}
+
+void HiddenLayer::ComputeGradient(int flag, Metric* perf) {
+ auto data = NewTensor2(&data_);
+ auto src = NewTensor2(srclayers_[0]->mutable_data(this));
+ auto grad = NewTensor2(&grad_);
+ auto weight = NewTensor2(weight_->mutable_data());
+ auto gweight = NewTensor2(weight_->mutable_grad());
+ auto gbias = NewTensor1(bias_->mutable_grad());
+
+ grad = expr::F<op::stanh_grad>(data) * grad;
+ gbias = expr::sum_rows(grad);
+ gweight = dot(grad.T(), src);
+ if (srclayers_[0]->mutable_grad(this) != nullptr) {
+ auto gsrc = NewTensor2(srclayers_[0]->mutable_grad(this));
+ gsrc = dot(grad, weight);
+ }
+}
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/full/hidden_layer.h
----------------------------------------------------------------------
diff --git a/examples/mlp/full/hidden_layer.h b/examples/mlp/full/hidden_layer.h
new file mode 100644
index 0000000..c305261
--- /dev/null
+++ b/examples/mlp/full/hidden_layer.h
@@ -0,0 +1,22 @@
+#include "neuralnet/layer.h"
+#include "myproto.pb.h"
+
+namespace singa {
+class HiddenLayer : public NeuronLayer {
+ public:
+ ~HiddenLayer();
+ void Setup(const LayerProto& proto, int npartitions) override;
+ void ComputeFeature(int flag, Metric* perf) override;
+ void ComputeGradient(int flag, Metric* perf) override;
+ const std::vector<Param*> GetParams() const override {
+ std::vector<Param*> params{weight_, bias_};
+ return params;
+ }
+
+ private:
+ int batchsize_;
+ int vdim_, hdim_;
+ bool transpose_;
+ Param *weight_, *bias_;
+};
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/full/job.conf
----------------------------------------------------------------------
diff --git a/examples/mlp/full/job.conf b/examples/mlp/full/job.conf
new file mode 100644
index 0000000..163bc19
--- /dev/null
+++ b/examples/mlp/full/job.conf
@@ -0,0 +1,95 @@
+name: "mlp"
+train_steps: 1200
+test_steps:10
+test_freq:60
+disp_freq:10
+train_one_batch {
+ alg: kBP
+}
+updater{
+ type: kSGD
+ learning_rate{
+ type : kStep
+ base_lr: 0.001
+ step_conf{
+ change_freq: 60
+ gamma: 0.997
+ }
+ }
+}
+neuralnet {
+ layer {
+ name: "data"
+ type: kShardData
+ sharddata_conf {
+ path: "examples/mnist/mnist_train_shard"
+ batchsize: 1000
+ }
+ exclude: kTest
+ }
+
+ layer {
+ name: "data"
+ type: kShardData
+ sharddata_conf {
+ path: "examples/mnist/mnist_test_shard"
+ batchsize: 1000
+ }
+ exclude: kTrain
+ }
+
+ layer{
+ name:"mnist"
+ type: kMnist
+ srclayers: "data"
+ mnist_conf {
+ norm_a: 127.5
+ norm_b: 1
+ }
+ }
+ layer{
+ name: "label"
+ type: kLabel
+ srclayers: "data"
+ }
+
+
+ layer{
+ name: "hid1"
+ user_type: "kHidden"
+ srclayers:"mnist"
+ [singa.hidden_conf] {
+ num_output: 10
+ }
+ param{
+ name: "w1"
+ init {
+ type: kUniform
+ low:-0.05
+ high:0.05
+ }
+ }
+ param{
+ name: "b1"
+ init {
+ type : kUniform
+ low: -0.05
+ high:0.05
+ }
+ }
+ }
+ layer{
+ name: "loss"
+ type:kSoftmaxLoss
+ softmaxloss_conf{
+ topk:1
+ }
+ srclayers:"hid1"
+ srclayers:"label"
+ }
+}
+cluster {
+ nworker_groups: 1
+ nserver_groups: 1
+ workspace: "examples/mnist"
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/full/main.cc
----------------------------------------------------------------------
diff --git a/examples/mlp/full/main.cc b/examples/mlp/full/main.cc
new file mode 100644
index 0000000..c27f38d
--- /dev/null
+++ b/examples/mlp/full/main.cc
@@ -0,0 +1,25 @@
+#include <string>
+#include "singa.h"
+#include "hidden_layer.h"
+#include "myproto.pb.h"
+#include "utils/common.h"
+
+int main(int argc, char **argv) {
+ // must create driver at the beginning and call its Init method.
+ singa::Driver driver;
+ driver.Init(argc, argv);
+
+ // if -resume in argument list, set resume to true; otherwise false
+ int resume_pos = singa::ArgPos(argc, argv, "-resume");
+ bool resume = (resume_pos != -1);
+
+ // users can register new subclasses of layer, updater, etc.
+ driver.RegisterLayer<singa::HiddenLayer, std::string>("kHidden");
+
+ // get the job conf, and custmize it if need
+ singa::JobProto jobConf = driver.job_conf();
+
+ // submit the job
+ driver.Submit(resume, jobConf);
+ return 0;
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/full/myproto.proto
----------------------------------------------------------------------
diff --git a/examples/mlp/full/myproto.proto b/examples/mlp/full/myproto.proto
new file mode 100644
index 0000000..ba62ac2
--- /dev/null
+++ b/examples/mlp/full/myproto.proto
@@ -0,0 +1,11 @@
+package singa;
+import "job.proto";
+
+message HiddenProto {
+ required int32 num_output = 1;
+ optional bool use_stanh = 2 [default = true];
+}
+
+extend LayerProto {
+ optional HiddenProto hidden_conf = 102;
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/hidden_layer.cc
----------------------------------------------------------------------
diff --git a/examples/mlp/hidden_layer.cc b/examples/mlp/hidden_layer.cc
index f4dacf2..14eee23 100644
--- a/examples/mlp/hidden_layer.cc
+++ b/examples/mlp/hidden_layer.cc
@@ -32,14 +32,8 @@ void HiddenLayer::Setup(const LayerProto& proto, int npartitions) {
CHECK_EQ(srclayers_.size(), 1);
const auto& src = srclayers_[0]->data(this);
batchsize_ = src.shape()[0];
- vdim_ = src.count() / batchsize_;
- hdim_ = layer_proto_.GetExtension(hidden_conf).num_output();
- data_.Reshape(vector<int>{batchsize_, hdim_});
- grad_.ReshapeLike(data_);
- weight_ = Param::Create(proto.param(0));
- bias_ = Param::Create(proto.param(1));
- weight_->Setup(vector<int>{hdim_, vdim_});
- bias_->Setup(vector<int>{hdim_});
+
+ // please add code for setup this layer
}
void HiddenLayer::ComputeFeature(int flag, Metric* perf) {
@@ -47,10 +41,7 @@ void HiddenLayer::ComputeFeature(int flag, Metric* perf) {
auto src = NewTensor2(srclayers_[0]->mutable_data(this));
auto weight = NewTensor2(weight_->mutable_data());
auto bias = NewTensor1(bias_->mutable_data());
- data = dot(src, weight.T());
- // repmat: repeat bias vector into batchsize rows
- data += expr::repmat(bias, batchsize_);
- data = expr::F<op::stanh>(data);
+ // please add code for computing the hidden feature
}
void HiddenLayer::ComputeGradient(int flag, Metric* perf) {
@@ -61,9 +52,8 @@ void HiddenLayer::ComputeGradient(int flag, Metric* perf) {
auto gweight = NewTensor2(weight_->mutable_grad());
auto gbias = NewTensor1(bias_->mutable_grad());
- grad = expr::F<op::stanh_grad>(data) * grad;
- gbias = expr::sum_rows(grad);
- gweight = dot(grad.T(), src);
+ // please add code for computing the gradients
+
if (srclayers_[0]->mutable_grad(this) != nullptr) {
auto gsrc = NewTensor2(srclayers_[0]->mutable_grad(this));
gsrc = dot(grad, weight);
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/hidden_layer.h
----------------------------------------------------------------------
diff --git a/examples/mlp/hidden_layer.h b/examples/mlp/hidden_layer.h
index c305261..5dbae12 100644
--- a/examples/mlp/hidden_layer.h
+++ b/examples/mlp/hidden_layer.h
@@ -8,15 +8,6 @@ class HiddenLayer : public NeuronLayer {
void Setup(const LayerProto& proto, int npartitions) override;
void ComputeFeature(int flag, Metric* perf) override;
void ComputeGradient(int flag, Metric* perf) override;
- const std::vector<Param*> GetParams() const override {
- std::vector<Param*> params{weight_, bias_};
- return params;
- }
- private:
- int batchsize_;
- int vdim_, hdim_;
- bool transpose_;
- Param *weight_, *bias_;
-};
+// please fill HiddenLayer class declaration
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/job.conf
----------------------------------------------------------------------
diff --git a/examples/mlp/job.conf b/examples/mlp/job.conf
index 163bc19..6f57677 100644
--- a/examples/mlp/job.conf
+++ b/examples/mlp/job.conf
@@ -3,20 +3,9 @@ train_steps: 1200
test_steps:10
test_freq:60
disp_freq:10
-train_one_batch {
- alg: kBP
-}
-updater{
- type: kSGD
- learning_rate{
- type : kStep
- base_lr: 0.001
- step_conf{
- change_freq: 60
- gamma: 0.997
- }
- }
-}
+
+# please add the config for updater and train_one_batch
+
neuralnet {
layer {
name: "data"
@@ -53,31 +42,8 @@ neuralnet {
srclayers: "data"
}
+# please add the config for the hidden layer
- layer{
- name: "hid1"
- user_type: "kHidden"
- srclayers:"mnist"
- [singa.hidden_conf] {
- num_output: 10
- }
- param{
- name: "w1"
- init {
- type: kUniform
- low:-0.05
- high:0.05
- }
- }
- param{
- name: "b1"
- init {
- type : kUniform
- low: -0.05
- high:0.05
- }
- }
- }
layer{
name: "loss"
type:kSoftmaxLoss
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/main.cc
----------------------------------------------------------------------
diff --git a/examples/mlp/main.cc b/examples/mlp/main.cc
index c27f38d..cf104e6 100644
--- a/examples/mlp/main.cc
+++ b/examples/mlp/main.cc
@@ -13,8 +13,7 @@ int main(int argc, char **argv) {
int resume_pos = singa::ArgPos(argc, argv, "-resume");
bool resume = (resume_pos != -1);
- // users can register new subclasses of layer, updater, etc.
- driver.RegisterLayer<singa::HiddenLayer, std::string>("kHidden");
+ // please register the HiddenLayer here
// get the job conf, and custmize it if need
singa::JobProto jobConf = driver.job_conf();
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5a8a1a31/examples/mlp/myproto.proto
----------------------------------------------------------------------
diff --git a/examples/mlp/myproto.proto b/examples/mlp/myproto.proto
index ba62ac2..deea1d4 100644
--- a/examples/mlp/myproto.proto
+++ b/examples/mlp/myproto.proto
@@ -1,11 +1,4 @@
package singa;
import "job.proto";
-message HiddenProto {
- required int32 num_output = 1;
- optional bool use_stanh = 2 [default = true];
-}
-
-extend LayerProto {
- optional HiddenProto hidden_conf = 102;
-}
+// please add the definition of HiddenProto
[07/21] incubator-singa git commit: SINGA-51 Improve the convolution
and pooling operations
Posted by wa...@apache.org.
SINGA-51 Improve the convolution and pooling operations
minor changes
- format test_common.cc
- format include patterns in driver, layers
all cc file include own h file in first line
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/d5d817e1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/d5d817e1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/d5d817e1
Branch: refs/heads/tutorial
Commit: d5d817e145b1b8d60d8ae3781c24aa0916f4ddf7
Parents: e769142
Author: wang sheng <wa...@gmail.com>
Authored: Mon Sep 14 16:57:10 2015 +0800
Committer: wang sheng <wa...@gmail.com>
Committed: Mon Sep 14 16:57:10 2015 +0800
----------------------------------------------------------------------
include/driver.h | 9 +-
include/neuralnet/connection_layer.h | 7 +-
include/neuralnet/input_layer.h | 2 +
include/neuralnet/layer.h | 6 +-
include/neuralnet/loss_layer.h | 5 +-
include/neuralnet/neuron_layer.h | 4 +-
include/neuralnet/output_layer.h | 2 +
src/driver.cc | 4 +-
src/neuralnet/connection_layer.cc | 6 +-
src/neuralnet/input_layer.cc | 7 +-
src/neuralnet/layer.cc | 2 +
src/neuralnet/loss_layer.cc | 7 +-
src/neuralnet/neuron_layer.cc | 21 ++--
src/neuralnet/output_layer.cc | 3 +-
src/test/test_common.cc | 169 +++++++++++++++---------------
src/utils/common.cc | 17 ++-
16 files changed, 142 insertions(+), 129 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/driver.h
----------------------------------------------------------------------
diff --git a/include/driver.h b/include/driver.h
index 5a9ddfc..b16cef3 100644
--- a/include/driver.h
+++ b/include/driver.h
@@ -1,5 +1,6 @@
#ifndef SINGA_DRIVER_H_
#define SINGA_DRIVER_H_
+
#include "singa.h"
namespace singa {
@@ -110,24 +111,28 @@ int Driver::RegisterParam(const Type& type) {
factory->Register(type, CreateInstance(Subclass, Param));
return 1;
}
+
template<typename Subclass, typename Type>
int Driver::RegisterParamGenerator(const Type& type) {
auto factory = Singleton<Factory<singa::ParamGenerator>>::Instance();
factory->Register(type, CreateInstance(Subclass, ParamGenerator));
return 1;
}
+
template<typename Subclass, typename Type>
int Driver::RegisterUpdater(const Type& type) {
auto factory = Singleton<Factory<singa::Updater>>::Instance();
factory->Register(type, CreateInstance(Subclass, Updater));
return 1;
}
+
template<typename Subclass, typename Type>
int Driver::RegisterLRGenerator(const Type& type) {
auto factory = Singleton<Factory<singa::LRGenerator>>::Instance();
factory->Register(type, CreateInstance(Subclass, LRGenerator));
return 1;
}
+
template<typename Subclass, typename Type>
int Driver::RegisterWorker(const Type& type) {
auto factory = Singleton<Factory<singa::Worker>>::Instance();
@@ -135,8 +140,6 @@ int Driver::RegisterWorker(const Type& type) {
return 1;
}
-
} // namespace singa
-#endif // SINGA_DRIVER_H_
-
+#endif // SINGA_DRIVER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/neuralnet/connection_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/connection_layer.h b/include/neuralnet/connection_layer.h
index e44f4f3..233714d 100644
--- a/include/neuralnet/connection_layer.h
+++ b/include/neuralnet/connection_layer.h
@@ -1,5 +1,7 @@
#ifndef SINGA_NEURALNET_CONNECTION_LAYER_H_
#define SINGA_NEURALNET_CONNECTION_LAYER_H_
+
+#include <vector>
#include "neuralnet/layer.h"
/**
@@ -120,6 +122,7 @@ class SplitLayer : public ConnectionLayer {
protected:
Blob<float> grads_;
};
-}
-// namespace singa
+
+} // namespace singa
+
#endif // SINGA_NEURALNET_CONNECTION_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/neuralnet/input_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/input_layer.h b/include/neuralnet/input_layer.h
index 62595c6..67af2eb 100644
--- a/include/neuralnet/input_layer.h
+++ b/include/neuralnet/input_layer.h
@@ -1,6 +1,7 @@
#ifndef SINGA_NEURALNET_INPUT_LAYER_H_
#define SINGA_NEURALNET_INPUT_LAYER_H_
+#include <string>
#include <vector>
#include "neuralnet/layer.h"
#include "utils/data_shard.h"
@@ -165,6 +166,7 @@ class PrefetchLayer : public Layer {
protected:
std::thread thread_;
};
+
} // namespace singa
#endif // SINGA_NEURALNET_INPUT_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/neuralnet/layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/layer.h b/include/neuralnet/layer.h
index 56201f5..4f153d3 100644
--- a/include/neuralnet/layer.h
+++ b/include/neuralnet/layer.h
@@ -5,7 +5,6 @@
#include <string>
#include <thread>
#include <vector>
-
#include "proto/common.pb.h"
#include "proto/job.pb.h"
#include "utils/common.h"
@@ -13,8 +12,6 @@
#include "utils/param.h"
namespace singa {
-using std::vector;
-using std::string;
/**
* Base layer class.
@@ -207,10 +204,11 @@ class LossLayer : public Layer {
};
} // namespace singa
+
#include "neuralnet/connection_layer.h"
#include "neuralnet/input_layer.h"
#include "neuralnet/loss_layer.h"
#include "neuralnet/neuron_layer.h"
#include "neuralnet/output_layer.h"
-#endif // SINGA_NEURALNET_BASE_LAYER_H_
+#endif // SINGA_NEURALNET_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/neuralnet/loss_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/loss_layer.h b/include/neuralnet/loss_layer.h
index 8358bd6..c9f6681 100644
--- a/include/neuralnet/loss_layer.h
+++ b/include/neuralnet/loss_layer.h
@@ -41,6 +41,7 @@ class SoftmaxLossLayer : public LossLayer {
float scale_;
int topk_;
};
-}
-// namespace singa
+
+} // namespace singa
+
#endif // SINGA_NEURALNET_LOSS_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/neuralnet/neuron_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/neuron_layer.h b/include/neuralnet/neuron_layer.h
index dd45eec..86b55a3 100644
--- a/include/neuralnet/neuron_layer.h
+++ b/include/neuralnet/neuron_layer.h
@@ -1,9 +1,10 @@
#ifndef SINGA_NEURALNET_NEURON_LAYER_H_
#define SINGA_NEURALNET_NEURON_LAYER_H_
-#include <vector>
+#include <vector>
#include "neuralnet/layer.h"
#include "proto/job.pb.h"
+
/**
* \file this file includes the declarations neuron layer classes that conduct
* the transformation of features.
@@ -221,7 +222,6 @@ class RBMHidLayer: public RBMLayer {
RBMLayer *vis_layer_;
};
-
} // namespace singa
#endif // SINGA_NEURALNET_NEURON_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/include/neuralnet/output_layer.h
----------------------------------------------------------------------
diff --git a/include/neuralnet/output_layer.h b/include/neuralnet/output_layer.h
index c507e1c..ac83d00 100644
--- a/include/neuralnet/output_layer.h
+++ b/include/neuralnet/output_layer.h
@@ -1,4 +1,6 @@
#ifndef SINGA_NEURALNET_OUTPUT_LAYER_H_
#define SINGA_NEURALNET_OUTPUT_LAYER_H_
+
// currently no output sub-classes are defined
+
#endif // SINGA_NEURALNET_OUTPUT_LAYER_H_
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/driver.cc
----------------------------------------------------------------------
diff --git a/src/driver.cc b/src/driver.cc
index a891a08..88bf4aa 100644
--- a/src/driver.cc
+++ b/src/driver.cc
@@ -1,10 +1,8 @@
+#include "driver.h"
#include <cblas.h>
#include <glog/logging.h>
#include <string>
-
-#include "singa.h"
-
#include "utils/tinydir.h"
namespace singa {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/neuralnet/connection_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/connection_layer.cc b/src/neuralnet/connection_layer.cc
index a3d0a75..e247161 100644
--- a/src/neuralnet/connection_layer.cc
+++ b/src/neuralnet/connection_layer.cc
@@ -1,7 +1,9 @@
-#include <vector>
-#include "neuralnet/layer.h"
+#include "neuralnet/connection_layer.h"
namespace singa {
+
+using std::vector;
+
/************* Implementation for ConcateLayer ***********/
void ConcateLayer::Setup(const LayerProto& proto, int npartitions) {
// CHECK_EQ(npartitions, 1);
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/neuralnet/input_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/input_layer.cc b/src/neuralnet/input_layer.cc
index b1c6986..f7167da 100644
--- a/src/neuralnet/input_layer.cc
+++ b/src/neuralnet/input_layer.cc
@@ -1,8 +1,7 @@
-#include <vector>
-#include <string>
+#include "neuralnet/input_layer.h"
-#include "neuralnet/layer.h"
#include "mshadow/tensor.h"
+
namespace singa {
using namespace mshadow;
@@ -10,6 +9,8 @@ using mshadow::cpu;
using mshadow::Shape4;
using mshadow::Tensor;
+using std::string;
+using std::vector;
/************* Implementation for ParserLayer ***********/
void ParserLayer::ComputeFeature(int flag, Metric *perf) {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/neuralnet/layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/layer.cc b/src/neuralnet/layer.cc
index 7e2e107..f38d592 100644
--- a/src/neuralnet/layer.cc
+++ b/src/neuralnet/layer.cc
@@ -9,6 +9,8 @@
namespace singa {
+using std::string;
+
Layer* Layer::Create(const LayerProto& proto) {
auto* factory = Singleton<Factory<Layer>>::Instance();
Layer* layer = nullptr;
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/neuralnet/loss_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/loss_layer.cc b/src/neuralnet/loss_layer.cc
index 118456a..f9b80a9 100644
--- a/src/neuralnet/loss_layer.cc
+++ b/src/neuralnet/loss_layer.cc
@@ -1,9 +1,10 @@
+#include "neuralnet/loss_layer.h"
+
#include <glog/logging.h>
-#include "neuralnet/layer.h"
#include "mshadow/tensor.h"
-
namespace singa {
+
using namespace mshadow;
using mshadow::cpu;
@@ -14,6 +15,8 @@ using mshadow::Shape3;
using mshadow::Shape4;
using mshadow::Tensor;
+using std::string;
+using std::vector;
/********** * Implementation for EuclideanLossLayer*************************/
void EuclideanLossLayer::ComputeFeature(int flag, Metric* perf) {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/neuralnet/neuron_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/neuron_layer.cc b/src/neuralnet/neuron_layer.cc
index edfa022..a98b40d 100644
--- a/src/neuralnet/neuron_layer.cc
+++ b/src/neuralnet/neuron_layer.cc
@@ -1,10 +1,11 @@
+#include "neuralnet/neuron_layer.h"
+
#include <glog/logging.h>
#include <algorithm>
-
-#include "neuralnet/layer.h"
#include "utils/singleton.h"
#include "mshadow/tensor.h"
#include "mshadow/cxxnet_op.h"
+
namespace singa {
using namespace mshadow;
@@ -165,7 +166,7 @@ void CConvolutionLayer::ComputeGradient(int flag, Metric* perf) {
if (gsrcblob != nullptr)
gsrc.dptr = gsrcblob->mutable_cpu_data();
gbias = expr::sumall_except_dim<1>(grad);
- for(int n = 0; n < batchsize_; n++) {
+ for (int n = 0; n < batchsize_; n++) {
Im2col(src[n].dptr, channels_, height_, width_,
kernel_, kernel_, pad_, pad_, stride_, stride_, col.dptr);
gweight += dot(grad[n], col.T());
@@ -230,7 +231,7 @@ Blob<float>* RBMLayer::Sample(int flag) {
&sample_ : &neg_sample_;
}
void RBMLayer::Setup(const LayerProto& proto, int npartitions) {
- CHECK_EQ(npartitions, 1); // TODO test for npartitions > 1
+ CHECK_EQ(npartitions, 1); // TODO(wangwei) test for npartitions > 1
Layer::Setup(proto, npartitions);
hdim_ = proto.rbm_conf().hdim();
gaussian_ = proto.rbm_conf().gaussian();
@@ -523,15 +524,15 @@ void PoolingLayer::ComputeGradient(int flag, Metric* perf) {
void CPoolingLayer::Setup(const LayerProto& proto, int npartitions) {
PoolingLayer::Setup(proto, npartitions);
- if(pool_ == PoolingProto_PoolMethod_MAX)
- mask_.ReshapeLike(data_);
+ if (pool_ == PoolingProto_PoolMethod_MAX)
+ mask_.ReshapeLike(data_);
}
void CPoolingLayer::ComputeFeature(int flag, Metric* perf) {
- if(pool_ == PoolingProto_PoolMethod_MAX)
+ if (pool_ == PoolingProto_PoolMethod_MAX)
ForwardMaxPooling(srclayers_[0]->mutable_data(this)->mutable_cpu_data(),
batchsize_, channels_, height_, width_, kernel_, kernel_, pad_, pad_,
stride_, stride_, data_.mutable_cpu_data(), mask_.mutable_cpu_data());
- else if(pool_ == PoolingProto_PoolMethod_AVG)
+ else if (pool_ == PoolingProto_PoolMethod_AVG)
ForwardAvgPooling(srclayers_[0]->mutable_data(this)->mutable_cpu_data(),
batchsize_, channels_, height_, width_, kernel_, kernel_, pad_, pad_,
stride_, stride_, data_.mutable_cpu_data());
@@ -540,11 +541,11 @@ void CPoolingLayer::ComputeFeature(int flag, Metric* perf) {
}
void CPoolingLayer::ComputeGradient(int flag, Metric* perf) {
- if(pool_ == PoolingProto_PoolMethod_MAX)
+ if (pool_ == PoolingProto_PoolMethod_MAX)
BackwardMaxPooling(grad_.cpu_data(), mask_.cpu_data(), batchsize_,
channels_, height_, width_, kernel_, kernel_, pad_, pad_,
stride_, stride_,srclayers_[0]->mutable_grad(this)->mutable_cpu_data());
- else if(pool_ == PoolingProto_PoolMethod_AVG)
+ else if (pool_ == PoolingProto_PoolMethod_AVG)
BackwardAvgPooling(grad_.cpu_data(), batchsize_,
channels_, height_, width_, kernel_, kernel_, pad_, pad_,
stride_, stride_,srclayers_[0]->mutable_grad(this)->mutable_cpu_data());
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/neuralnet/output_layer.cc
----------------------------------------------------------------------
diff --git a/src/neuralnet/output_layer.cc b/src/neuralnet/output_layer.cc
index dfc547b..535480e 100644
--- a/src/neuralnet/output_layer.cc
+++ b/src/neuralnet/output_layer.cc
@@ -2,5 +2,4 @@
namespace singa {
-
-}
+} // namespace singa
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/test/test_common.cc
----------------------------------------------------------------------
diff --git a/src/test/test_common.cc b/src/test/test_common.cc
index b84e860..e30c9cb 100644
--- a/src/test/test_common.cc
+++ b/src/test/test_common.cc
@@ -1,113 +1,112 @@
-#include "gtest/gtest.h"
-#include "utils/common.h"
-#include <unordered_map>
#include <string>
+#include <unordered_map>
#include <vector>
+#include "gtest/gtest.h"
+#include "utils/common.h"
using std::string;
using std::vector;
using namespace singa;
TEST(CommonTest, TestIntVecToString) {
-
- vector<int> num_vec {2, 3, 5, 7, 11};
- string str = "(2, 3, 5, 7, 11, )";
- ASSERT_EQ(str, IntVecToString(num_vec));
+ vector<int> num_vec {2, 3, 5, 7, 11};
+ string str = "(2, 3, 5, 7, 11, )";
+ ASSERT_EQ(str, IntVecToString(num_vec));
}
TEST(CommonTest, TestStringPrintf) {
- const char* str_a = "abc";
- const char* str_b = "edfgh";
- const char* str_c = " !@#";
- const char* str_d = "1";
- const char* str_e = "2";
- const char* str_f = "3";
+ const char* str_a = "abc";
+ const char* str_b = "edfgh";
+ const char* str_c = " !@#";
+ const char* str_d = "1";
+ const char* str_e = "2";
+ const char* str_f = "3";
- string fmt_a = "%s%s%s";
- string fmt_b = "[%s] [%s] [%s] ";
+ string fmt_a = "%s%s%s";
+ string fmt_b = "[%s] [%s] [%s] ";
- string str_d_a = "abcedfgh !@#";
- string str_d_b = "[1] [2] [3] ";
+ string str_d_a = "abcedfgh !@#";
+ string str_d_b = "[1] [2] [3] ";
- ASSERT_EQ(str_d_a, StringPrintf(fmt_a, str_a, str_b, str_c));
- ASSERT_EQ(str_d_b, StringPrintf(fmt_b, str_d, str_e, str_f));
+ ASSERT_EQ(str_d_a, StringPrintf(fmt_a, str_a, str_b, str_c));
+ ASSERT_EQ(str_d_b, StringPrintf(fmt_b, str_d, str_e, str_f));
}
TEST(CommonTest, TestGCDLCM) {
- int a = 2, b = 5, c = 10, d = 15;
+ int a = 2, b = 5, c = 10, d = 15;
- ASSERT_EQ(1, gcd(a, b));
- ASSERT_EQ(5, gcd(c, d));
- ASSERT_EQ(10, LeastCommonMultiple(b, c));
- ASSERT_EQ(30, LeastCommonMultiple(c, d));
+ ASSERT_EQ(1, gcd(a, b));
+ ASSERT_EQ(5, gcd(c, d));
+ ASSERT_EQ(10, LeastCommonMultiple(b, c));
+ ASSERT_EQ(30, LeastCommonMultiple(c, d));
}
TEST(CommonTest, TestMetric) {
- string str, msg;
- Metric metric;
- metric.Add("a", 0.5);
- metric.Add("b", 0.5);
- metric.Add("a", 1.5);
- str = metric.ToLogString();
- msg = metric.ToString();
- metric.Reset();
- metric.ParseFrom(msg);
- ASSERT_EQ(str, metric.ToLogString());
+ string str, msg;
+ Metric metric;
+ metric.Add("a", 0.5);
+ metric.Add("b", 0.5);
+ metric.Add("a", 1.5);
+ str = metric.ToLogString();
+ msg = metric.ToString();
+ metric.Reset();
+ metric.ParseFrom(msg);
+ ASSERT_EQ(str, metric.ToLogString());
}
TEST(CommonTest, TestSlice) {
- vector<vector<int>> slices_0;
- vector<int> sizes {14112, 96, 256, 884736, 384};
- ASSERT_EQ(slices_0, Slice(0, sizes));
-
- vector<vector<int>> slices_1 {
- { 14112 },
- { 96 },
- { 256 },
- { 884736 },
- { 384 },
- };
-
- vector<vector<int>> slices_2 {
- { 14112 },
- { 96 },
- { 256 },
- { 435328, 449408 },
- { 384 },
- };
-
- vector<vector<int>> slices_4 {
- { 14112 },
- { 96 },
- { 256 },
- { 210432,224896,224896,224512 },
- { 384 },
- };
-
- vector<vector<int>> slices_8 {
- { 14112 },
- { 96 },
- { 256 },
- { 97984,112448,112448,112448,112448,112448,112448,112064 },
- { 384 },
- };
-
- ASSERT_EQ(slices_1, Slice(1, sizes));
- ASSERT_EQ(slices_2, Slice(2, sizes));
- ASSERT_EQ(slices_4, Slice(4, sizes));
- ASSERT_EQ(slices_8, Slice(8, sizes));
+ vector<vector<int>> slices_0;
+ vector<int> sizes {14112, 96, 256, 884736, 384};
+ ASSERT_EQ(slices_0, Slice(0, sizes));
+
+ vector<vector<int>> slices_1 {
+ {14112},
+ {96},
+ {256},
+ {884736},
+ {384},
+ };
+
+ vector<vector<int>> slices_2 {
+ {14112},
+ {96},
+ {256},
+ {435328, 449408},
+ {384},
+ };
+
+ vector<vector<int>> slices_4 {
+ {14112},
+ {96},
+ {256},
+ {210432, 224896, 224896, 224512},
+ {384},
+ };
+
+ vector<vector<int>> slices_8 {
+ {14112},
+ {96},
+ {256},
+ {97984, 112448, 112448, 112448, 112448, 112448, 112448, 112064},
+ {384},
+ };
+
+ ASSERT_EQ(slices_1, Slice(1, sizes));
+ ASSERT_EQ(slices_2, Slice(2, sizes));
+ ASSERT_EQ(slices_4, Slice(4, sizes));
+ ASSERT_EQ(slices_8, Slice(8, sizes));
}
TEST(CommonTest, TestPartitionSlices) {
- vector<int> slices {
- 97984,112448,112448,112448,112448,112448,112448,112064
- };
- vector<int> box_1 { 0, 0, 0, 0, 0, 0, 0, 0 };
- vector<int> box_2 { 0, 0, 0, 0, 1, 1, 1, 1 };
- vector<int> box_4 { 0, 0, 1, 1, 2, 2, 3, 3 };
- vector<int> box_8 { 0, 1, 2, 3, 4, 5, 6, 7 };
- ASSERT_EQ(box_1, PartitionSlices(1, slices));
- ASSERT_EQ(box_2, PartitionSlices(2, slices));
- ASSERT_EQ(box_4, PartitionSlices(4, slices));
- ASSERT_EQ(box_8, PartitionSlices(8, slices));
+ vector<int> slices {
+ 97984, 112448, 112448, 112448, 112448, 112448, 112448, 112064
+ };
+ vector<int> box_1 {0, 0, 0, 0, 0, 0, 0, 0};
+ vector<int> box_2 {0, 0, 0, 0, 1, 1, 1, 1};
+ vector<int> box_4 {0, 0, 1, 1, 2, 2, 3, 3};
+ vector<int> box_8 {0, 1, 2, 3, 4, 5, 6, 7};
+ ASSERT_EQ(box_1, PartitionSlices(1, slices));
+ ASSERT_EQ(box_2, PartitionSlices(2, slices));
+ ASSERT_EQ(box_4, PartitionSlices(4, slices));
+ ASSERT_EQ(box_8, PartitionSlices(8, slices));
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/d5d817e1/src/utils/common.cc
----------------------------------------------------------------------
diff --git a/src/utils/common.cc b/src/utils/common.cc
index 3c3dc39..4cf9a89 100644
--- a/src/utils/common.cc
+++ b/src/utils/common.cc
@@ -357,8 +357,8 @@ void ForwardMaxPooling(const float* bottom, const int num, const int channels,
const int height, const int width, const int kernel_h, const int kernel_w,
const int pad_h, const int pad_w, const int stride_h, const int stride_w,
float* top, float* mask) {
- int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
- int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ int top_height = (height + pad_h * 2 -kernel_h) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w) / stride_w + 1;
int top_count = num * top_height * top_width * channels;
for (int i = 0; i < top_count; i++) {
mask[i] = -1;
@@ -402,8 +402,8 @@ void BackwardMaxPooling(const float* top, const float* mask, const int num,
const int kernel_h, const int kernel_w, const int pad_h, const int pad_w,
const int stride_h, const int stride_w,
float* bottom) {
- int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
- int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ int top_height = (height + pad_h * 2 -kernel_h) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w) / stride_w + 1;
const int top_offset = top_height * top_width;
const int bottom_offset = height * width;
memset(bottom, 0, sizeof(float) * num * channels * bottom_offset);
@@ -427,8 +427,8 @@ void ForwardAvgPooling(const float* bottom, const int num, const int channels,
const int height, const int width, const int kernel_h, const int kernel_w,
const int pad_h, const int pad_w, const int stride_h, const int stride_w,
float* top) {
- int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
- int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ int top_height = (height + pad_h * 2 -kernel_h) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w) / stride_w + 1;
int top_count = num * top_height * top_width * channels;
for (int i = 0; i < top_count; i++) {
top[i] = 0;
@@ -470,8 +470,8 @@ void BackwardAvgPooling(const float* top, const int num, const int channels,
const int height, const int width, const int kernel_h, const int kernel_w,
const int pad_h, const int pad_w, const int stride_h, const int stride_w,
float* bottom) {
- int top_height = (height + pad_h * 2 -kernel_h ) / stride_h + 1;
- int top_width = (width + pad_w * 2 -kernel_w ) / stride_w + 1;
+ int top_height = (height + pad_h * 2 -kernel_h) / stride_h + 1;
+ int top_width = (width + pad_w * 2 -kernel_w) / stride_w + 1;
const int top_offset = top_height * top_width;
const int bottom_offset = height * width;
memset(bottom, 0, sizeof(float) * num * channels * bottom_offset);
@@ -495,7 +495,6 @@ void BackwardAvgPooling(const float* top, const int num, const int channels,
bottom[index] += top[top_index] / pool_size;
}
}
-
}
}
top += top_offset;
[02/21] incubator-singa git commit: SINGA-69 Add debug option in
compilation configuration Two updates: 1.User can use "--enable-singatest"
option while running configure script to generate test binary file and
corresponding library. 2.User can use "--en
Posted by wa...@apache.org.
SINGA-69 Add debug option in compilation configuration
Two updates:
1.User can use "--enable-singatest" option while running configure script to generate test binary file and corresponding library.
2.User can use "--enable-debug" option while running configure script to compile with "-g" parameter. The optimization level is adapted to "-O2".
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/e0de439e
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/e0de439e
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/e0de439e
Branch: refs/heads/tutorial
Commit: e0de439ea797b3276bf6e59e01a3c243cea57ece
Parents: bc400a7
Author: xiezl <xi...@comp.nus.edu.sg>
Authored: Sat Sep 12 21:07:27 2015 +0800
Committer: xiezl <xi...@comp.nus.edu.sg>
Committed: Sat Sep 12 21:07:27 2015 +0800
----------------------------------------------------------------------
Makefile.am | 55 ++++++++++++++++++++++++++++++++-----------------------
configure.ac | 27 ++++++++++++++++++++++++++-
2 files changed, 58 insertions(+), 24 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e0de439e/Makefile.am
----------------------------------------------------------------------
diff --git a/Makefile.am b/Makefile.am
index 732b84e..baff947 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -4,9 +4,12 @@ AUTOMAKE_OPTIONS = foreign subdir-objects
AM_CPPFLAGS = -I$(top_srcdir)/src
MSHADOW_FLAGS = -DMSHADOW_USE_CUDA=0 -DMSHADOW_USE_CBLAS=1 -DMSHADOW_USE_MKL=0
+DEFAULT_FLAGS = -Wall -pthread -fPIC -std=c++11 -Wno-unknown-pragmas \
+ $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -DTHREADED
-AC_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -Wno-unknown-pragmas \
- $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops
+CFLAGS = -O2 $(DEBUG)
+CXXFLAGS = -O2 $(DEBUG)
+AC_CXXFLAGS = -O2 $(DEBUG)
INCLUDES = -I$(top_srcdir)/include
@@ -87,31 +90,28 @@ TEST_SRCS := include/gtest/gtest_main.cc \
src/test/test_paramslicer.cc \
src/test/test_shard.cc
-lib_LTLIBRARIES = libsinga.la
+#EXTRA_PROGRAMS = $(PROGS)
+EXTRA_PROGRAMS = singatest
+#EXTRA_LTLIBRARIES = $(LTLIBS)
+EXTRA_LTLIBRARIES = libgtest.la
+
+lib_LTLIBRARIES = libsinga.la $(LTLIBS)
+bin_PROGRAMS = singa singatool $(PROGS)
+
+#lib_LTLIBRARIES = libsinga.la
libsinga_la_SOURCES = $(PROTO_HDRS) $(PROTO_SRCS) $(SINGA_HDRS) $(SINGA_SRCS)
-libsinga_la_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -Wno-unknown-pragmas \
- $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -gdwarf-2 -msse3 \
- -gstrict-dwarf -Woverloaded-virtual -DTHREADED -fpermissive
+libsinga_la_CXXFLAGS = $(DEFAULT_FLAGS) -gdwarf-2 -msse3 \
+ -gstrict-dwarf -Woverloaded-virtual -fpermissive
if LMDB
libsinga_la_CXXFLAGS += -DUSE_LMDB
endif
libsinga_la_LDFLAGS = -I./include
-lib_LTLIBRARIES += libgtest.la
-libgtest_la_SOURCES = $(GTEST_HDRS) $(GTEST_SRCS)
-libgtest_la_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -Wno-unknown-pragmas \
- $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -gdwarf-2 -msse3 \
- -gstrict-dwarf -Woverloaded-virtual -DTHREADED -fpermissive
-if LMDB
-libgtest_la_CXXFLAGS += -DUSE_LMDB
-endif
-libgtest_la_LDFLAGS = -I./include
-bin_PROGRAMS = singa
+#bin_PROGRAMS = singa
singa_SOURCES = src/main.cc
-singa_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -MMD -Wno-unknown-pragmas \
- $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -DTHREADED
+singa_CXXFLAGS = $(DEFAULT_FLAGS) -MMD
singa_LDFLAGS = -I./include \
-lsinga \
-lglog \
@@ -125,9 +125,9 @@ if LMDB
singa_LDFLAGS += -llmdb
endif
-bin_PROGRAMS += singatool
+#bin_PROGRAMS += singatool
singatool_SOURCES = src/utils/tool.cc
-singatool_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -MMD -Wno-unknown-pragmas \
+singatool_CXXFLAGS = -Wall -pthread -fPIC -std=c++11 -MMD -Wno-unknown-pragmas \
-funroll-loops -DTHREADED
singatool_LDFLAGS = -I./include \
-lsinga \
@@ -135,10 +135,19 @@ singatool_LDFLAGS = -I./include \
-lprotobuf \
-lzookeeper_mt
-bin_PROGRAMS += singatest
+#lib_LTLIBRARIES += libgtest.la
+libgtest_la_SOURCES = $(GTEST_HDRS) $(GTEST_SRCS)
+libgtest_la_CXXFLAGS = $(DEFAULT_FLAGS) -gdwarf-2 -msse3 \
+ -gstrict-dwarf -Woverloaded-virtual -fpermissive
+if LMDB
+libgtest_la_CXXFLAGS += -DUSE_LMDB
+endif
+libgtest_la_LDFLAGS = -I./include
+
+#bin_PROGRAMS += singatest
+
singatest_SOURCES = $(GTEST_HDRS) $(TEST_SRCS)
-singatest_CXXFLAGS = -O3 -Wall -pthread -fPIC -std=c++11 -MMD -Wno-unknown-pragmas \
- $(MSHADOW_FLAGS) -DCPU_ONLY=1 -funroll-loops -DTHREADED
+singatest_CXXFLAGS = $(DEFAULT_FLAGS)
singatest_LDFLAGS = -I./include \
-lsinga \
-lglog \
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/e0de439e/configure.ac
----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index 569c406..1ff528f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -8,7 +8,7 @@ AC_CONFIG_MACRO_DIR(config)
AC_CONFIG_SRCDIR([src/utils/common.cc])
AC_CONFIG_HEADER([config.h])
AM_INIT_AUTOMAKE([subdir-objects foreign])
-m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
+#m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
ACLOCAL_AMFLAGS = -I m4 --install
# Checks for programs.
@@ -44,6 +44,31 @@ if test x"$enable_lmdb" = x"yes"; then
AC_DEFINE(LMDB, 1, [Enable Option layer])
fi
+AC_ARG_ENABLE(singatest,
+ AS_HELP_STRING([--enable-singatest],[enable singa test]),
+ [enable_singatest=yes],[enable_singatest=no])
+AM_CONDITIONAL(SINGATEST, test "$enable_singatest" = yes)
+if test x"$enable_singatest" != x"no"; then
+ PROGS='singatest'
+ LTLIBS='libgtest.la'
+else
+ PROGS=''
+ LTLIBS=''
+fi
+AC_SUBST([PROGS])
+AC_SUBST([LTLIBS])
+
+AC_ARG_ENABLE(debug,
+ AS_HELP_STRING([--enable-debug],[enable debug mode]),
+ [enable_debug=yes],[enable_debug=no])
+AM_CONDITIONAL(DEBUG, test "$enable_debug" = yes)
+if test x"$enable_debug" != x"no"; then
+ DEBUG='-g'
+else
+ DEBUG=''
+fi
+AC_SUBST([DEBUG])
+
#AC_CHECK_LIB([opencv_imgproc], [main], [], [
# AC_MSG_ERROR([unable to find opencv_imgproc lib])
# ])
[03/21] incubator-singa git commit: SINGA-69 Add debug option in
compilation configuration Build in a clean mode. Change "--enable-singatest"
to "--enable-test".
Posted by wa...@apache.org.
SINGA-69 Add debug option in compilation configuration
Build in a clean mode.
Change "--enable-singatest" to "--enable-test".
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/5d7d53f5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/5d7d53f5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/5d7d53f5
Branch: refs/heads/tutorial
Commit: 5d7d53f52451840a09d577423745dc977ae8184f
Parents: e0de439
Author: xiezl <xi...@comp.nus.edu.sg>
Authored: Sun Sep 13 16:21:01 2015 +0800
Committer: xiezl <xi...@comp.nus.edu.sg>
Committed: Sun Sep 13 16:21:01 2015 +0800
----------------------------------------------------------------------
configure.ac | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/5d7d53f5/configure.ac
----------------------------------------------------------------------
diff --git a/configure.ac b/configure.ac
index 1ff528f..fa54546 100644
--- a/configure.ac
+++ b/configure.ac
@@ -8,7 +8,7 @@ AC_CONFIG_MACRO_DIR(config)
AC_CONFIG_SRCDIR([src/utils/common.cc])
AC_CONFIG_HEADER([config.h])
AM_INIT_AUTOMAKE([subdir-objects foreign])
-#m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
+m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
ACLOCAL_AMFLAGS = -I m4 --install
# Checks for programs.
@@ -44,11 +44,11 @@ if test x"$enable_lmdb" = x"yes"; then
AC_DEFINE(LMDB, 1, [Enable Option layer])
fi
-AC_ARG_ENABLE(singatest,
- AS_HELP_STRING([--enable-singatest],[enable singa test]),
- [enable_singatest=yes],[enable_singatest=no])
-AM_CONDITIONAL(SINGATEST, test "$enable_singatest" = yes)
-if test x"$enable_singatest" != x"no"; then
+AC_ARG_ENABLE(test,
+ AS_HELP_STRING([--enable-test],[enable singa test]),
+ [enable_test=yes],[enable_test=no])
+AM_CONDITIONAL(SINGATEST, test "$enable_test" = yes)
+if test x"$enable_test" != x"no"; then
PROGS='singatest'
LTLIBS='libgtest.la'
else
[09/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
SINGA-68 Preparation for Release -- License etc.
Minor edits on the README file
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/17914421
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/17914421
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/17914421
Branch: refs/heads/tutorial
Commit: 1791442112091814c74a96418c99883ce9a0927b
Parents: 9cded5e
Author: Anh Dinh <ug...@gmail.com>
Authored: Tue Sep 15 21:34:01 2015 +0800
Committer: wang sheng <wa...@gmail.com>
Committed: Wed Sep 16 11:09:25 2015 +0800
----------------------------------------------------------------------
README | 20 --------------------
README.md | 42 +++++++++++++++++++++++-------------------
2 files changed, 23 insertions(+), 39 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/17914421/README
----------------------------------------------------------------------
diff --git a/README b/README
deleted file mode 100644
index 6ccaac4..0000000
--- a/README
+++ /dev/null
@@ -1,20 +0,0 @@
-TODO
-
-##Apache SINGA
-
-Distributed deep learning system
-
-[Project Website](http://singa.incubator.apache.org)
-
-All the details can be found in project website.
-
-The current code depends on the following external libraries:
- * glog (New BSD)
- * google-protobuf (New BSD)
- * openblas (New BSD)
- * zeromq (LGPLv3 + static link exception)
- * czmq (Mozilla Public License Version 2.0)
- * zookeeper (Apache 2.0)
- * lmdb (OpenLDAP)
-
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/17914421/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 0777c6b..b7d4528 100644
--- a/README.md
+++ b/README.md
@@ -12,43 +12,47 @@ All the details can be found in [Project Website](http://singa.incubator.apache.
* [Development Mailing List](mailto:dev-subscribe@singa.incubator.apache.org)([Archive](http://mail-archives.apache.org/mod_mbox/singa-dev/))
* [Commits Mailing List](mailto:commits-subscribe@singa.incubator.apache.org)([Archive](http://mail-archives.apache.org/mod_mbox/singa-commits/))
-##Documentation
+##Dependencies
+The current code depends on the following external libraries:
+ * glog (New BSD)
+ * google-protobuf (New BSD)
+ * openblas (New BSD)
+ * zeromq (LGPLv3 + static link exception)
+ * czmq (Mozilla Public License Version 2.0)
+ * zookeeper (Apache 2.0)
+ * lmdb (OpenLDAP)
-Documentation is available in [Official Documentation](https://singa.incubator.apache.org/docs/overview.html#).
+##Documentation
-##Building SINGA
+Full documentation is available online at [Official Documentation](https://singa.incubator.apache.org/docs/overview.html#).
-Just clone our github repo and execute following commands:
- $ git clone git@github.com:apache/incubator-singa.git
- $ cd incubator-singa
+##Building SINGA
+
$ ./autogen.sh
$ ./configure
$ make
-You can also download the release package from the [Project Website](http://singa.incubator.apache.org) and follow the instructions on [Official Installation Guide](http://singa.incubator.apache.org/docs/installation.html).
-
-##Running an Example
+##Running Examples
-After installation, you may want to run an example to try SINGA.
-Let us train the [CNN model](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks) over the
-[CIFAR-10](http://www.cs.toronto.edu/~kriz/cifar.html) dataset without parallelism as an instance.
-The hyper-parameters are set following
-[cuda-convnet](https://code.google.com/p/cuda-convnet/). More details about this example are
-available at [CNN example](http://singa.incubator.apache.org/docs/cnn).
+Let us train the [CNN
+model](http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks) over the
+[CIFAR-10](http://www.cs.toronto.edu/~kriz/cifar.html) dataset without parallelism as an example. The hyper-parameters
+are set following [cuda-convnet](https://code.google.com/p/cuda-convnet/). More details about this example are available
+at [CNN example](http://singa.incubator.apache.org/docs/cnn).
-We firstly download the dataset and create the data shard:
+First, download the dataset and create data shards:
$ cd examples/cifar10/
$ make download
$ make create
-Then we start training:
+Next, start the training:
$ cd ../../
- $ ./bin/singa-run.sh -conf examples/cifar10/job.conf
+ $ ./bin/singa-run.sh -conf examples/cifar10/job.conf
-Now we just need to wait until it is done! About this part, you can also refer to the [Quick Start](http://singa.incubator.apache.org/docs/quick-start.html).
+Now we just need to wait until it is done!
##LICENSE
[21/21] incubator-singa git commit: SINGA-10 Add Support for
Recurrent Neural Networks (RNN)
Posted by wa...@apache.org.
SINGA-10 Add Support for Recurrent Neural Networks (RNN)
Revise Makefile.example;
Add job.conf;
Update src/utils/tool.cc to properly parse job.conf;
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/cca37b97
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/cca37b97
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/cca37b97
Branch: refs/heads/tutorial
Commit: cca37b970bbd782954d1428c488ee72eae57c8bf
Parents: b4a8d2b
Author: chonho <le...@comp.nus.edu.sg>
Authored: Tue Sep 15 19:07:26 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Wed Sep 16 11:39:16 2015 +0800
----------------------------------------------------------------------
examples/rnnlm/Makefile.example | 16 ++---
examples/rnnlm/job.conf | 136 +++++++++++++++++++++++++++++++++++
examples/rnnlm/rnnlm.cc | 1 +
examples/rnnlm/rnnlm.proto | 3 +-
src/utils/tool.cc | 43 ++++++++++-
5 files changed, 185 insertions(+), 14 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/cca37b97/examples/rnnlm/Makefile.example
----------------------------------------------------------------------
diff --git a/examples/rnnlm/Makefile.example b/examples/rnnlm/Makefile.example
index b4505cf..83e2522 100644
--- a/examples/rnnlm/Makefile.example
+++ b/examples/rnnlm/Makefile.example
@@ -9,25 +9,23 @@ numclass = 100
dirshards = train_shard valid_shard test_shard
-.PHONY: all download create
-download: rnnlm
-
-rnnlm:
+download:
wget $(filelink)/$(filename)
tar zxf $(filename)
rm $(filename)
create:
- $(CXX) create_shard.cc -std=c++11 -lsinga -lprotobuf -lzookeeper_mt -lglog -I../../include \
+ protoc --proto_path=../../src/proto --proto_path=. --cpp_out=. rnnlm.proto
+ $(CXX) create_shard.cc rnnlm.pb.cc -std=c++11 -lsinga -lprotobuf -lzookeeper_mt -lglog -I../../include -I../../include/proto \
-L../../.libs/ -L/usr/local/lib -Wl,-unresolved-symbols=ignore-in-shared-libs -Wl,-rpath=../../.libs/ \
-o create_shard.bin
for d in $(dirshards); do mkdir -p $${d}; done
- ./create_shard.bin -train $(dirname)/train -class_size $(numclass) -test $(dirname)/test
+ ./create_shard.bin -train $(dirname)/train -test $(dirname)/test -valid $(dirname)/valid -class_size $(numclass)
-all:
+rnnlm:
protoc --proto_path=../../src/proto --proto_path=. --cpp_out=. rnnlm.proto
- $(CXX) main.cc rnnlm.cc rnnlm.pb.cc $(MSHADOW_FLAGS) -std=c++11 -lsinga -lglog -lprotobuf -lopenblas -I../../include\
- -I../../include/proto/ -L../../.libs/ -L/usr/local -Wl,-unresolved-symbols=ignore-in-shared-libs -Wl,-rpath=../../.libs/\
+ $(CXX) main.cc rnnlm.cc rnnlm.pb.cc $(MSHADOW_FLAGS) -std=c++11 -lsinga -lglog -lprotobuf -lopenblas -I../../include -I../../include/proto \
+ -L../../.libs/ -L/usr/local -Wl,-unresolved-symbols=ignore-in-shared-libs -Wl,-rpath=../../.libs/\
-o rnnlm.bin
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/cca37b97/examples/rnnlm/job.conf
----------------------------------------------------------------------
diff --git a/examples/rnnlm/job.conf b/examples/rnnlm/job.conf
new file mode 100644
index 0000000..81bdb94
--- /dev/null
+++ b/examples/rnnlm/job.conf
@@ -0,0 +1,136 @@
+cluster {
+ nworker_groups: 1
+ nserver_groups: 1
+ nservers_per_group: 1
+ nworkers_per_group: 1
+ nservers_per_procs: 1
+ nworkers_per_procs: 1
+ workspace: "examples/rnnlm/"
+}
+
+name: "recurrent-neural-network-language-model"
+#To scan the training file (71350) 30 times
+train_steps:214050
+#To scan the validation file (5829) once
+test_steps:583
+test_freq:7135
+#disp_freq is specific to training
+disp_freq:7135
+
+train_one_batch {
+ alg: kBP
+}
+
+updater{
+ type: kSGD
+ #weight_decay:0.0000001
+ learning_rate {
+ type: kFixedStep
+ fixedstep_conf:{
+ step:0
+ step:42810
+ step:49945
+ step:57080
+ step:64215
+ step_lr:0.1
+ step_lr:0.05
+ step_lr:0.025
+ step_lr:0.0125
+ step_lr:0.00625
+ }
+ }
+}
+
+neuralnet {
+layer {
+ name: "data"
+ user_type: "kRnnData"
+ [singa.input_conf] {
+ path: "examples/rnnlm/train_shard"
+ max_window: 10
+ }
+ exclude: kTest
+}
+
+layer {
+ name: "data"
+ user_type: "kRnnData"
+ [singa.input_conf] {
+ path: "examples/rnnlm/test_shard"
+ max_window: 10
+ }
+ exclude: kTrain
+}
+
+layer{
+ name:"wordlayer"
+ user_type: "kWord"
+ srclayers: "data"
+}
+
+layer{
+ name:"labellayer"
+ user_type: "kRnnLabel"
+ srclayers: "data"
+}
+
+layer{
+ name: "embeddinglayer"
+ user_type: "kEmbedding"
+ [singa.embedding_conf] {
+ word_dim: 15
+ vocab_size: 3720
+ }
+ srclayers: "wordlayer"
+ param {
+ name: "w1"
+ init {
+ type: kUniform
+ low:-0.3
+ high:0.3
+ }
+ }
+}
+
+layer{
+ name: "hiddenlayer"
+ user_type: "kHidden"
+ srclayers:"embeddinglayer"
+ param{
+ name: "w2"
+ init {
+ type: kUniform
+ low:-0.3
+ high:0.3
+ }
+ }
+}
+layer{
+ name: "outputlayer"
+ user_type: "kOutput"
+ srclayers:"hiddenlayer"
+ srclayers:"labellayer"
+ [singa.output_conf] {
+ nclass:100
+ vocab_size: 3720
+ }
+ param{
+ name: "w3"
+ init {
+ type: kUniform
+ low:-0.3
+ high:0.3
+ }
+ }
+ param{
+ name: "w4"
+ init {
+ type: kUniform
+ low:-0.3
+ high:0.3
+ }
+ }
+}
+
+}
+
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/cca37b97/examples/rnnlm/rnnlm.cc
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.cc b/examples/rnnlm/rnnlm.cc
index 180300f..4d39b5f 100644
--- a/examples/rnnlm/rnnlm.cc
+++ b/examples/rnnlm/rnnlm.cc
@@ -69,6 +69,7 @@ void WordLayer::Setup(const LayerProto& proto, int npartitions) {
Layer::Setup(proto, npartitions);
CHECK_EQ(srclayers_.size(), 1);
int max_window = static_cast<RnnDataLayer*>(srclayers_[0])->max_window();
+ LOG(ERROR) << "clee " << max_window;
data_.Reshape(vector<int>{max_window});
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/cca37b97/examples/rnnlm/rnnlm.proto
----------------------------------------------------------------------
diff --git a/examples/rnnlm/rnnlm.proto b/examples/rnnlm/rnnlm.proto
index 65c34ec..01580c1 100644
--- a/examples/rnnlm/rnnlm.proto
+++ b/examples/rnnlm/rnnlm.proto
@@ -1,6 +1,5 @@
package singa;
import "job.proto";
-import "common.proto";
message EmbeddingProto {
@@ -30,4 +29,4 @@ message WordRecord {
optional int32 class_index = 3;
optional int32 class_start = 4;
optional int32 class_end = 5;
-}
\ No newline at end of file
+}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/cca37b97/src/utils/tool.cc
----------------------------------------------------------------------
diff --git a/src/utils/tool.cc b/src/utils/tool.cc
index ba453b2..305cbe7 100644
--- a/src/utils/tool.cc
+++ b/src/utils/tool.cc
@@ -20,6 +20,7 @@
*************************************************************/
#include <glog/logging.h>
+#include <google/protobuf/text_format.h>
#include <algorithm>
#include <fstream>
#include <iostream>
@@ -51,12 +52,48 @@ int create() {
return SUCCESS;
}
+// extract cluster configuration part from the job config file
+// TODO improve this function to make it robust
+const std::string extract_cluster(const char* jobfile) {
+ std::ifstream fin;
+ fin.open(jobfile, std::ifstream::in);
+ CHECK(fin.is_open()) << "cannot open job conf file " << jobfile;
+ std::string line;
+ std::string cluster;
+ while (std::getline(fin, line)) {
+ // end of extraction (cluster config has not nested messages)
+ if (line.find("}") != std::string::npos && cluster.length()) {
+ cluster += line.substr(0, line.find("}"));
+ break;
+ }
+ unsigned int pos = 0;
+ while (pos < line.length() && line.at(pos) == ' ' ) pos++;
+ if (line.find("cluster", pos) == pos) { // start with <whitespace> cluster
+ pos += 7;
+ do { // looking for the first '{', which may be in the next lines
+ while (pos < line.length() &&
+ (line.at(pos) == ' ' || line.at(pos) =='\t')) pos++;
+ if (pos < line.length()) {
+ CHECK_EQ(line.at(pos), '{') << "error around 'cluster' field";
+ cluster = " "; // start extraction
+ break;
+ } else
+ pos = 0;
+ }while(std::getline(fin, line));
+ } else if (cluster.length()) {
+ cluster += line + "\n";
+ }
+ }
+ return cluster;
+}
+
+
// generate a host list
int genhost(char* job_conf) {
// compute required #process from job conf
- singa::JobProto job;
- singa::ReadProtoFromTextFile(job_conf, &job);
- singa::ClusterProto cluster = job.cluster();
+ singa::ClusterProto cluster;
+ google::protobuf::TextFormat::ParseFromString(extract_cluster(job_conf),
+ &cluster);
int nworker_procs = cluster.nworker_groups() * cluster.nworkers_per_group()
/ cluster.nworkers_per_procs();
int nserver_procs = cluster.nserver_groups() * cluster.nservers_per_group()
[08/21] incubator-singa git commit: SINGA-57 Improve Distributed
Hogwild
Posted by wa...@apache.org.
SINGA-57 Improve Distributed Hogwild
The ClusterProto::sync_freq field controls the frequency of sync between
server groups.
After updating of Param (slice), the server checks the num of updates
since last sync. It also checks the num of pending syncs (i.e., requests
haven't received reponses) to avoid sending too many msgs to stopped
servers (the msgs would be occupy the memory of the sending buffer)
The server respones to every sync requests with the latest Param values.
Note: current does not support (there is bug) multiple worker groups in
one process for the distributed hogwild framework. We recommend to
replace this cluster topology with in-memory hogwild, i.e., launching
one worker group with multiple workers and one server group.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/ed9e3736
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/ed9e3736
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/ed9e3736
Branch: refs/heads/tutorial
Commit: ed9e37369c69dd76078e8285bc33d6b04ba60e9f
Parents: d5d817e
Author: Wei Wang <wa...@comp.nus.edu.sg>
Authored: Tue Sep 15 15:51:28 2015 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Tue Sep 15 16:03:37 2015 +0800
----------------------------------------------------------------------
examples/cifar10/job.conf | 2 +-
include/trainer/server.h | 62 +++++++++++----
include/trainer/trainer.h | 10 ---
include/utils/cluster.h | 16 ++--
src/proto/common.proto | 1 -
src/proto/job.proto | 7 +-
src/trainer/server.cc | 167 ++++++++++++++++++-----------------------
src/trainer/trainer.cc | 124 ++++++++----------------------
src/utils/cluster.cc | 21 ++++++
9 files changed, 190 insertions(+), 220 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/examples/cifar10/job.conf
----------------------------------------------------------------------
diff --git a/examples/cifar10/job.conf b/examples/cifar10/job.conf
index b36c45a..343d969 100644
--- a/examples/cifar10/job.conf
+++ b/examples/cifar10/job.conf
@@ -2,7 +2,7 @@ name: "cifar10-convnet"
train_steps: 1000
test_steps: 100
test_freq:300
-disp_freq:30
+disp_freq: 30
train_one_batch {
alg: kBP
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/include/trainer/server.h
----------------------------------------------------------------------
diff --git a/include/trainer/server.h b/include/trainer/server.h
index 8cc37c5..869d10a 100644
--- a/include/trainer/server.h
+++ b/include/trainer/server.h
@@ -23,8 +23,8 @@ class Server{
Server(int thread_id, int group_id, int server_id);
virtual ~Server();
void Setup(const UpdaterProto& proto,
- std::unordered_map<int, ParamEntry*>* shard,
- const std::vector<int>& slice2group);
+ const std::vector<int>& slice2group,
+ const std::vector<int>& slice2server);
void Run();
const int grp_id() const {
return grp_id_;
@@ -38,13 +38,30 @@ class Server{
/**
* Process GET request.
*
- * @return the orignal message or response message
+ * @return the orignal message or a response message which contains the values
+ * of the Param with the request version.
*/
virtual Msg* HandleGet(Msg** msg);
/**
* Process Update request.
*
+ * It waits until received the gradients from all workers from the same worker
+ * group. After updating, it responses to each sender with the new Param
+ * values. It may generate a sync message to the server group that maintains
+ * the global version of the updated Param (slice).
+ *
+ * Note: there is no counter for each worker group on the number of received
+ * update requests. Hence it is possible that the server would conduct the
+ * update when it receives x requests from group a and y requests from group
+ * b where x + y = group size. To avoid this problem, we can
+ * 1. maintain request list for each group for each Param at the server side
+ * 2. do not span a worker group among multiple nodes. then the updates from
+ * the same group would be locally aggregated on the worker node. And the
+ * server would conduct the update immediately after receiving the aggregated
+ * request.
+ * 3. launch only one worker group.
+ *
* @return the orignal message or response message
*/
const std::vector<Msg*> HandleUpdate(Msg **msg);
@@ -52,30 +69,47 @@ class Server{
/**
* Process PUT request.
*
- * @return the original message or response message. If we don't want need to
+ * @return the original message or response message. If we don't want to
* acknowledge the put request, then return nullptr.
*/
virtual Msg* HandlePut(Msg **msg);
/**
- * TODO Process SYNC request.
- */
+ * Handle sync request from other server groups.
+ *
+ * It adds updates of Param (slice) from other server groups directly to
+ * local Param (slice). Currently, each Param (slice) has a master group,
+ * i.e., slice2group_[sliceid], which would receive such requests from all
+ * other server groups for the Param object.
+ *
+ * @param msg request msg containing the parameter updates
+ * @return response msg that contains the fresh parameter values.
+ */
virtual Msg* HandleSyncRequest(Msg** msg);
/**
- * Generate sync message which sends local mastered Param slice to other
- * server groups
- * @param param slice to be sync with others
- * @return sync messages
+ * Handle sync response.
+ *
+ * The response msg includes the latest values of a Param object, for which
+ * this server sent the sync request to the master/maintainer group.
+ * The local Param values are replaced with the addition result of local
+ * udpates since the sync request was sent and the received Param values.
+ *
+ * @param response message
*/
- const std::vector<Msg*> GenSyncMsgs(Param* param);
+ void HandleSyncResponse(Msg** msg);
protected:
int thread_id_,grp_id_, id_;
Updater* updater_;
- std::unordered_map<int, ParamEntry*> *shard_;
- std::vector<int> slice2group_;
- std::unordered_map<int, std::shared_ptr<Blob<float>>> last_data_;
+ //!< map from slice ID to slice and deleted in the destructor
+ std::unordered_map<int, ParamEntry*> shard_;
+ std::vector<int> slice2group_, slice2server_;
+ //!< num of updates from last sync with master server group for a param/slice
+ std::vector<int> nUpdates_;
+ //!< num of sync requests that have not been responded
+ std::vector<int> nPendingSync_;
+ std::vector<Blob<float>> last_sync_;
std::unordered_map<int, std::vector<Msg*>> buffer_requests_;
};
} /* Server */
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/include/trainer/trainer.h
----------------------------------------------------------------------
diff --git a/include/trainer/trainer.h b/include/trainer/trainer.h
index 8be5269..ed50705 100644
--- a/include/trainer/trainer.h
+++ b/include/trainer/trainer.h
@@ -80,14 +80,6 @@ class Trainer{
void Run(const vector<Worker*>& workers, const vector<Server*>& servers);
/**
- * Generate msg to trigger synchronization with other server groups.
- *
- * @param server the local server index whom the message is sent to
- * @param servers all local servers
- * @return sync msg
- */
- Msg* GenSyncReminderMsg(int server, const vector<Server*>& servers);
- /**
* Display metrics to log (standard output)
*/
void DisplayMetric(Msg** msg);
@@ -143,8 +135,6 @@ class Trainer{
int procs_id_;
Router *router_;
std::unordered_map<int, ParamEntry*> worker_shard_;
- //!< map from slice ID to slice, used by servers and deleted in the destructor
- std::unordered_map<int, ParamEntry*> server_shard_;
//!< map from slice to the server that updates it
vector<int> slice2server_;
};
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/include/utils/cluster.h
----------------------------------------------------------------------
diff --git a/include/utils/cluster.h b/include/utils/cluster.h
index be0e0de..73474af 100644
--- a/include/utils/cluster.h
+++ b/include/utils/cluster.h
@@ -90,12 +90,8 @@ class Cluster {
const int worker_timeout() const { return cluster_.worker_timeout(); }
const int server_timeout() const { return cluster_.server_timeout(); }
*/
- inline bool server_update() const { return cluster_.server_update(); }
inline bool share_memory() const { return cluster_.share_memory(); }
- /**
- * bandwidth Bytes/s
- */
- inline int bandwidth() const { return cluster_.bandwidth(); }
+ inline int sync_freq() const { return cluster_.sync_freq(); }
inline int poll_time() const { return cluster_.poll_time(); }
ClusterRuntime* runtime() const { return cluster_rt_; }
@@ -106,6 +102,16 @@ class Cluster {
return procs_ids_.at(Hash(group_id, id, flag));
}
inline std::string hostip() const { return hostip_; }
+
+ /**
+ * @param pid, processs ID
+ * @param group_size, num of executors in a group
+ * @param procs_size, num of executors in a procs
+ *
+ * @return a vector with 4 integers:
+ * [group start, group end), [start executor, end executor)
+ */
+ const std::vector<int> ExecutorRng(int pid, int group_size, int procs_size);
/**
* Register this process.
*
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/src/proto/common.proto
----------------------------------------------------------------------
diff --git a/src/proto/common.proto b/src/proto/common.proto
index 671d6ad..e166299 100644
--- a/src/proto/common.proto
+++ b/src/proto/common.proto
@@ -13,7 +13,6 @@ enum MsgType {
kRUpdate = 9;
kConnect = 10;
kMetric = 11;
- kSyncReminder = 12;
};
enum EntityType {
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/src/proto/job.proto
----------------------------------------------------------------------
diff --git a/src/proto/job.proto b/src/proto/job.proto
index 7861eae..80998e1 100644
--- a/src/proto/job.proto
+++ b/src/proto/job.proto
@@ -129,15 +129,14 @@ message ClusterProto {
// servers and workers in different processes?
optional bool server_worker_separate = 20 [default = false];
+ // sync frequency between server groups
+ optional int32 sync_freq = 21 [default = 1];
+
// port number used by ZeroMQ
optional int32 start_port = 60 [default = 6723];
- // conduct updates at server side; otherwise do it at worker side
- optional bool server_update = 61 [default = true];
// share memory space between worker groups in one procs
optional bool share_memory = 62 [default = true];
- // bandwidth of ethernet, Bytes per second, default is 1 Gbps
- optional int32 bandwidth = 80 [default = 134217728];
// poll time in milliseconds
optional int32 poll_time = 81 [default = 100];
}
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/src/trainer/server.cc
----------------------------------------------------------------------
diff --git a/src/trainer/server.cc b/src/trainer/server.cc
index b4c386f..601a837 100644
--- a/src/trainer/server.cc
+++ b/src/trainer/server.cc
@@ -18,15 +18,22 @@ Server::Server(int thread_id,int group_id, int server_id):
}
void Server::Setup(const UpdaterProto& proto,
- std::unordered_map<int, ParamEntry*>* shard,
- const vector<int>& slice2group) {
+ const vector<int>& slice2group,
+ const vector<int>& slice2server) {
updater_ = Updater::Create(proto);
- shard_ = shard;
slice2group_ = slice2group;
+ slice2server_ = slice2server;
+ nUpdates_.resize(slice2group_.size(), 0);
+ nPendingSync_.resize(slice2group_.size(), 0);
+ last_sync_.resize(slice2group_.size());
}
Server::~Server() {
delete updater_;
+ // free Params (i.e., slices) in server shard
+ for (auto entry : shard_)
+ for (auto param : entry.second->shares)
+ delete param;
}
void Stop(void * running) {
@@ -35,6 +42,7 @@ void Stop(void * running) {
void Server::Run() {
LOG(ERROR) << "Server (group = " << grp_id_ <<", id = " << id_ << ") start";
+
auto dealer = new Dealer(2*thread_id_);
CHECK(dealer->Connect(kInprocRouterEndpoint));
Msg* ping = new Msg(Addr(grp_id_, id_, kServer), Addr(-1, -1, kStub));
@@ -44,13 +52,10 @@ void Server::Run() {
auto cluster = Cluster::Get();
bool running = true;
CHECK(cluster->runtime()->WatchSGroup(grp_id_, id_, Stop, &running));
-
- int nserver_grps = cluster->nserver_groups();
- vector<Param*> master_params;
- size_t syncEntry=0;
Poller poll(dealer);
// start recv loop and process requests
while (running) {
+ // must use poller here; otherwise Receive() gets stuck after workers stop.
auto *sock = poll.Wait(cluster->poll_time());
if (poll.Terminated()) {
LOG(ERROR) << "Connection broken!";
@@ -58,34 +63,18 @@ void Server::Run() {
} else if (sock == nullptr) {
continue;
}
- Msg* msg=dealer->Receive();
- if (msg==nullptr) break;
- Msg* response=nullptr;
- int type=msg->type();
+ Msg* msg = dealer->Receive();
+ if (msg == nullptr) break; // interrupted
+ Msg* response = nullptr;
+ int type = msg->type();
int slice_id = SliceID(msg->trgt_val());
if (type == kPut) {
response = HandlePut(&msg);
- if(slice2group_[slice_id] == grp_id_)
- master_params.push_back(shard_->at(slice_id)->shares.at(0));
} else {
- if (shard_->find(slice_id) == shard_->end()) {
- // delay the processing by re-queue the msg.
+ if (shard_.find(slice_id) == shard_.end()) {
+ // delay the processing by re-queue the msg. May sleep for a while?
response = msg;
- } else if (type == kSyncReminder) {
- DeleteMsg(&msg);
- if(syncEntry >= master_params.size())
- continue;
- auto param = master_params.at(syncEntry);
- // control the frequency of synchronization
- // currently sync is triggerred only when the slice is updated
- // by local worker or other workers for at least nserver_groups times.
- // TODO may optimize the trigger condition.
- if (abs(param->local_version() - param->version()) >= nserver_grps) {
- for (auto msg : GenSyncMsgs(param))
- dealer->Send(&msg);
- syncEntry = (syncEntry+1) % master_params.size();
- }
- } else {
+ } else {
switch (type) {
case kGet:
response = HandleGet(&msg);
@@ -97,6 +86,9 @@ void Server::Run() {
case kSyncRequest:
response = HandleSyncRequest(&msg);
break;
+ case kSyncResponse:
+ HandleSyncResponse(&msg);
+ break;
default:
LOG(ERROR)<<"Unknown message type "<<type;
break;
@@ -117,31 +109,10 @@ void Server::Run() {
delete dealer;
}
-const vector<Msg*> Server::GenSyncMsgs(Param* param) {
- vector<Msg*> ret;
- // TODO replace the argument (0,0) to sync a chunk instead of a slice
- auto msg = param->GenSyncMsg(0, 0);
- auto cluster = Cluster::Get();
- for (int i = 0; i < cluster->nserver_groups(); i++) {
- if (i != grp_id_) {
- Msg* tmp = msg;
- if (i < cluster->nserver_groups() - 1)
- tmp = new Msg(*msg);
- // assume only one server per group, TODO generalize it
- tmp->set_dst(Addr(i, 0, kServer));
- tmp->set_src(Addr(grp_id_, id_, kServer));
- ret.push_back(tmp);
- param->set_version(param->local_version());
- //LOG(ERROR)<<"sync slice="<<param->id()<<" to procs "<<i;
- }
- }
- return ret;
-}
-
Msg* Server::HandlePut(Msg **msg) {
int version = (*msg)->trgt_version();
int slice_id = SliceID((*msg)->trgt_val());
- if (shard_->find(slice_id) != shard_->end())
+ if (shard_.find(slice_id) != shard_.end())
LOG(FATAL) << "Param (" << slice_id << ") is put more than once";
// TODO(wangwei) replace hard coded param type 0
@@ -152,17 +123,15 @@ Msg* Server::HandlePut(Msg **msg) {
if ((*msg)->NextFrame())
(*msg)->ParseFormatFrame("i", &num_shares);
DeleteMsg(msg);
- (*shard_)[slice_id] = new ParamEntry(num_shares, param);
+ shard_[slice_id] = new ParamEntry(num_shares, param);
// must set version after HandlePutMsg which allocates the memory
param->set_version(version);
param->set_local_version(version);
param->set_id(slice_id);
- //LOG(ERROR)<<"put norm "<<param->data().asum_data()<<", "<<pid;
// allocate blob for param sync between groups.
- if (Cluster::Get()->nserver_groups() > 1 && slice2group_[slice_id] != grp_id_) {
- last_data_[slice_id] = std::make_shared<Blob<float>>();
- last_data_[slice_id]->ReshapeLike(param->data());
- last_data_[slice_id]->CopyFrom(param->data());
+ if (slice2group_[slice_id] != grp_id_) {
+ last_sync_[slice_id].ReshapeLike(param->data());
+ last_sync_[slice_id].CopyFrom(param->data());
}
LOG(INFO)<<"server (group = " << grp_id_ << ", id = " << id_ <<") put slice="
<< slice_id << " size=" << param->size();
@@ -171,7 +140,7 @@ Msg* Server::HandlePut(Msg **msg) {
Msg* Server::HandleGet(Msg **msg) {
int val = (*msg)->trgt_val();
- auto param = shard_->at(SliceID(val))->shares.at(0);
+ auto param = shard_.at(SliceID(val))->shares.at(0);
// re-queue the request if the param is not updated to the required version
if(param->version()<(*msg)->trgt_version())
return *msg;
@@ -186,15 +155,14 @@ Msg* Server::HandleGet(Msg **msg) {
const vector<Msg*> Server::HandleUpdate(Msg **msg) {
vector<Msg*> ret;
int sliceid = SliceID((*msg)->trgt_val());
- auto entry = shard_->at(sliceid);
+ auto entry = shard_.at(sliceid);
buffer_requests_[sliceid].push_back(*msg);
int num_update;
(*msg)->LastFrame();
(*msg)->ParseFormatFrame("i", &num_update);
(*msg)->FirstFrame();
entry->num_update += num_update;
- // LOG(ERROR) << "update "<<sliceid<< " from "<<(*msg)->src_second()
- // << ", " << num_update << " total " << entry->num_total;
+ // LOG(ERROR) << "update "<< sliceid << " from " << AddrGrp((*msg)->src()) << ", " << num_update << " total " << entry->num_total;
// do update until recv gradients from all shares of this param/slice
if (entry->num_update >= entry->num_total) {
CHECK_EQ(entry->num_update, entry->num_total);
@@ -211,6 +179,26 @@ const vector<Msg*> Server::HandleUpdate(Msg **msg) {
ret.push_back(response);
}
entry->num_update = 0;
+ nUpdates_[sliceid]++;
+ // sync with master group after at least sync_freq local updates
+ // the last check is to avoid sending msg to stopped servers
+ if (slice2group_[sliceid] != grp_id_
+ && nUpdates_[sliceid] >= Cluster::Get()->sync_freq()
+ && nPendingSync_[sliceid] <= Cluster::Get()->sync_freq()) {
+ auto shape = Shape1(param->size());
+ Tensor<cpu, 1> tmp(last_sync_[sliceid].mutable_cpu_data(), shape);
+ Tensor<cpu, 1> cur(param->mutable_cpu_data(), shape);
+ tmp = cur - tmp;
+ int addr = Addr(slice2group_[sliceid], slice2server_[sliceid], kServer);
+ Msg* sync = new Msg(Addr(grp_id_, id_, kServer), addr);
+ sync->set_type(kSyncRequest);
+ sync->set_trgt((*msg)->trgt_val(), param->local_version());
+ sync->AddFrame(tmp.dptr, param->size() * sizeof(float));
+ Copy(tmp, cur);
+ ret.push_back(sync);
+ nUpdates_[sliceid] = 0;
+ nPendingSync_[sliceid]++;
+ }
}
*msg = nullptr;
return ret;
@@ -219,38 +207,33 @@ const vector<Msg*> Server::HandleUpdate(Msg **msg) {
Msg* Server::HandleSyncRequest(Msg **msg) {
Msg* msgg = *msg;
int slice = SliceID(msgg->trgt_val());
- auto param = shard_->at(slice)->shares.at(0);
- Msg* response=nullptr;
- auto shape=Shape1(param->size());
+ auto param = shard_.at(slice)->shares.at(0);
+ auto shape = Shape1(param->size());
CHECK_EQ(msgg->FrameSize(), param->size()*sizeof(float));
- Tensor<cpu, 1> tmp(static_cast<float*>(msgg->FrameData()), shape);
+ Tensor<cpu, 1> inc(static_cast<float*>(msgg->FrameData()), shape);
Tensor<cpu, 1> cur(param->mutable_cpu_data(), shape);
- //LOG(ERROR)<<"Recv sync for "<<param->id();
- if (slice2group_[slice] == grp_id_) {
- // recv sync msg on slice I am mastering
- cur+=tmp;
- param->set_local_version(param->local_version()+1);
- } else { // recv sync msg on slice mastered by others
- TensorContainer<cpu, 1> diff(shape);
- Tensor<cpu, 1> prev(last_data_[param->id()]->mutable_cpu_data(), shape);
- diff=cur-prev;
- msgg->NextFrame();
- int bandwidth;
- msgg->ParseFormatFrame("i", &bandwidth);
- if (bandwidth > 0) {
- // send back my updates to the server group mastering this param
- response=new Msg(msgg->dst(), msgg->src());
- response->set_type(kSyncRequest);
- response->set_trgt(param->id(), param->version());
- response->AddFrame(diff.dptr, param->size()*sizeof(float));
- prev=diff+tmp;
- Copy(cur, prev);
- } else { // no bandwidth, aggregate my updates for next sync
- Copy(prev, tmp);
- cur=tmp+diff;
- }
- }
+ // recv sync msg on the slice I am maintaining
+ cur += inc;
+ msgg->SwapAddr();
+ msgg->set_type(kSyncResponse);
+ // copy the fresh param value into the response msg
+ Copy(inc, cur);
+ return msgg;
+}
+
+// recv sync msg on slice mastered by others
+void Server::HandleSyncResponse(Msg **msg) {
+ Msg* msgg = *msg;
+ int slice = SliceID(msgg->trgt_val());
+ auto param = shard_.at(slice)->shares.at(0);
+ auto shape=Shape1(param->size());
+ Tensor<cpu, 1> prev(last_sync_[param->id()].mutable_cpu_data(), shape);
+ Tensor<cpu, 1> cur(param->mutable_cpu_data(), shape);
+ Tensor<cpu, 1> master(static_cast<float*>(msgg->FrameData()), shape);
+ cur += master - prev; // cur = master + (cur - prev);
+ Copy(prev, cur);
DeleteMsg(msg);
- return response;
+ nPendingSync_[slice]--;
}
+
} /* singa */
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/src/trainer/trainer.cc
----------------------------------------------------------------------
diff --git a/src/trainer/trainer.cc b/src/trainer/trainer.cc
index b6dc729..b02ef3e 100644
--- a/src/trainer/trainer.cc
+++ b/src/trainer/trainer.cc
@@ -21,10 +21,6 @@ using std::make_shared;
/***********************Trainer****************************/
Trainer::~Trainer() {
- // free Params (i.e., slices) in server shard
- for (auto entry : server_shard_)
- for (auto param : entry.second->shares)
- delete param;
delete router_;
}
@@ -120,10 +116,11 @@ void Trainer::SetupWorkerServer(
// partition among server groups, each group maintains one sub-set for sync
auto slice2group = PartitionSlices(cluster->nserver_groups(), slices);
- for (auto server : servers)
- server->Setup(job_conf.updater(), &server_shard_, slice2group);
// partition within one server group, each server updates for one sub-set
slice2server_ = PartitionSlices(cluster->nservers_per_group(), slices);
+
+ for (auto server : servers)
+ server->Setup(job_conf.updater(), slice2group, slice2server_);
}
vector<Server*> Trainer::CreateServers(int nthreads, const JobProto& job) {
@@ -132,46 +129,33 @@ vector<Server*> Trainer::CreateServers(int nthreads, const JobProto& job) {
if (!cluster->has_server())
return servers;
- int pid = cluster->procs_id();
+ int server_procs = cluster->procs_id();
// if true, server procs (logical) id starts after worker procs
if (cluster->server_worker_separate())
- pid -= cluster->nworker_procs();
- int procs_size = cluster->nservers_per_procs();
- int grp_size = cluster->nservers_per_group();
- int gid = pid * procs_size / grp_size;
- int start = pid * procs_size % grp_size;
- int end = start + procs_size;
- for (int sid = start; sid < end; sid++) {
- auto server = new Server(nthreads++, gid, sid);
- servers.push_back(server);
+ server_procs -= cluster->nworker_procs();
+ const vector<int> rng = cluster->ExecutorRng(server_procs,
+ cluster->nservers_per_group(),
+ cluster->nservers_per_procs());
+ int gstart = rng[0], gend = rng[1], start = rng[2], end = rng[3];
+ for (int gid = gstart; gid < gend; gid++) {
+ for (int sid = start; sid < end; sid++) {
+ auto server = new Server(nthreads++, gid, sid);
+ servers.push_back(server);
+ }
}
return servers;
}
+
vector<Worker*> Trainer::CreateWorkers(int nthreads, const JobProto& job) {
auto cluster=Cluster::Get();
vector<Worker*> workers;
if(!cluster->has_worker())
return workers;
- int pid = cluster->procs_id();
- int grp_size = cluster->nworkers_per_group();
- int procs_size = cluster->nworkers_per_procs();
- int gstart, gend, wstart, wend;
- if (grp_size >= procs_size) {
- // all workers in this procs are from the same group
- gstart = pid * procs_size / grp_size;
- gend = gstart + 1;
- wstart = pid * procs_size % grp_size;
- wend = wstart + procs_size;
- } else {
- // there are multiple (complete) groups in this procs.
- CHECK_EQ(procs_size % grp_size, 0);
- int groups_per_procs = procs_size / grp_size;
- gstart = pid * groups_per_procs;
- gend = (pid+1) * groups_per_procs;
- wstart = 0;
- wend = grp_size;
- }
+ const vector<int> rng = cluster->ExecutorRng(cluster->procs_id(),
+ cluster->nworkers_per_group(),
+ cluster->nworkers_per_procs());
+ int gstart = rng[0], gend = rng[1], wstart = rng[2], wend = rng[3];
for (int gid = gstart; gid < gend; gid++) {
for (int wid = wstart; wid < wend; wid++) {
auto *worker = Worker::Create(job);
@@ -260,12 +244,6 @@ void Trainer::Start(bool resume, const SingaProto& singaConf, JobProto* job) {
delete worker;
}
-inline int bandwidth(int bytes, system_clock::time_point start) {
- auto now=system_clock::now();
- auto duration=duration_cast<std::chrono::milliseconds> (now - start);
- return static_cast<int>(bytes*1000.f/duration.count());
-}
-
void Trainer::Run(
const vector<Worker*>& workers,
const vector<Server*>& servers) {
@@ -274,42 +252,20 @@ void Trainer::Run(
procs_id_ = cluster->procs_id();
LOG(INFO) << "Stub in process " << procs_id_ << " starts";
- // for sync among server groups
- auto start = std::chrono::system_clock::now();
- float trans_size = 0.f; // total size of msg transferred since start time
- int sync_server_id = 0;
- int max_bandwidth = cluster->bandwidth();
- int nserver_grps = cluster->nserver_groups();
-
map<int, Dealer*> inter_dealers; // for sending msg to other procs
std::queue<Msg*> msg_queue;
- Poller poll(router_);
- bool stop=false;
- while (!stop || !msg_queue.empty()) {
+ while (true) {
+ Msg* msg = nullptr;
if (msg_queue.empty()) {
- // if the poll time is large, then the poller may not expire
- // if it is small, then many reminder messages will be sent which may
- // slow done the process of other request. TODO tune it.
- auto *sock = poll.Wait(cluster->poll_time());
- if (poll.Terminated()) {
- LOG(ERROR) << "Connection broken!";
- exit(0);
- } else if (sock == nullptr) {
- if (nserver_grps > 1 && bandwidth(trans_size, start) < max_bandwidth) {
- Msg* msg = GenSyncReminderMsg(sync_server_id, servers);
- router_->Send(&msg) ;
- sync_server_id = (sync_server_id + 1) % nservers;
- }
- continue;
- }
- Msg* msg = router_->Receive();
- msg_queue.push(msg);
+ msg = router_->Receive();
+ } else {
+ msg = msg_queue.front();
+ msg_queue.pop();
}
- Msg* msg = msg_queue.front();
- msg_queue.pop();
int type = msg->type(), dst = msg->dst(), flag = AddrType(dst);
if (flag == kStub && (AddrProc(dst) == procs_id_ || AddrGrp(dst) == -1)) {
+ // the following statements are ordered!
if (type == kConnect) {
DeleteMsg(&msg);
} else if (type == kMetric) {
@@ -320,28 +276,18 @@ void Trainer::Run(
else if (src_flag == kWorkerParam) nworkers--;
DeleteMsg(&msg);
if (nworkers == 0 && nservers == 0) break;
- } else if (nserver_grps > 0) {
- HandleLocalMsg(&msg_queue, &msg);
} else {
- DeleteMsg(&msg);
+ HandleLocalMsg(&msg_queue, &msg);
}
} else {
int dst_procs = AddrProc(dst);
if (flag != kStub)
dst_procs = cluster->ProcsIDOf(AddrGrp(dst), AddrID(dst), flag);
if (dst_procs != procs_id_) {
- if (bandwidth(trans_size, start) <= cluster->bandwidth()) {
- start = std::chrono::system_clock::now();
- trans_size = 0;
- }
- trans_size += msg->size();
-
if (inter_dealers.find(dst_procs) == inter_dealers.end())
inter_dealers[dst_procs] = CreateInterProcsDealer(dst_procs);
inter_dealers[dst_procs]->Send(&msg);
} else {
- if (type == kSyncRequest)
- msg->AddFormatFrame("i", max_bandwidth - bandwidth(trans_size, start));
router_->Send(&msg);
}
}
@@ -351,14 +297,6 @@ void Trainer::Run(
delete entry.second;
}
-Msg* Trainer::GenSyncReminderMsg(int server, const vector<Server*>& servers ) {
- Msg* msg = new Msg();
- msg->set_src(Addr(-1,-1, kStub));
- msg->set_dst(Addr(servers[server]->grp_id(), servers[server]->id(), kServer));
- msg->set_type(kSyncReminder);
- return msg;
-}
-
void Trainer::DisplayMetric(Msg** msg) {
Msg* msgg = *msg;
// only display metrics from the first group
@@ -436,16 +374,16 @@ void Trainer::GenMsgs(int type, int version, ParamEntry* entry,
for (int idx = 0 ; idx < param->num_slices(); idx++) {
int slice_id =param->slice_start() + idx;
int server = slice2server_[slice_id];
- int procs = Cluster::Get()->ProcsIDOf(dst_grp, server, kServer);
+ int dst_procs = Cluster::Get()->ProcsIDOf(dst_grp, server, kServer);
Msg* new_msg = nullptr;
if (type == kPut) {
CHECK_GT(entry->num_total, 0);
- new_msg = param->GenPutMsg(procs != procs_id_, idx);
+ new_msg = param->GenPutMsg(dst_procs != procs_id_, idx);
new_msg->AddFormatFrame("i", entry->num_total);
} else if (type == kGet) {
- new_msg = param->GenGetMsg(procs != procs_id_, idx);
+ new_msg = param->GenGetMsg(dst_procs != procs_id_, idx);
} else if (type == kUpdate) {
- new_msg = param->GenUpdateMsg(procs != procs_id_, idx);
+ new_msg = param->GenUpdateMsg(dst_procs != procs_id_, idx);
new_msg->AddFormatFrame("i", entry->num_local);
} else {
LOG(FATAL) << "Wrong type";
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/ed9e3736/src/utils/cluster.cc
----------------------------------------------------------------------
diff --git a/src/utils/cluster.cc b/src/utils/cluster.cc
index 9664064..a1716a1 100644
--- a/src/utils/cluster.cc
+++ b/src/utils/cluster.cc
@@ -6,6 +6,7 @@
#include <fstream>
namespace singa {
+using std::vector;
Cluster* Cluster::Setup(int job, const SingaProto& singaConf,
const ClusterProto& clusterConf) {
@@ -71,6 +72,26 @@ void Cluster::SetupFolders(const ClusterProto &cluster) {
mkdir(checkpoint_folder().c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
}
+const vector<int> Cluster::ExecutorRng(int pid, int grp_size, int procs_size) {
+ int gstart, gend, start, end;
+ if (grp_size >= procs_size) {
+ // all workers in this procs are from the same group
+ gstart = pid * procs_size / grp_size;
+ gend = gstart + 1;
+ start = pid * procs_size % grp_size;
+ end = start + procs_size;
+ } else {
+ // there are multiple (complete) groups in this procs.
+ CHECK_EQ(procs_size % grp_size, 0);
+ int groups_per_procs = procs_size / grp_size;
+ gstart = pid * groups_per_procs;
+ gend = (pid+1) * groups_per_procs;
+ start = 0;
+ end = grp_size;
+ }
+ return vector<int>{gstart, gend, start, end};
+}
+
int Cluster::Hash(int gid, int id, int flag) {
int ret = -1;
if (flag == kServer) {
[06/21] incubator-singa git commit: SINGA-51 Improve the convolution
and pooling operations
Posted by wa...@apache.org.
SINGA-51 Improve the convolution and pooling operations
merge with pull request #64
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/e7691422
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/e7691422
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/e7691422
Branch: refs/heads/tutorial
Commit: e7691422c1c09b3ebea5ff399be1724ec72a5c9d
Parents: 6d59eec 077d380
Author: wang sheng <wa...@gmail.com>
Authored: Mon Sep 14 12:11:56 2015 +0800
Committer: wang sheng <wa...@gmail.com>
Committed: Mon Sep 14 12:11:56 2015 +0800
----------------------------------------------------------------------
Makefile.am | 69 +++++++++++++++++++++-----
configure.ac | 25 ++++++++++
src/test/test_common.cc | 113 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 196 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
[10/21] incubator-singa git commit: SINGA-68 Preparation for Release
-- License etc.
Posted by wa...@apache.org.
SINGA-68 Preparation for Release -- License etc.
Add RELEASE_NOTES file to describe the features, bugs, etc.
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/be035dff
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/be035dff
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/be035dff
Branch: refs/heads/tutorial
Commit: be035dfffaf1948eea0dfb9b79fc3d183fdc5bd9
Parents: 7486647
Author: Wei Wang <wa...@comp.nus.edu.sg>
Authored: Fri Sep 11 17:27:07 2015 +0800
Committer: wang sheng <wa...@gmail.com>
Committed: Wed Sep 16 11:09:25 2015 +0800
----------------------------------------------------------------------
RELEASE_NOTES | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 93 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/be035dff/RELEASE_NOTES
----------------------------------------------------------------------
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
new file mode 100644
index 0000000..2a114b5
--- /dev/null
+++ b/RELEASE_NOTES
@@ -0,0 +1,93 @@
+Release Notes - SINGA - Version singa-incubating-0.1-rc0
+-----------------------------------------
+This release includes these features:
+
+ * Job management
+ * [SINGA-3] - Use Zookeeper to check stopping (finish) time of the system
+ * [SINGA-16] - Runtime Process id Management
+ * [SINGA-25] - Setup glog output path
+ * [SINGA-26] - Run distributed training in a single command
+ * [SINGA-30] - Enhance easy-to-use feature and support concurrent jobs
+ * [SINGA-33] - Automatically launch a number of processes in the cluster
+ * [SINGA-34] - Support external zookeeper service
+ * [SINGA-38] - Support concurrent jobs
+ * [SINGA-39] - Avoid ssh in scripts for single node environment
+ * [SINGA-43] - Remove Job-related output from workspace
+ * [SINGA-56] - No automatic launching of zookeeper service
+
+ * Installation with GNU Auto tool
+ * [SINGA-4] - Refine thirdparty-dependency installation
+ * [SINGA-13] - Separate intermediate files of compilation from source files
+ * [SINGA-17] - Add root permission within thirdparty/install.
+ * [SINGA-27] - Generate python modules for proto objects
+ * [SINGA-53] - Add lmdb compiling options
+ * [SINGA-62] - Remove building scrips and auxiliary files
+ * [SINGA-67] - Add singatest into build targets
+
+ * Distributed training
+ * [SINGA-7] - Implement shared memory Hogwild algorithm
+ * [SINGA-8] - Implement distributed Hogwild
+ * [SINGA-19] - Slice large Param objects for load-balance
+ * [SINGA-29] - Update NeuralNet class to enable layer partition type customization
+ * [SINGA-24] - Implement Downpour training framework
+ * [SINGA-32] - Implement AllReduce training framework
+ * [SINGA-57] - Improve Distributed Hogwild
+
+ * Training algorithms for different model categories
+ * [SINGA-9] - Add Support for Restricted Boltzman Machine (RBM) model
+ * [SINGA-10] - Add Support for Recurrent Neural Networks (RNN)
+
+ * Checkpoint and restore
+ * [SINGA-12] - Support Checkpoint and Restore
+
+ * Unit test
+ * [SINGA-64] - Add the test module for utils/common
+
+ * Programming model
+ * [SINGA-36] - Refactor job configuration, driver program and scripts
+ * [SINGA-37] - Enable users to set parameter sharing in model configuration
+ * [SINGA-54] - Refactor job configuration to move fields in ModelProto out
+ * [SINGA-55] - Refactor main.cc and singa.h
+ * [SINGA-61] - Support user defined classes
+ * [SINGA-65] - Add an example of writing user-defined layers
+
+ * Other features
+ * [SINGA-6] - Implement thread-safe singleton
+ * [SINGA-18] - Update API for displaying performance metric
+
+Some bugs are fixed during the development of this release
+ * [SINGA-2] Check failed: zsock_connect
+ * [SINGA-5] Server early terminate when zookeeper singa folder is not initially empty
+ * [SINGA-15] Fixg a bug from ConnectStub function which gets stuck for connecting layer_dealer_
+ * [SINGA-22] Cannot find openblas library when it is installed in default path
+ * [SINGA-23] Libtool version mismatch error.
+ * [SINGA-28] Fix a bug from topology sort of Graph
+ * [SINGA-42] Issue when loading checkpoints
+ * [SINGA-44] A bug when reseting metric values
+ * [SINGA-46] Fix a bug in updater.cc to scale the gradients
+ * [SINGA-47] Fix a bug in data layers that leads to out-of-memory when group size is too large
+ * [SINGA-48] Fix a bug in trainer.cc that assigns the same NeuralNet instance to workers from diff groups
+ * [SINGA-49] Fix a bug in HandlePutMsg func that sets param fields to invalid values
+ * [SINGA-66] Fix bugs in Worker::RunOneBatch function and ClusterProto
+
+Some tasks should be done in the next release
+ * [SINGA-11] Start SINGA using Mesos
+ * [SINGA-31] Extend Blob to support xpu (cpu or gpu)
+ * [SINGA-35] Add random number generators
+ * [SINGA-40] Support sparse Param update
+ * [SINGA-41] Support single node single GPU training
+
+The current code depends on the following external libraries:
+ * glog (New BSD)
+ * google-protobuf (New BSD)
+ * openblas (New BSD)
+ * zeromq (LGPLv3 + static link exception)
+ * czmq (Mozilla Public License Version 2.0)
+ * zookeeper (Apache 2.0)
+ * lmdb (OpenLDAP)
+
+Some source files/folders include third-party code:
+ * include/gtest (New BSD)
+ * include/mshadow (Apache 2.0)
+ * include/utils/tinydir.h (BSD 2-clause)
+ * caffe <include/utils/blob.h, src/utils/blob.cc, include/utils/common.h, src/utils/common.cc> (BSD 2-clause)