You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@singa.apache.org by zh...@apache.org on 2017/01/24 09:05:45 UTC
[2/6] incubator-singa git commit: preparing for v1.1-rc1: remove
outdated files; fix links
preparing for v1.1-rc1: remove outdated files; fix links
Project: http://git-wip-us.apache.org/repos/asf/incubator-singa/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-singa/commit/b2e21c62
Tree: http://git-wip-us.apache.org/repos/asf/incubator-singa/tree/b2e21c62
Diff: http://git-wip-us.apache.org/repos/asf/incubator-singa/diff/b2e21c62
Branch: refs/heads/master
Commit: b2e21c626fa6983a6f9166b7ba426a5e004b2073
Parents: 5a6c303
Author: wang wei <wa...@comp.nus.edu.sg>
Authored: Mon Jan 23 23:25:16 2017 +0800
Committer: Wei Wang <wa...@comp.nus.edu.sg>
Committed: Tue Jan 24 16:55:06 2017 +0800
----------------------------------------------------------------------
CONTRIBUTING.md | 2 +-
NOTICE | 2 +-
RELEASE_NOTES | 32 +-
bin/singa-cleanup.sh | 32 --
bin/singa-console.sh | 85 ----
bin/singa-env.sh | 68 ---
bin/singa-run.sh | 107 ----
bin/singa-stop.sh | 47 --
bin/zk-service.sh | 72 ---
doc/en/develop/how-contribute.md | 10 +-
doc/en/develop/schedule.rst | 81 +--
rat-excludes | 3 +-
thirdparty/install.sh | 714 ---------------------------
tool/graph.py | 40 --
tool/mesos/Makefile | 25 -
tool/mesos/README.md | 86 ----
tool/mesos/docker/README.md | 189 -------
tool/mesos/docker/mesos/.bashrc | 32 --
tool/mesos/docker/mesos/Dockerfile | 40 --
tool/mesos/docker/mesos/core-site.xml | 24 -
tool/mesos/docker/mesos/hdfs-site.xml | 34 --
tool/mesos/docker/mesos/install.sh | 35 --
tool/mesos/docker/mesos/mapred-site.xml | 22 -
tool/mesos/docker/mesos/yarn-site.xml | 32 --
tool/mesos/docker/singa/.bashrc | 30 --
tool/mesos/docker/singa/Dockerfile | 34 --
tool/mesos/docker/singa/Dockerfile_gpu | 80 ---
tool/mesos/docker/singa/ssh.conf | 5 -
tool/mesos/scheduler.conf | 3 -
tool/mesos/scheduler.proto | 33 --
tool/mesos/singa_scheduler.cc | 426 ----------------
tool/node.sh | 88 ----
32 files changed, 66 insertions(+), 2447 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/CONTRIBUTING.md
----------------------------------------------------------------------
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 442f477..6fd833c 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -4,6 +4,6 @@
[How to contribute code](./doc/en/develop/contribute-code.md)
-[How to contribute documentation](./doc/en/develop/contribute-docs.md)
+[How to contribute documentation](./doc/README.md)
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/NOTICE
----------------------------------------------------------------------
diff --git a/NOTICE b/NOTICE
index 092ec36..6fc258b 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,5 +1,5 @@
Apache SINGA
-Copyright 2016 The Apache Software Foundation
+Copyright 2017 The Apache Software Foundation
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/RELEASE_NOTES
----------------------------------------------------------------------
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index d9970e3..9cafffe 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -8,36 +8,36 @@ This release includes following features:
* [SINGA-296] Add sign and to_host function for pysinga tensor module
* Model components
+ * [SINGA-254] Implement Adam for V1
* [SINGA-264] Extend the FeedForwardNet to accept multiple inputs
- * [SINGA-278] Convert trained caffe parameters to singa
- * [SINGA-287] Add memory size check for cudnn convolution
- * [SINGA-275] Add Cross Entropy Loss for multiple labels
* [SINGA-267] Add spatial mode in batch normalization layer
- * [SINGA-254] Implement Adam for V1
* [SINGA-271] Add Concat and Slice layers
+ * [SINGA-275] Add Cross Entropy Loss for multiple labels
+ * [SINGA-278] Convert trained caffe parameters to singa
+ * [SINGA-287] Add memory size check for cudnn convolution
* Utility functions and CI
- * [SINGA-261] Add version ID into the checkpoint files
+ * [SINGA-242] Compile all source files into a single library.
+ * [SINGA-244] Separating swig interface and python binding files
+ * [SINGA-246] Imgtool for image augmentation
+ * [SINGA-247] Add windows support for singa
* [SINGA-251] Implement image loader for pysinga
- * [SINGA-284] Add python unittest into Jenkins and link static libs into whl file
- * [SINGA-280] Jenkins CI support
- * [SINGA-288] Publish wheel of PySINGA generated by Jenkins to public servers
- * [SINGA-266] Add Rafiki python toolkits
+ * [SINGA-252] Use the snapshot methods to dump and load models for pysinga
* [SINGA-255] Compile mandatory depedent libaries together with SINGA code
- * [SINGA-247] Add windows support for singa
* [SINGA-259] Add maven pom file for building java classes
- * [SINGA-246] Imgtool for image augmentation
- * [SINGA-252] Use the snapshot methods to dump and load models for pysinga
- * [SINGA-242] Compile all source files into a single library.
- * [SINGA-251] Implement image loader for pysinga
- * [SINGA-244] Separating swig interface and python binding files
+ * [SINGA-261] Add version ID into the checkpoint files
+ * [SINGA-266] Add Rafiki python toolkits
* [SINGA-273] Improve license and contributions
+ * [SINGA-284] Add python unittest into Jenkins and link static libs into whl file
+ * [SINGA-280] Jenkins CI support
+ * [SINGA-288] Publish wheel of PySINGA generated by Jenkins to public servers
* Documentation and usability
- * [SINGA-268] Add IPython notebooks to the documentation
* [SINGA-263] Create Amazon Machine Image
+ * [SINGA-268] Add IPython notebooks to the documentation
* [SINGA-276] Create docker images
* [SINGA-289] Update SINGA website automatically using Jenkins
+ * [SINGA-295] Add an example of image classification using GoogleNet
* Bugs fixed
* [SINGA-245] float as the first operand can not multiply with a tensor object
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/bin/singa-cleanup.sh
----------------------------------------------------------------------
diff --git a/bin/singa-cleanup.sh b/bin/singa-cleanup.sh
deleted file mode 100755
index 88b947d..0000000
--- a/bin/singa-cleanup.sh
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/usr/bin/env bash
-#
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#
-# kill all singa jobs and clean up zookeeper
-#
-
-# get environment variables
-. `dirname "${BASH_SOURCE-$0}"`/singa-env.sh
-cd $SINGA_HOME
-
-# kill singa jobs
-$SINGA_BIN/singa-stop.sh || exit 1
-
-# cleanup whole zookeeper
-./singatool cleanup || exit 1
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/bin/singa-console.sh
----------------------------------------------------------------------
diff --git a/bin/singa-console.sh b/bin/singa-console.sh
deleted file mode 100755
index 06ee945..0000000
--- a/bin/singa-console.sh
+++ /dev/null
@@ -1,85 +0,0 @@
-#!/usr/bin/env bash
-#
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#
-# console to list/view/kill singa jobs
-#
-
-usage="Usage: singa-console.sh <command> <args>\n
- list : list running singa jobs\n
- view <job id> : view procs of a singa job\n
- kill <job id> : kill a singa job"
-
-if [ $# == 0 ]; then
- echo -e $usage
- exit 1
-fi
-
-# get environment variables
-. `dirname "${BASH_SOURCE-$0}"`/singa-env.sh
-cd $SINGA_HOME
-
-case $1 in
- list)
- if [ $# != 1 ]; then
- echo -e $usage
- exit 1
- fi
- ./singatool list || exit 1
- ;;
-
- view)
- if [ $# != 2 ]; then
- echo -e $usage
- exit 1
- fi
- ./singatool view $2 || exit 1
- ;;
-
- kill)
- if [ $# != 2 ]; then
- echo -e $usage
- exit 1
- fi
- hosts=`./singatool view "$2"`
- [ $? == 0 ] || exit 1
- ssh_options="-oStrictHostKeyChecking=no \
- -oUserKnownHostsFile=/dev/null \
- -oLogLevel=quiet"
- if [ `head -1 "$SINGA_CONF"/hostfile` == localhost ]; then
- local_procs=1
- fi
- for i in ${hosts[@]}; do
- proc=(`echo $i | tr '|' ' '`)
- singa_kill="kill -9 "${proc[1]}
- if [ -z $local_procs ]; then
- echo Kill singa @ $i ...
- ssh $ssh_options ${proc[0]} $singa_kill
- else
- echo Kill singa @ ${proc[1]} ...
- $singa_kill
- fi
- done
- ./singatool remove $2 || exit 1
- ;;
-
- *)
- echo -e $usage
- exit 1
-esac
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/bin/singa-env.sh
----------------------------------------------------------------------
diff --git a/bin/singa-env.sh b/bin/singa-env.sh
deleted file mode 100755
index 8adeade..0000000
--- a/bin/singa-env.sh
+++ /dev/null
@@ -1,68 +0,0 @@
-#!/usr/bin/env bash
-#
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#
-# set singa environment variables, includes:
-# * SINGA_HOME
-# * SINGA_BIN
-# * SINGA_CONF
-# * SINGA_LOG
-# * ZK_HOME
-# * SINGA_MANAGES_ZK
-#
-
-# exit if varaiables already set
-[ -z $SINGA_ENV_DONE ] || exit 0
-
-# set SINGA_BIN
-if [ -z $SINGA_BIN ]; then
- SINGA_BIN=`dirname "${BASH_SOURCE-$0}"`
- SINGA_BIN=`cd "$SINGA_BIN">/dev/null; pwd`
-fi
-
-# set SINGA_HOME
-if [ -z $SINGA_HOME ]; then
- SINGA_HOME=`cd "$SINGA_BIN/..">/dev/null; pwd`
-fi
-
-# set SINGA_CONF
-if [ -z $SINGA_CONF ]; then
- SINGA_CONF=$SINGA_HOME/conf
-fi
-
-# set SINGA_LOG
-if [ -z $SINGA_LOG ]; then
- # add -confdir arg, so no need to run under SINGA_HOME
- SINGA_LOG=`"$SINGA_HOME"/singatool getlogdir -confdir "$SINGA_CONF"`
- [ $? == 0 ] || exit 1
-fi
-
-# set ZK_HOME
-if [ -z $ZK_HOME ]; then
- ZK_HOME=$SINGA_HOME/thirdparty/zookeeper-3.4.6
- SINGA_MANAGES_ZK=true
-fi
-
-# set SINGA_MANAGES_ZK
-if [ -z $SINGA_MANAGES_ZK ]; then
- SINGA_MANAGES_ZK=false
-fi
-
-# mark that we have done all
-SINGA_ENV_DONE=1
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/bin/singa-run.sh
----------------------------------------------------------------------
diff --git a/bin/singa-run.sh b/bin/singa-run.sh
deleted file mode 100755
index 763130d..0000000
--- a/bin/singa-run.sh
+++ /dev/null
@@ -1,107 +0,0 @@
-#!/usr/bin/env bash
-#
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#
-# run a singa job
-#
-
-usage="Usage: singa-run.sh [ arguments ]\n
- -exec <binary or python script> : if want to use own singa driver\n
- -conf <job config file> : need cluster conf if train in a cluster
- -resume : if want to recover a job"
-
-# parse arguments
-# extract and remove '-exec' and '-conf'
-# other arguments remain untouched
-exe=./singa
-while [ $# != 0 ]; do
- if [ $1 == "-exec" ]; then
- shift
- exe=$1
- elif [ $1 == "-conf" ]; then
- shift
- conf=$1
- else
- args="$args $1"
- fi
- shift
-done
-
-# get environment variables
-. `dirname "${BASH_SOURCE-$0}"`/singa-env.sh
-
-# change conf to an absolute path
-if [ ! -z $conf ]; then
- conf_dir=`dirname "$conf"`
- conf_dir=`cd "$conf_dir">/dev/null; pwd`
- conf_base=`basename "$conf"`
- job_conf=$conf_dir/$conf_base
- if [ ! -f $job_conf ]; then
- echo $job_conf not exists
- exit 1
- fi
-fi
-
-# go to singa home to execute binary
-cd $SINGA_HOME
-
-# generate unique job id
-job_id=`./singatool create`
-[ $? == 0 ] || exit 1
-echo Unique JOB_ID is $job_id
-
-# generate job info dir
-# format: job-JOB_ID-YYYYMMDD-HHMMSS
-log_dir=$SINGA_LOG/job-info/job-$job_id-$(date '+%Y%m%d-%H%M%S')
-mkdir -p $log_dir
-echo Record job information to $log_dir
-
-# generate host file
-host_file=$log_dir/job.hosts
-./singatool genhost $job_conf 1>$host_file || exit 1
-
-# set command to run singa
-singa_run="$exe $args \
- -singa_conf $SINGA_HOME/conf/singa.conf \
- -singa_job $job_id"
-# add -conf if exists
-if [ ! -z $job_conf ]; then
- singa_run="$singa_run -conf $job_conf"
-fi
-singa_sshrun="cd $SINGA_HOME; source $SINGA_HOME/conf/profile; $singa_run"
-
-# ssh and start singa processes
-ssh_options="-oStrictHostKeyChecking=no \
--oUserKnownHostsFile=/dev/null \
--oLogLevel=quiet"
-hosts=`cat $host_file | cut -d ' ' -f 1`
-for i in ${hosts[@]} ; do
- if [ $i = localhost ] ; then
- echo Executing : $singa_run
- $singa_run &
- else
- echo Executing @ $i : $singa_sshrun
- ssh $ssh_options $i $singa_sshrun " -host " $i &
- fi
-done
-
-# generate pid list for this job
-sleep 2
-./singatool view $job_id 1>$log_dir/job.pids || exit 1
-wait
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/bin/singa-stop.sh
----------------------------------------------------------------------
diff --git a/bin/singa-stop.sh b/bin/singa-stop.sh
deleted file mode 100755
index 446eaf8..0000000
--- a/bin/singa-stop.sh
+++ /dev/null
@@ -1,47 +0,0 @@
-#!/usr/bin/env bash
-#
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#
-# kill all singa jobs
-#
-
-# get environment variables
-. `dirname "${BASH_SOURCE-$0}"`/singa-env.sh
-cd $SINGA_HOME
-
-# kill singa processes
-host_file=$SINGA_CONF/hostfile
-ssh_options="-oStrictHostKeyChecking=no \
- -oUserKnownHostsFile=/dev/null \
- -oLogLevel=quiet"
-hosts=`cat $host_file | cut -d ' ' -f 1`
-singa_kill="killall -q -s SIGKILL -r singa"
-for i in ${hosts[@]}; do
- echo Kill singa @ $i ...
- if [ $i == localhost ]; then
- $singa_kill
- else
- ssh $ssh_options $i $singa_kill
- fi
-done
-# wait for killall command
-sleep 2
-
-# remove job paths in zookeeper
-./singatool removeall || exit 1
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/bin/zk-service.sh
----------------------------------------------------------------------
diff --git a/bin/zk-service.sh b/bin/zk-service.sh
deleted file mode 100755
index e7bb67b..0000000
--- a/bin/zk-service.sh
+++ /dev/null
@@ -1,72 +0,0 @@
-#!/usr/bin/env bash
-#
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#
-# manage ZooKeeper service
-#
-
-usage="Usage: zk-service.sh [start|stop]"
-
-if [ $# != 1 ]; then
- echo $usage
- exit 1
-fi
-
-# get environment variables
-. `dirname "${BASH_SOURCE-$0}"`/singa-env.sh
-
-# check if singa manages zookeeper service
-if [ $SINGA_MANAGES_ZK != true ]; then
- echo "Singa does not manage a valid zookeeper service (SINGA_MANAGES_ZK != true)"
- exit 1
-fi
-
-# check zookeeper installation
-if [ ! -d $ZK_HOME ]; then
- echo "zookeeper not found at $ZK_HOME"
- echo "if you do not have zookeeper service, please install:"
- echo " $SINGA_HOME/thirdparty/install.sh zookeeper"
- echo "otherwise, please set ZK_HOME correctly"
- exit 1
-fi
-
-# get command
-case $1 in
- start)
- # start zk service
- # check zoo.cfg
- if [ ! -f $ZK_HOME/conf/zoo.cfg ]; then
- echo "zoo.cfg not found, create from sample.cfg"
- cp $ZK_HOME/conf/zoo_sample.cfg $ZK_HOME/conf/zoo.cfg
- fi
- # cd to SINGA_HOME as zookeeper.out will be here
- cd $SINGA_HOME
- $ZK_HOME/bin/zkServer.sh start
- ;;
-
- stop)
- # stop zk service
- $ZK_HOME/bin/zkServer.sh stop
- ;;
-
- *)
- echo $usage
- exit 1
-esac
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/doc/en/develop/how-contribute.md
----------------------------------------------------------------------
diff --git a/doc/en/develop/how-contribute.md b/doc/en/develop/how-contribute.md
index 8e8e5c4..bfc4cec 100644
--- a/doc/en/develop/how-contribute.md
+++ b/doc/en/develop/how-contribute.md
@@ -2,8 +2,8 @@
As with any open source project, there are several ways you can help:
-* Join the [mailing list](../community/mail-lists.html) and answer other user's questions.
-* [Build Singa](../docs/installation.html) by yourself.
-* Report bugs, feature requests and other issues in the [issue tracking](../community/issue-tracking.html) application.
-* Check SINGA's [development schedule](schedule.html) and [contribute code](contribute-code.html) by providing patches.
-* [Help with the documentation](contribute-docs.html) by updating webpages that are lacking or unclear.
+* Join the [mailing list](http://singa.apache.org/en/community/mail-lists.html) and answer other user's questions.
+* [Build Singa](http://singa.apache.org/en/docs/installation.html) by yourself.
+* Report bugs, feature requests and other issues in the [issue tracking](http://singa.apache.org/en/community/issue-tracking.html) application.
+* Check SINGA's [development schedule](http://singa.apache.org/en/develop/schedule.html) and [contribute code](http://singa.apache.org/en/develop/contribute-code.html) by providing patches.
+* [Help with the documentation](http://singa.apache.org/en/develop/contribute-docs.html) by updating webpages that are lacking or unclear.
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/doc/en/develop/schedule.rst
----------------------------------------------------------------------
diff --git a/doc/en/develop/schedule.rst b/doc/en/develop/schedule.rst
index c097407..0eb4f90 100644
--- a/doc/en/develop/schedule.rst
+++ b/doc/en/develop/schedule.rst
@@ -22,42 +22,45 @@ Development Schedule
.. csv-table::
:header: "Release","Module","Feature"
- "0.1 Sep 2015 ","Neural Network ","Feed forward neural network, including CNN, MLP "
- " "," ","RBM-like model, including RBM "
- " "," ","Recurrent neural network, including standard RNN "
- " ","Architecture ","One worker group on single node (with data partition) "
- " "," ","Multi worker groups on single node using `Hogwild <http://www.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf>`_ "
- " "," ","Distributed Hogwild"
- " "," ","Multi groups across nodes, like `Downpour <http://papers.nips.cc/paper/4687-large-scale-distributed-deep-networks>`_"
- " "," ","All-Reduce training architecture like `DeepImage <http://arxiv.org/abs/1501.02876>`_ "
- " "," ","Load-balance among servers "
- " ","Failure recovery ","Checkpoint and restore "
- " ","Tools ","Installation with GNU auto Tools "
- "0.2 Jan 2016 ","Neural Network ","Feed forward neural network, including AlexNet, cuDNN layers,Tools "
- " "," ","Recurrent neural network, including GRULayer and BPTT "
- " "," ","Model partition and hybrid partition "
- " ","Tools ","Integration with Mesos for resource management "
- " "," ","Prepare Docker images for deployment"
- " "," ","Visualization of neural net and debug information "
- " ","Binding ","Python binding for major components "
- " ","GPU ","Single node with multiple GPUs "
- "0.3 April 2016 ","GPU ","Multiple nodes, each with multiple GPUs"
- " "," ","Heterogeneous training using both GPU and CPU `CcT <http://arxiv.org/abs/1504.04343>`_"
- " "," ","Support cuDNN v4 "
- " ","Installation ","Remove dependency on ZeroMQ, CZMQ, Zookeeper for single node training"
- " ","Updater ","Add new SGD updaters including Adam, AdamMax and AdaDelta"
- " ","Binding ","Enhance Python binding for training"
- "1.0 Sep 2016 ","Programming abstraction ","Tensor with linear algebra, neural net and random operations "
- " "," ","Updater for distributed parameter updating "
- " ","Hardware ","Use Cuda and Cudnn for Nvidia GPU"
- " "," ","Use OpenCL for AMD GPU or other devices"
- " ","Cross-platform ","To extend from Linux to MacOS"
- " "," ","Large image models, e.g., `VGG <https://arxiv.org/pdf/1409.1556.pdf>`_ and `Residual Net <http://arxiv.org/abs/1512.03385>`_"
- "1.1 Dec 2016 ","Model Zoo ","Health-care models and popular image models"
- " ","Caffe converter ","Use SINGA to train models configured in caffe proto files"
- " ","Memory optimization ","Replace CNMEM with new memory pool to reduce memory footprint"
- " ","Distributed training ","Migrate distributed training frameworks from V0.3"
- " ","Compilation and installation ","Windows suppport"
- " "," ","Simplify the installation by compiling protobuf and openblas together with SINGA"
- " "," ","Build python wheel automatically using Jenkins"
- " "," ","Deploy SINGA programs on Android phones for prediction tasks"
+ "v0.1 Sep 2015 ","Neural Network ","Feed forward neural network, including CNN, MLP "
+ " "," ","RBM-like model, including RBM "
+ " "," ","Recurrent neural network, including standard RNN "
+ " ","Architecture ","One worker group on single node (with data partition) "
+ " "," ","Multi worker groups on single node using `Hogwild <http://www.eecs.berkeley.edu/~brecht/papers/hogwildTR.pdf>`_ "
+ " "," ","Distributed Hogwild"
+ " "," ","Multi groups across nodes, like `Downpour <http://papers.nips.cc/paper/4687-large-scale-distributed-deep-networks>`_"
+ " "," ","All-Reduce training architecture like `DeepImage <http://arxiv.org/abs/1501.02876>`_ "
+ " "," ","Load-balance among servers "
+ " ","Failure recovery ","Checkpoint and restore "
+ " ","Tools ","Installation with GNU auto Tools "
+ "v0.2 Jan 2016 ","Neural Network ","Feed forward neural network, including AlexNet, cuDNN layers,Tools "
+ " "," ","Recurrent neural network, including GRULayer and BPTT "
+ " "," ","Model partition and hybrid partition "
+ " ","Tools ","Integration with Mesos for resource management "
+ " "," ","Prepare Docker images for deployment"
+ " "," ","Visualization of neural net and debug information "
+ " ","Binding ","Python binding for major components "
+ " ","GPU ","Single node with multiple GPUs "
+ "v0.3 April 2016 ","GPU ","Multiple nodes, each with multiple GPUs"
+ " "," ","Heterogeneous training using both GPU and CPU `CcT <http://arxiv.org/abs/1504.04343>`_"
+ " "," ","Support cuDNN v4 "
+ " ","Installation ","Remove dependency on ZeroMQ, CZMQ, Zookeeper for single node training"
+ " ","Updater ","Add new SGD updaters including Adam, AdamMax and AdaDelta"
+ " ","Binding ","Enhance Python binding for training"
+ "v1.0 Sep 2016 ","Programming abstraction ","Tensor with linear algebra, neural net and random operations "
+ " "," ","Updater for distributed parameter updating "
+ " ","Hardware ","Use Cuda and Cudnn for Nvidia GPU"
+ " "," ","Use OpenCL for AMD GPU or other devices"
+ " ","Cross-platform ","To extend from Linux to MacOS"
+ " "," ","Large image models, e.g., `VGG <https://arxiv.org/pdf/1409.1556.pdf>`_ and `Residual Net <http://arxiv.org/abs/1512.03385>`_"
+ "v1.1 Jan 2017 ","Model Zoo ","GoogleNet; Health-care models"
+ " ","Caffe converter ","Use SINGA to train models configured in caffe proto files"
+ " ","Model components ","Add concat and slice layers; accept multiple inputs to the net"
+ " ","Compilation and installation ","Windows suppport"
+ " "," ","Simplify the installation by compiling protobuf and openblas together with SINGA"
+ " "," ","Build python wheel automatically using Jenkins"
+ " "," ","Install SINGA from Debian packages"
+ "v1.2 April 2017 ","Numpy API ","Implement functions for the tensor module of PySINGA following numpy API"
+ " ","Distributed training ","Migrate distributed training frameworks from V0.3"
+ "v1.3 July 2017 ","Memory optimization ","Replace CNMEM with new memory pool to reduce memory footprint"
+ " ","Execution optimization ","Runtime optimization of execution scheduling"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/rat-excludes
----------------------------------------------------------------------
diff --git a/rat-excludes b/rat-excludes
index f7f79d2..9dd92e6 100644
--- a/rat-excludes
+++ b/rat-excludes
@@ -6,9 +6,8 @@ configure
doc/*
config/*
\.dirstamp
-autom4te.cache/*
config.*
-libtool
stamp-h1
.*\.conf
.*\.md
+control
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/thirdparty/install.sh
----------------------------------------------------------------------
diff --git a/thirdparty/install.sh b/thirdparty/install.sh
deleted file mode 100755
index bc3c159..0000000
--- a/thirdparty/install.sh
+++ /dev/null
@@ -1,714 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-#!/usr/bin/env bash
-
-function install_cmake()
-{
- if [ ! -e "cmake-3.2.1.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/cmake-3.2.1.tar.gz;
- fi
-
- rm -rf cmake-3.2.1;
- tar zxvf cmake-3.2.1.tar.gz && cd cmake-3.2.1;
-
- if [ $# == 1 ]
- then
- echo "install cmake in $1";
- ./configure --prefix=$1;
- make && make install;
- elif [ $# == 0 ]
- then
- echo "install cmake in default path";
- ./configure;
- make && sudo make install;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
-
- PATH=$1/bin:$PATH;
- echo $PATH;
- export PATH=$1/bin:$PATH;
- cd ..;
- return 0;
-}
-
-function install_czmq()
-{
- if [ ! -e "czmq-3.0.0-rc1.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/czmq-3.0.0-rc1.tar.gz;
- fi
- rm -rf czmq-3.0.0;
- tar zxvf czmq-3.0.0-rc1.tar.gz && cd czmq-3.0.0;
-
- if [ $# == 2 ]
- then
- if [ $1 == "null" ]
- then
- echo "install czmq in default path. libzmq path is $2";
- ./configure --with-libzmq=$2;
- make && sudo make install;
- else
- echo "install czmq in $1. libzmq path is $2";
- ./configure --prefix=$1 --with-libzmq=$2;
- make && make install;
- fi
- elif [ $# == 1 ]
- then
- if [ $1 == "null" ]
- then
- echo "install czmq in default path.";
- ./configure;
- make && sudo make install;
- else
- echo "install czmq in $1";
- ./configure --prefix=$1;
- make && make install;
- fi
- else
- echo "ERROR: wrong command.";
- return -1;
- fi
-
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- cd ..;
- return 0;
-}
-
-function install_glog()
-{
- if [ ! -e "glog-0.3.3.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/glog-0.3.3.tar.gz;
- fi
-
- rm -rf glog-0.3.3;
- tar zxvf glog-0.3.3.tar.gz && cd glog-0.3.3;
-
- if [ $# == 1 ]
- then
- echo "install glog in $1";
- ./configure --prefix=$1;
- make && make install;
- elif [ $# == 0 ]
- then
- echo "install glog in default path";
- ./configure;
- make && sudo make install;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- cd ..;
- return 0;
-}
-
-function install_lmdb()
-{
- if [ ! -e "lmdb-0.9.10.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/lmdb-0.9.10.tar.gz;
- fi
-
- rm -rf mdb-mdb;
- tar zxvf lmdb-0.9.10.tar.gz && cd mdb-mdb/libraries/liblmdb;
-
- if [ $# == 1 ]
- then
- echo "install lmdb in $1";
- sed -i "26s#^.*#prefix=$1#g" Makefile;
- make && make install;
- elif [ $# == 0 ]
- then
- echo "install lmdb in default path";
- make && sudo make install;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ../../..;
- return -1;
- fi
- cd ../../..;
- return 0;
-}
-
-function install_openblas()
-{
- if [ ! -e "OpenBLAS.zip" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/OpenBLAS.zip;
- fi
-
- rm -rf OpenBLAS-develop;
- unzip OpenBLAS.zip && cd OpenBLAS-develop;
-
- make ONLY_CBLAS=1;
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- if [ $# == 1 ]
- then
- echo "install OpenBLAS in $1";
- make PREFIX=$1 install;
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- elif [ $# == 0 ]
- then
- echo "install OpenBLAS in default path"
- sudo make install;
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- else
- echo "wrong commands";
- fi
- cd ..;
-
- return 0;
-}
-
-function install_opencv()
-{
- if [ ! -e "opencv-2.4.10.zip" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/opencv-2.4.10.zip;
- fi
-
- rm -rf opencv-2.4.10;
- unzip opencv-2.4.10.zip && cd opencv-2.4.10;
-
- if [ $# == 1 ]
- then
- echo "install opencv in $1";
- cmake -DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=$1;
- make && make install;
- elif [ $# == 0 ]
- then
- echo "install opencv in default path";
- cmake -DCMAKE_BUILD_TYPE=RELEASE;
- make && sudo make install;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- cd ..;
- return 0;
-}
-
-function install_protobuf()
-{
- if [ ! -e "protobuf-2.6.0.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/protobuf-2.6.0.tar.gz;
- fi
-
- rm -rf protobuf-2.6.0;
- tar zxvf protobuf-2.6.0.tar.gz && cd protobuf-2.6.0;
-
- if [ $# == 1 ]
- then
- echo "install protobuf in $1";
- ./configure --prefix=$1;
- make && make install;
- #cd python;
- #python setup.py build;
- #python setup.py install --prefix=$1;
- #cd ..;
- elif [ $# == 0 ]
- then
- echo "install protobuf in default path";
- ./configure;
- make && sudo make install;
- #cd python;
- #python setup.py build;
- #sudo python setup.py install;
- #cd ..;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- cd ..;
- return 0;
-}
-
-function install_zeromq()
-{
- if [ ! -e "zeromq-3.2.2.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/zeromq-3.2.2.tar.gz;
- fi
-
- rm -rf zeromq-3.2.2;
- tar zxvf zeromq-3.2.2.tar.gz && cd zeromq-3.2.2;
-
- if [ $# == 1 ]
- then
- echo "install zeromq in $1";
- ./configure --prefix=$1;
- make && make install;
- elif [ $# == 0 ]
- then
- echo "install zeromq in default path";
- ./configure;
- make && sudo make install;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ..;
- return -1;
- fi
- cd ..;
- return 0;
-}
-
-function install_zookeeper()
-{
- if [ ! -e "zookeeper-3.4.6.tar.gz" ]
- then
- wget http://www.comp.nus.edu.sg/~dbsystem/singa/assets/file/thirdparty/zookeeper-3.4.6.tar.gz;
- fi
-
- rm -rf zookeeper-3.4.6;
- tar zxvf zookeeper-3.4.6.tar.gz;
- cd zookeeper-3.4.6/src/c;
-
- if [ $# == 1 ]
- then
- echo "install zookeeper in $1";
- ./configure --prefix=$1;
- make && make install;
- elif [ $# == 0 ]
- then
- echo "install zookeeper in default path";
- ./configure;
- make && sudo make install;
- else
- echo "wrong commands";
- fi
-
- if [ $? -ne 0 ]
- then
- cd ../../..;
- return -1;
- fi
-
- cd ../../..;
- return 0;
-}
-
-BIN=`dirname "${BASH_SOURCE-$0}"`
-BIN=`cd "$BIN">/dev/null; pwd`
-BASE=`cd "$BIN/..">/dev/null; pwd`
-
-cd $BIN
-
-while [ $# != 0 ]
-do
- case $1 in
-# "cmake")
-# echo "install cmake";
-# if [[ $2 == */* ]];then
-# install_cmake $2;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during cmake installation" ;
-# exit;
-# fi
-# shift
-# shift
-# else
-# install_cmake;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during cmake installation" ;
-# exit;
-# fi
-# shift
-# fi
-# ;;
- "czmq")
- echo "install czmq";
- if [ $2 == "-f" ]
- then
- if [[ $4 == */* ]]
- then
- install_czmq $4 $3;
- else
- install_czmq null $3;
- fi
- if [ $? -ne 0 ]
- then
- echo "ERROR during czmq installation" ;
- exit;
- fi
- shift
- shift
- shift
- shift
- elif [ $3 == "-f" ]
- then
- install_czmq $2 $4;
- if [ $? -ne 0 ]
- then
- echo "ERROR during czmq installation" ;
- exit;
- fi
- shift
- shift
- shift
- shift
- elif [[ $2 == */* ]]
- then
- install_czmq $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during czmq installation" ;
- exit;
- fi
- shift
- shift
- else
- install_czmq null;
- if [ $? -ne 0 ]
- then
- echo "ERROR during czmq installation" ;
- exit;
- fi
- shift
- fi
- ;;
- "glog")
- echo "install glog";
- if [[ $2 == */* ]];then
- install_glog $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during glog installation" ;
- exit;
- fi
- shift
- shift
- else
- install_glog;
- if [ $? -ne 0 ]
- then
- echo "ERROR during glog installation" ;
- exit;
- fi
- shift
- fi
- ;;
- "lmdb")
- echo "install lmdb";
- if [[ $2 == */* ]];then
- install_lmdb $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during lmdb installation" ;
- exit;
- fi
- shift
- shift
- else
- install_lmdb;
- if [ $? -ne 0 ]
- then
- echo "ERROR during lmdb installation" ;
- exit;
- fi
- shift
- fi
- ;;
- "OpenBLAS")
- echo "install OpenBLAS";
- if [[ $2 == */* ]];then
- install_openblas $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during openblas installation" ;
- exit;
- fi
- shift
- shift
- else
- install_openblas;
- if [ $? -ne 0 ]
- then
- echo "ERROR during openblas installation" ;
- exit;
- fi
- shift
- fi
- ;;
-# "opencv")
-# echo "install opencv";
-# if [[ $2 == */* ]];then
-# install_opencv $2;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during opencv installation" ;
-# exit;
-# fi
-# shift
-# shift
-# else
-# install_opencv;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during opencv installation" ;
-# exit;
-# fi
-# shift
-# fi
-# ;;
- "protobuf")
- echo "install protobuf";
- if [[ $2 == */* ]];then
- install_protobuf $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during protobuf installation" ;
- exit;
- fi
- shift
- shift
- else
- install_protobuf;
- if [ $? -ne 0 ]
- then
- echo "ERROR during protobuf installation" ;
- exit;
- fi
- shift
- fi
- ;;
- "zeromq")
- echo "install zeromq";
- if [[ $2 == */* ]];then
- install_zeromq $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zeromq installation" ;
- exit;
- fi
- shift
- shift
- else
- install_zeromq;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zeromq installation" ;
- exit;
- fi
- shift
- fi
- ;;
- "zookeeper")
- echo "install zookeeper";
- if [[ $2 == */* ]];then
- install_zookeeper $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zookeeper installation" ;
- exit;
- fi
- shift
- shift
- else
- install_zookeeper;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zookeeper installation" ;
- exit;
- fi
- shift
- fi
- ;;
- "all")
- echo "install all dependencies";
- if [[ $2 == */* ]];then
-# install_cmake $2;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during cmake installation" ;
-# exit;
-# fi
- install_zeromq $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zeromq installation" ;
- exit;
- fi
- install_czmq $2 $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during czmq installation" ;
- exit;
- fi
- install_glog $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during glog installation" ;
- exit;
- fi
-# install_lmdb $2;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during lmdb installation" ;
-# exit;
-# fi
- install_openblas $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during openblas installation" ;
- exit;
- fi
-# install_opencv $2;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during opencv installation" ;
-# exit;
-# fi
- install_protobuf $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during protobuf installation" ;
- exit;
- fi
- install_zookeeper $2;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zookeeper installation" ;
- exit;
- fi
- shift
- shift
- else
-# install_cmake;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during cmake installation" ;
-# exit;
-# fi
- install_zeromq;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zeromq installation" ;
- exit;
- fi
- install_czmq null;
- if [ $? -ne 0 ]
- then
- echo "ERROR during czmq installation" ;
- exit;
- fi
- install_glog;
- if [ $? -ne 0 ]
- then
- echo "ERROR during glog installation" ;
- exit;
- fi
-# install_lmdb;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during lmdb installation" ;
-# exit;
-# fi
- install_openblas;
- if [ $? -ne 0 ]
- then
- echo "ERROR during openblas installation" ;
- exit;
- fi
-# install_opencv;
-# if [ $? -ne 0 ]
-# then
-# echo "ERROR during opencv installation" ;
-# exit;
-# fi
- install_protobuf;
- if [ $? -ne 0 ]
- then
- echo "ERROR during protobuf installation" ;
- exit;
- fi
- install_zookeeper;
- if [ $? -ne 0 ]
- then
- echo "ERROR during zookeeper installation" ;
- exit;
- fi
- shift
- fi
- ;;
- *)
- echo "USAGE: ./install.sh [MISSING_LIBRARY_NAME1] [YOUR_INSTALL_PATH1] [MISSING_LIBRARY_NAME2] [YOUR_INSTALL_PATH2] ...";
- echo " MISSING_LIBRARY_NAME can be: "
-# echo " cmake"
- echo " czmq"
- echo " glog"
- echo " lmdb"
- echo " OpenBLAS"
-# echo " opencv"
- echo " protobuf"
- echo " zeromq"
- echo " zookeeper"
- echo " To install all dependencies, you can run: "
- echo " ./install.sh all"
- exit;
- esac
-done
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/graph.py
----------------------------------------------------------------------
diff --git a/tool/graph.py b/tool/graph.py
deleted file mode 100644
index e2870ed..0000000
--- a/tool/graph.py
+++ /dev/null
@@ -1,40 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-import sys
-import pygraphviz
-import networkx as nx
-from networkx.readwrite import json_graph
-import json
-
-
-if __name__=='__main__':
- print sys.argv
- if len(sys.argv)<3:
- print 'usage: draw the network graph\npython graph.py JSON_DAT FIG_FILE'
- sys.exit()
-
- with open(sys.argv[1]) as fd:
- nodelink=json.load(fd)
- G=json_graph.node_link_graph(nodelink)
- A = nx.to_agraph(G)
- A.layout('dot', args='-Nfontsize=10 -Nwidth=".2" -Nheight=".2" -Nmargin=0 \
- -Gfontsize=8')
- A.draw(sys.argv[2])
-
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/Makefile
----------------------------------------------------------------------
diff --git a/tool/mesos/Makefile b/tool/mesos/Makefile
deleted file mode 100644
index d64fb5b..0000000
--- a/tool/mesos/Makefile
+++ /dev/null
@@ -1,25 +0,0 @@
-CXX_FLAGS=-I ../include -std=c++11 -I /usr/local/include/hdfs -I ../../include
-LD_FLAGS=-lmesos -lsinga -lhdfs3
-EXE=scheduler
-OBJS=singa_scheduler.o scheduler.pb.o
-PROTOS=scheduler.proto
-PROTO_HDRS=scheduler.pb.h
-PROTO_SRCS=scheduler.pb.cc
-
-CXX=g++
-
-all: $(PROTO_HDRS) $(EXE)
-
-$(PROTO_SRCS) $(PROTO_HDRS): $(PROTOS)
- protoc --cpp_out=. $(PROTOS)
-
-$(EXE): $(OBJS)
- $(CXX) -o $@ $(OBJS) $(LD_FLAGS)
- rm -rf *.o
-
-%.o: %.cc
- $(CXX) $(CXX_FLAGS) -c $<
-
-clean:
- rm -rf *.o $(EXE) $(PROTO_HDRS) $(PROTO_SRCS)
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/README.md
----------------------------------------------------------------------
diff --git a/tool/mesos/README.md b/tool/mesos/README.md
deleted file mode 100644
index 5330801..0000000
--- a/tool/mesos/README.md
+++ /dev/null
@@ -1,86 +0,0 @@
-This guide explains how to start SINGA distributed training on a Mesos cluster. It assumes that both Mesos and HDFS are
-already running, and every node has SINGA installed.
-
-For this guide, we assume the following architecture, in which a cluster is set up from a set of containers (when the
-cluster is already set up, the user can immediate [start the training](#job_start)). Refer to [Docker guide](https://github.com/ug93tad/incubator-singa/tree/SINGA-89/tool/docker/README.md) for details of how to start individual nodes and set up network connection between them (make sure [weave](http://weave.works/guides/weave-docker-ubuntu-simple.html) is running at each node, and the cluster's headnode is running in container `node0`)
-
-<img src="http://www.comp.nus.edu.sg/~dinhtta/files/singa_mesos.png" align="center">
-
-## Start HDFS and Mesos
-Go inside each container, using:
-````
-docker exec -it nodeX /bin/bash
-````
-and configure it as follows:
-
-* On container `node0`
-
- hadoop namenode -format
- hadoop-daemon.sh start namenode
- /opt/mesos-0.22.0/build/bin/mesos-master.sh --work_dir=/opt --log_dir=/opt --quiet > /dev/null &
- zk-service.sh start
-
-* On container `node1, node2, ...`
-
- hadoop-daemon.sh start datanode
- /opt/mesos-0.22.0/build/bin/mesos-slave.sh --master=node0:5050 --log_dir=/opt --quiet > /dev/null &
-
-To check if the setup has been successful, check that HDFS namenode has registered `N` datanodes, via:
-
-````
-hadoop dfsadmin -report
-````
-
-#### Mesos logs
-Mesos logs are stored at `/opt/lt-mesos-master.INFO` on `node0` and `/opt/lt-mesos-slave.INFO` at other nodes.
-
----
-
-## Starting SINGA training on Mesos
-Assumed that Mesos and HDFS are already started, SINGA job can be launched at **any** container.
-
-#### Launching job
-
-1. Log in to any container, then
-
- cd incubator-singa/tool/mesos
-
-<a name="job_start"></a>
-2. Check that configuration files are correct:
- + `scheduler.conf` contains information about the master nodes
- + `singa.conf` contains information about Zookeeper node
- + Job configuration file `job.conf` **contains full path to the examples directories (NO RELATIVE PATH!).**
-
-3. Start the job:
- + If starting for the first time:
-
- ./scheduler <job config file> -scheduler_conf <scheduler config file> -singa_conf <SINGA config file>
-
- + If not the first time:
-
- ./scheduler <job config file>
-
-**Notes.** Each running job is given a `frameworkID`. Look for the log message of the form:
-
- Framework registered with XXX-XXX-XXX-XXX-XXX-XXX
-
-#### Monitoring and Debugging
-
-Each Mesos job is given a `frameworkID` and a *sandbox* directory is created for each job.
-The directory is in the specified `work_dir` (or `/tmp/mesos`) by default. For example, the error
-during SINGA execution can be found at:
-
- /tmp/mesos/slaves/xxxxx-Sx/frameworks/xxxxx/executors/SINGA_x/runs/latest/stderr
-
-Other artifacts, like files downloaded from HDFS (`job.conf`) and `stdout` can be found in the same
-directory.
-
-#### Stopping
-
-There are two way to kill the running job:
-
-1. If the scheduler is running in the foreground, simply kill it (using `Ctrl-C`, for example).
-2. If the scheduler is running in the background, kill it using Mesos's REST API:
-
- curl -d "frameworkId=XXX-XXX-XXX-XXX-XXX-XXX" -X POST http://<master>/master/shutdown
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/README.md
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/README.md b/tool/mesos/docker/README.md
deleted file mode 100644
index fd33e7e..0000000
--- a/tool/mesos/docker/README.md
+++ /dev/null
@@ -1,189 +0,0 @@
-This guide explains how to set up a development environment for SINGA. A cluster environment can be launched over one or multiple physical nodes.
-
-1. [Build SINGA base](#build_base)
-2. [Build SINGA with Mesos and Hadoop](#build_mesos)
-3. [Pre-built images](#pre_built)
-4. [Launch and stop SINGA (stand alone mode)](#launch_stand_alone)
-5. [Launch pseudo-distributed SINGA on one node](#launch_pseudo)
-6. [Launch fully distributed SINGA on multiple nodes](#launch_distributed)
-
----
-
-<a name="build_base"></a>
-#### Build SINGA base image
-
-````
-$ cd tool/docker/singa
-$ sudo docker build -t singa/base .
-$ sudo docker images
-REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-singa/base latest XXXX XXX 2.01 GB
-````
-
-The result is the image containing a built version of SINGA.
-
- 
-
- *Figure 1. singa/base Docker image, containing library dependencies and SINGA built from source.*
-
---
-
-<a name="build_mesos"></a>
-#### Build SINGA with Mesos and Hadoop
-````
-$ cd tool/docker/mesos
-$ sudo docker build -t singa/mesos .
-$ sudo docker images
-REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-singa/mesos latest XXXX XXX 4.935 GB
-````
- 
-
- *Figure 2. singa/mesos Docker image, containing Hadoop and Mesos built on
-top of SINGA. The default namenode address for Hadoop is `node0:9000`*
-
-**Notes** A common failure observed during the build process is caused by network failure occuring when downloading dependencies. Simply re-run the build command.
-
---
-
-<a name="pre_built"></a>
-#### Pre-built images on epiC cluster
-For users with access to the `epiC` cluster, there are pre-built and loaded Docker images at the following nodes:
-
- ciidaa-c18
- ciidaa-c19
-
-The available images at those nodes are:
-````
-REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
-singa/base latest XXXX XXX 2.01 GB
-singa/mesos latest XXXX XXX 4.935 GB
-weaveworks/weaveexec 1.1.1 XXXX 11 days ago 57.8 MB
-weaveworks/weave 1.1.1 XXXX 11 days ago 17.56 MB
-````
-
---
-
-<a name="launch_stand_alone"></a>
-#### Launch and stop SINGA in stand-alone mode
-To launch a test environment for a single-node SINGA training, simply start a container from `singa/base` image. The following starts a container called
-`XYZ`, then launches a shell in the container:
-
-````
-$ sudo docker run -dt --name XYZ singa/base /usr/bin/supervisord
-$ sudo docker exec -it XYZ /bin/bash
-````
-
- 
-
- *Figure 3. Launch SINGA in stand-alone mode: single node training*
-
-Inside the launched container, the SINGA source directory can be found at `/root/incubator-singa`.
-
-**Stopping the container**
-
-````
-$ sudo docker stop XYZ
-$ sudo docker rm ZYZ
-````
-
---
-
-<a name="launch_pseudo"></a>
-#### Launch SINGA on pseudo-distributed mode (single node)
-To simulate a distributed environment on a single node, one can repeat the
-previous step multiple times, each time giving a different name to the
-container. Network connections between these containers are already supported,
-thus SINGA instances/nodes in these container can readily communicate with each
-other.
-
-The previous approach requires the user to start SINGA instances individually
-at each container. Although there's a bash script for that, we provide a better
-way. In particular, multiple containers can be started from `singa/mesos` image
-which already bundles Mesos and Hadoop with SINGA. Using Mesos makes it easy to
-launch, stop and monitor the distributed execution from a single container.
-Figure 4 shows `N+1` containers running concurrently at the local host.
-
-````
-# sudo docker run -dt --name node0 singa/mesos /usr/bin/supervisord
-# sudo docker run -dt --name node1 singa/mesos /usr/bin/supervisord
-...
-````
- 
-
- *Figure 4. Launch SINGA in pseudo-distributed mode : multiple SINGA nodes over one single machine*
-
-**Starting SINGA distributed training**
-
-Refer to the [Mesos
-guide](https://github.com/ug93tad/incubator-singa/blob/SINGA-11/tool/mesos/README.md)
-for details of how to start training with multiple SINGA instances.
-
-**Important:** the container that assumes the role of Hadoop's namenode (and often Mesos's and Zookeeper's mater node as well) **must** be named `node0`. Otherwise, the user must log in to individual containers and change the Hadoop configuration separately.
-
---
-
-<a name="launch_distributed"></a>
-#### Launch SINGA on fully distributed mode (multiple nodes)
-The previous section has explained how to start a distributed environment on a
-single node. But running many containers on one node does not scale. When there
-are multiple physical hosts available, it is better to distribute the
-containers over them.
-
-The only extra requirement for the fully distributed mode, as compared with the
-pseudo distributed mode, is that the containers from different hosts are able
-to transparently communicate with each other. In the pseudo distributed mode,
-the local docker engine takes care of such communication. Here, we rely on
-[Weave](http://weave.works/guides/weave-docker-ubuntu-simple.html) to make the
-communication transparent. The resulting architecture is shown below.
-
- 
-
- *Figure 5. Launch SINGA in fully distributed mode: multiple SINGA nodes over multiple machines*
-
-**Install Weave at all hosts**
-```
-$ curl -L git.io/weave -o /usr/local/bin/weave
-$ chmod a+x /usr/local/bin/weave
-```
-
-**Starting Weave**
-
-Suppose `node0` will be launched at host with IP `111.222.111.222`.
-
-* At host `111.222.111.222`:
-````
-$ weave launch
-$ eval "$(weave env)" //if there's error, do `sudo -s` and try again
-````
-* At other hosts:
-````
-$ weave launch 111.222.111.222
-$ eval "$(weave env)" //if there's error, do `sudo -s` and try again
-````
-
-**Starting containers**
-
-The user logs in to each host and starts the container (same as in [pseudo-distributed](#launch_pseudo) mode). Note that container acting as the head node of the cluster must be named `node0` (and be running at the host with IP `111.222.111.222`, for example).
-
-**_Important_:** when there are other containers sharing the same host as `node0`, say `node1` and `node2` for example,
-there're additional changes to be made to `node1` and `node2`. Particularly, log in to each container and edit
-`/etc/hosts` file:
-
-````
-# modified by weave
-...
-X.Y.Z node0 node0.bridge //<- REMOVE this line
-..
-````
-This is to ensure that name resolutions (of `node0`'s address) from `node1` and `node2` are correct. By default,
-containers of the same host resolves each other's addresses via the Docker bridge. Instead, we want they to use
-addressed given by Weave.
-
-
-**Starting SINGA distributed training**
-
-Refer to the [Mesos
-guide](https://github.com/ug93tad/incubator-singa/blob/SINGA-11/tool/mesos/README.md)
-for details of how to start training with multiple SINGA instances.
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/.bashrc
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/.bashrc b/tool/mesos/docker/mesos/.bashrc
deleted file mode 100644
index c796a27..0000000
--- a/tool/mesos/docker/mesos/.bashrc
+++ /dev/null
@@ -1,32 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-export LIBRARY_PATH=/opt/OpenBLAS/lib:$LIBRARY_PATH
-export LD_LIBRARY_PATH=/opt/OpenBLAS/lib:/usr/local/lib:$LD_LIBRARY_PATH
-export CPLUS_INCLUDE_PATH=/opt/OpenBLAS/include:$CPLUS_INCLUDE_PATH
-export PATH=/opt/jdk1.8.0_60/bin:/opt/bin:$PATH
-export HADOOP_HOME=/opt/hadoop-2.6.0
-export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
-export JAVA_HOME=/opt/jdk1.8.0_60
-alias ls="ls --color=always"
-# some more ls aliases
-alias ll="ls -alF"
-alias la="ls -A"
-alias l="ls -CF"
-export SINGA_HOME=/root/incubator-singa
-export PATH=$PATH:$SINGA_HOME/bin
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/Dockerfile
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/Dockerfile b/tool/mesos/docker/mesos/Dockerfile
deleted file mode 100644
index 83142b3..0000000
--- a/tool/mesos/docker/mesos/Dockerfile
+++ /dev/null
@@ -1,40 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-#build Mesos+Hadoop on top of SINGA
-FROM singa/base
-MAINTAINER Anh Dinh<ug...@gmail.com>
-
-COPY incubator-singa/tool/docker/mesos/.bashrc /root/.bashrc
-COPY incubator-singa/tool/docker/mesos/install.sh /opt/install.sh
-COPY incubator-singa/tool/mesos/* /root/incubator-singa/tool/mesos/
-
-RUN cd /opt && source ./install.sh
-
-RUN apt-get -y install curl cmake libxml2 libxml2-dev uuid-dev protobuf-compiler libprotobuf-dev libgsasl7-dev libkrb5-dev libboost1.54-all-dev
-
-RUN cd /opt && source /root/.bashrc && wget -c http://www.eu.apache.org/dist/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz && tar -zxvf hadoop-2.6.0.tar.gz && git clone https://github.com/PivotalRD/libhdfs3.git && cd libhdfs3 && mkdir build && cd build && cmake ../ && make && make install
-
-#to be removed after SINGA-11 is merged (pulling from Anh's branch instead of pulling from the master)
-#RUN source ~/.bashrc && mkdir /root/mesos && cd /root/mesos && git clone https://github.com/ug93tad/incubator-singa && cd incubator-singa && git checkout SINGA-11 && cp -r tool/mesos /root/incubator-singa/tool/ && cd /root/incubator-singa/tool/mesos && make
-
-# update SINGA, but keep the examples directory
-RUN source ~/.bashrc && cd /root/incubator-singa/tool/mesos && make
-
-COPY incubator-singa/tool/docker/mesos/*.xml /opt/hadoop-2.6.0/etc/hadoop/
-RUN ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa && cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys && echo 'StrictHostKeyChecking no' >> ~/.ssh/config
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/core-site.xml
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/core-site.xml b/tool/mesos/docker/mesos/core-site.xml
deleted file mode 100644
index 49a39ca..0000000
--- a/tool/mesos/docker/mesos/core-site.xml
+++ /dev/null
@@ -1,24 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
-<!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<!-- Put site-specific property overrides in this file. -->
-
-<configuration>
- <property>
- <name>fs.default.name</name>
- <value>hdfs://node0:9000</value>
- </property>
-</configuration>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/hdfs-site.xml
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/hdfs-site.xml b/tool/mesos/docker/mesos/hdfs-site.xml
deleted file mode 100644
index 2e0f4c6..0000000
--- a/tool/mesos/docker/mesos/hdfs-site.xml
+++ /dev/null
@@ -1,34 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
-<!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<!-- Put site-specific property overrides in this file. -->
-
-<configuration>
-<property>
- <name>dfs.replication</name>
- <value>1</value>
- </property>
-
-<property>
- <name>dfs.name.dir</name>
- <value>file:///var/lib/hadoop/hdfs/namenode</value>
- </property>
-
- <property>
- <name>dfs.data.dir</name>
- <value>file:///var/lib/hadoop/hdfs/datanode</value>
- </property>
-</configuration>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/install.sh
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/install.sh b/tool/mesos/docker/mesos/install.sh
deleted file mode 100644
index 4b6ae79..0000000
--- a/tool/mesos/docker/mesos/install.sh
+++ /dev/null
@@ -1,35 +0,0 @@
-#!/bin/bash
-
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-source /root/.bashrc
-#download
-cd /opt
-wget -c http://archive.apache.org/dist/mesos/0.22.0/mesos-0.22.0.tar.gz
-wget https://www.comp.nus.edu.sg/~dinhtta/files/mesos_patch
-tar -zxvf mesos-0.22.0.tar.gz
-
-#patch and install mesos
-cd /opt/mesos-0.22.0
-patch -p5 < ../mesos_patch
-mkdir build; cd build
-../configure
-make
-sudo make install
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/mapred-site.xml
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/mapred-site.xml b/tool/mesos/docker/mesos/mapred-site.xml
deleted file mode 100644
index 42bb81c..0000000
--- a/tool/mesos/docker/mesos/mapred-site.xml
+++ /dev/null
@@ -1,22 +0,0 @@
-<?xml version="1.0"?>
-<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
-<!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
- http://www.apache.org/licenses/LICENSE-2.0
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-<!-- Put site-specific property overrides in this file. -->
-
-<configuration>
- <property>
- <name>mapreduce.framework.name</name>
- <value>yarn</value>
- </property>
-</configuration>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/mesos/yarn-site.xml
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/mesos/yarn-site.xml b/tool/mesos/docker/mesos/yarn-site.xml
deleted file mode 100644
index c137af9..0000000
--- a/tool/mesos/docker/mesos/yarn-site.xml
+++ /dev/null
@@ -1,32 +0,0 @@
-<?xml version="1.0"?>
-<!--
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-<configuration>
-
-<!-- Site specific YARN configuration properties -->
- <property>
- <name>yarn.nodemanager.aux-services</name>
- <value>mapreduce_shuffle</value>
- </property>
-
- <property>
- <name>yarn.resourcemanager.hostname</name>
- <value>node0</value>
- </property>
-
- <property>
- <name>yarn.resourcemanager.bind-host</name>
- <value>0.0.0.0</value>
- </property>
-</configuration>
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/singa/.bashrc
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/singa/.bashrc b/tool/mesos/docker/singa/.bashrc
deleted file mode 100644
index cf0712f..0000000
--- a/tool/mesos/docker/singa/.bashrc
+++ /dev/null
@@ -1,30 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-export LIBRARY_PATH=/opt/OpenBLAS/lib:$LIBRARY_PATH
-export LD_LIBRARY_PATH=/opt/OpenBLAS/lib:/usr/local/lib:$LD_LIBRARY_PATH
-export CPLUS_INCLUDE_PATH=/opt/OpenBLAS/include:$CPLUS_INCLUDE_PATH
-export PATH=$PATH:/opt/jdk1.8.0_60/bin:/opt/bin
-export JAVA_HOME=/opt/jdk1.8.0_60
-export SINGA_HOME=/root/incubator-singa
-export PATH=$PATH:$SINGA_HOME/bin
-alias ls="ls --color=always"
-# some more ls aliases
-alias ll="ls -alF"
-alias la="ls -A"
-alias l="ls -CF"
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/singa/Dockerfile
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/singa/Dockerfile b/tool/mesos/docker/singa/Dockerfile
deleted file mode 100644
index 0fc16bc..0000000
--- a/tool/mesos/docker/singa/Dockerfile
+++ /dev/null
@@ -1,34 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-#build singa from scratch
-FROM ubuntu:14.04
-MAINTAINER Anh Dinh<ug...@gmail.com>
-RUN rm /bin/sh && ln -s /bin/bash /bin/sh
-
-RUN apt-get update && apt-get -y install g++-4.8 build-essential git vim wget zip automake libtool man python-dev python-boto libcurl4-nss-dev libsasl2-dev maven libapr1-dev libsvn-dev openssh-server supervisor
-
-RUN cd /opt && wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u60-b27/jdk-8u60-linux-x64.tar.gz && cd /opt && tar -zxvf jdk-8u60-linux-x64.tar.gz && rm -rf jdk-8u60-linux-x64.tar.gz
-
-COPY incubator-singa/tool/docker/singa/ssh.conf /etc/supervisor/conf.d/
-COPY incubator-singa/tool/docker/singa/.bashrc /root/.bashrc
-COPY incubator-singa/tool/docker/singa/.vimrc /root/.vimrc
-ADD incubator-singa /root/incubator-singa
-
-RUN cd && source /root/.bashrc && cd incubator-singa && ./thirdparty/install.sh all && cd thirdparty/ && wget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gz && tar -zxvf protobuf-2.5.0.tar.gz && cd protobuf-2.5.0 && ./configure && make clean && make && make install && cd ../../ && cp thirdparty/install.sh . && rm -rf thirdparty/* && mv install.sh thirdparty/ && ./thirdparty/install.sh zookeeper && ./autogen.sh && ./configure && make && make install && cd examples/cifar10 && mv Makefile.example Makefile && make download && make create && service ssh start
-
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/singa/Dockerfile_gpu
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/singa/Dockerfile_gpu b/tool/mesos/docker/singa/Dockerfile_gpu
deleted file mode 100644
index da57479..0000000
--- a/tool/mesos/docker/singa/Dockerfile_gpu
+++ /dev/null
@@ -1,80 +0,0 @@
-#/**
-# * Licensed to the Apache Software Foundation (ASF) under one
-# * or more contributor license agreements. See the NOTICE file
-# * distributed with this work for additional information
-# * regarding copyright ownership. The ASF licenses this file
-# * to you under the Apache License, Version 2.0 (the
-# * "License"); you may not use this file except in compliance
-# * with the License. You may obtain a copy of the License at
-# *
-# * http://www.apache.org/licenses/LICENSE-2.0
-# *
-# * Unless required by applicable law or agreed to in writing, software
-# * distributed under the License is distributed on an "AS IS" BASIS,
-# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# * See the License for the specific language governing permissions and
-# * limitations under the License.
-# */
-
-FROM singa/base
-MAINTAINER NVIDIA CORPORATION <di...@nvidia.com>
-
-LABEL com.nvidia.volumes.needed="nvidia_driver"
-
-ENV NVIDIA_GPGKEY_SUM bd841d59a27a406e513db7d405550894188a4c1cd96bf8aa4f82f1b39e0b5c1c
-ENV NVIDIA_GPGKEY_FPR 889bee522da690103c4b085ed88c3d385c37d3be
-
-RUN apt-key adv --fetch-keys http://developer.download.nvidia.com/compute/cuda/repos/GPGKEY && \
- apt-key adv --export --no-emit-version -a $NVIDIA_GPGKEY_FPR | tail -n +2 > cudasign.pub && \
- echo "$NVIDIA_GPGKEY_SUM cudasign.pub" | sha256sum -c --strict - && rm cudasign.pub && \
- echo "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1404/x86_64 /" > /etc/apt/sources.list.d/cuda.list
-
-ENV CUDA_VERSION 7.0
-LABEL com.nvidia.cuda.version="7.0"
-
-ENV CUDA_PKG_VERSION 7-0=7.0-28
-RUN apt-get update && apt-get install -y --no-install-recommends --force-yes curl \
- cuda-nvrtc-$CUDA_PKG_VERSION \
- cuda-cusolver-$CUDA_PKG_VERSION \
- cuda-cublas-$CUDA_PKG_VERSION \
- cuda-cufft-$CUDA_PKG_VERSION \
- cuda-curand-$CUDA_PKG_VERSION \
- cuda-cusparse-$CUDA_PKG_VERSION \
- cuda-npp-$CUDA_PKG_VERSION \
- cuda-cudart-$CUDA_PKG_VERSION && \
- ln -s cuda-$CUDA_VERSION /usr/local/cuda
-
-RUN apt-get install -y --no-install-recommends --force-yes \
- cuda-core-$CUDA_PKG_VERSION \
- cuda-misc-headers-$CUDA_PKG_VERSION \
- cuda-command-line-tools-$CUDA_PKG_VERSION \
- cuda-license-$CUDA_PKG_VERSION \
- cuda-nvrtc-dev-$CUDA_PKG_VERSION \
- cuda-cusolver-dev-$CUDA_PKG_VERSION \
- cuda-cublas-dev-$CUDA_PKG_VERSION \
- cuda-cufft-dev-$CUDA_PKG_VERSION \
- cuda-curand-dev-$CUDA_PKG_VERSION \
- cuda-cusparse-dev-$CUDA_PKG_VERSION \
- cuda-npp-dev-$CUDA_PKG_VERSION \
- cuda-cudart-dev-$CUDA_PKG_VERSION \
- cuda-driver-dev-$CUDA_PKG_VERSION
-
-RUN echo "/usr/local/cuda/lib" >> /etc/ld.so.conf.d/cuda.conf && \
- echo "/usr/local/cuda/lib64" >> /etc/ld.so.conf.d/cuda.conf && \
- ldconfig
-
-ENV CUDNN_DOWNLOAD_SUM 98679d5ec039acfd4d81b8bfdc6a6352d6439e921523ff9909d364e706275c2b
-
-RUN curl -fsSL http://developer.download.nvidia.com/compute/redist/cudnn/v3/cudnn-7.0-linux-x64-v3.0-prod.tgz -O && \
- echo "$CUDNN_DOWNLOAD_SUM cudnn-7.0-linux-x64-v3.0-prod.tgz" | sha256sum -c --strict - && \
- tar -xzf cudnn-7.0-linux-x64-v3.0-prod.tgz -C /usr/local && \
- rm cudnn-7.0-linux-x64-v3.0-prod.tgz && \
- ldconfig
-
-RUN echo "/usr/local/nvidia/lib" >> /etc/ld.so.conf.d/nvidia.conf && \
- echo "/usr/local/nvidia/lib64" >> /etc/ld.so.conf.d/nvidia.conf
-
-ENV PATH /usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
-ENV LD_LIBRARY_PATH /usr/local/nvidia/lib:/usr/local/nvidia/lib64:${LD_LIBRARY_PATH}
-
-RUN cd && source /root/.bashrc && cd incubator-singa && ./configure --enable-cuda --with-cuda=/usr/local/cuda --enable-cudnn --with-cudnn=/usr/local/cuda && make && make install
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/docker/singa/ssh.conf
----------------------------------------------------------------------
diff --git a/tool/mesos/docker/singa/ssh.conf b/tool/mesos/docker/singa/ssh.conf
deleted file mode 100644
index ac04d7e..0000000
--- a/tool/mesos/docker/singa/ssh.conf
+++ /dev/null
@@ -1,5 +0,0 @@
-[supervisord]
-nodaemon=true
-
-[program:ssh]
-command=/usr/sbin/sshd -D
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/scheduler.conf
----------------------------------------------------------------------
diff --git a/tool/mesos/scheduler.conf b/tool/mesos/scheduler.conf
deleted file mode 100644
index 1ab5fba..0000000
--- a/tool/mesos/scheduler.conf
+++ /dev/null
@@ -1,3 +0,0 @@
-namenode: "node0:9000"
-master: "node0:5050"
-job_counter: 5
http://git-wip-us.apache.org/repos/asf/incubator-singa/blob/b2e21c62/tool/mesos/scheduler.proto
----------------------------------------------------------------------
diff --git a/tool/mesos/scheduler.proto b/tool/mesos/scheduler.proto
deleted file mode 100644
index 322f7bb..0000000
--- a/tool/mesos/scheduler.proto
+++ /dev/null
@@ -1,33 +0,0 @@
-/************************************************************
-*
-* Licensed to the Apache Software Foundation (ASF) under one
-* or more contributor license agreements. See the NOTICE file
-* distributed with this work for additional information
-* regarding copyright ownership. The ASF licenses this file
-* to you under the Apache License, Version 2.0 (the
-* "License"); you may not use this file except in compliance
-* with the License. You may obtain a copy of the License at
-*
-* http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing,
-* software distributed under the License is distributed on an
-* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-* KIND, either express or implied. See the License for the
-* specific language governing permissions and limitations
-* under the License.
-*
-*************************************************************/
-
-/**
- * Protocol Buffer message containing global information for the
- * scheduler.
- */
-message SchedulerProto{
- // HDFS namenode, for example node0:9000
- required string namenode=1;
- // Mesos master node, for example node0:5050
- required string master=2;
- // jobID, incremented at each run
- required int32 job_counter=3;
-}