You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@bigtop.apache.org by rv...@apache.org on 2017/03/22 20:26:49 UTC
[1/3] bigtop git commit: BIGTOP-2702. Fix Sandbox creation script
Repository: bigtop
Updated Branches:
refs/heads/master 0d3448b81 -> c1cd696fe
BIGTOP-2702. Fix Sandbox creation script
Project: http://git-wip-us.apache.org/repos/asf/bigtop/repo
Commit: http://git-wip-us.apache.org/repos/asf/bigtop/commit/170df6d6
Tree: http://git-wip-us.apache.org/repos/asf/bigtop/tree/170df6d6
Diff: http://git-wip-us.apache.org/repos/asf/bigtop/diff/170df6d6
Branch: refs/heads/master
Commit: 170df6d6e1e4a7546adec0b3ace530937f3db4cd
Parents: 0d3448b
Author: Roman Shaposhnik <rv...@apache.org>
Authored: Mon Mar 20 11:57:39 2017 -0700
Committer: Roman Shaposhnik <rv...@apache.org>
Committed: Wed Mar 22 11:50:03 2017 -0700
----------------------------------------------------------------------
.../vm/vagrant-puppet-vm/create-sandboxes.sh | 31 -------------------
provisioner/docker/README.md | 9 ++++--
provisioner/vagrant/README.md | 19 ++++++++++--
provisioner/vagrant/create-sandboxes.sh | 32 ++++++++++++++++++++
4 files changed, 55 insertions(+), 36 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/bigtop/blob/170df6d6/bigtop-deploy/vm/vagrant-puppet-vm/create-sandboxes.sh
----------------------------------------------------------------------
diff --git a/bigtop-deploy/vm/vagrant-puppet-vm/create-sandboxes.sh b/bigtop-deploy/vm/vagrant-puppet-vm/create-sandboxes.sh
deleted file mode 100755
index fb357e8..0000000
--- a/bigtop-deploy/vm/vagrant-puppet-vm/create-sandboxes.sh
+++ /dev/null
@@ -1,31 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-#!/bin/bash
-
-set -ex
-WORKDIR=${1:-$0.work.dir}
-echo "Cleaning $WORKDIR"
-rm -rf $WORKDIR
-mkdir -p $WORKDIR
-VERSION=${2:-1.0}
-NAME="vagrant-bigtop-v$VERSION"
-# clean everything
-vagrant destroy -f
-vagrant up
-vagrant package --output $WORKDIR/${NAME}.box
-vagrant halt
-vmname=`VBoxManage list vms | cut -d \" -f 2 | grep vagrant-puppet-vm_bigtop`
-VBoxManage export $vmname --output=$WORKDIR/${NAME}.ova --ovf20
-vagrant destroy -f
http://git-wip-us.apache.org/repos/asf/bigtop/blob/170df6d6/provisioner/docker/README.md
----------------------------------------------------------------------
diff --git a/provisioner/docker/README.md b/provisioner/docker/README.md
index ef52ff1..e784e3b 100644
--- a/provisioner/docker/README.md
+++ b/provisioner/docker/README.md
@@ -15,7 +15,7 @@
------------------------------------------------------------------------------------------------------------------------------------------------------
-#BigTop Docker provisioner
+# BigTop Docker provisioner
## Overview
@@ -33,6 +33,7 @@ This has been verified on Docker Engine 1.9.1, with api version 1.15, and Docker
### OS X and Windows
* Install [Docker Toolbox](https://www.docker.com/docker-toolbox)
+* Install Ruby
### Linux
@@ -40,6 +41,8 @@ This has been verified on Docker Engine 1.9.1, with api version 1.15, and Docker
* Install [Docker Compose](https://docs.docker.com/compose/install/)
+* Install Ruby
+
* Start the Docker daemon
```
@@ -115,7 +118,7 @@ usage: docker-hadoop.sh [-C file ] args
-h, --help
```
-##Configurations
+## Configurations
* There are several parameters can be configured in config.yaml:
@@ -135,7 +138,7 @@ If you've built packages using local cloned bigtop and produced the apt/yum repo
enable_local_repo = true
```
-##Configure Apache Hadoop ecosystem components
+## Configure Apache Hadoop ecosystem components
* Choose the ecosystem you want to be deployed by modifying components in config.yaml
```
http://git-wip-us.apache.org/repos/asf/bigtop/blob/170df6d6/provisioner/vagrant/README.md
----------------------------------------------------------------------
diff --git a/provisioner/vagrant/README.md b/provisioner/vagrant/README.md
index 0b1ff20..d4a1ec3 100644
--- a/provisioner/vagrant/README.md
+++ b/provisioner/vagrant/README.md
@@ -15,7 +15,7 @@
------------------------------------------------------------------------------------------------------------------------------------------------------
-#BigTop VM provisioner
+# BigTop VM provisioner
## Overview
@@ -88,7 +88,7 @@ or
$ MEMORY_SIZE=8000 vagrant up
```
-##Configure Apache Hadoop ecosystem components
+## Configure Apache Hadoop ecosystem components
* Choose the ecosystem you want to be deployed by modifying components in vagrantconfig.yaml
```
@@ -97,3 +97,18 @@ components: "hadoop, hbase, yarn,..."
By default, Apache Hadoop and YARN will be installed.
See `bigtop-deploy/puppet/hieradata/site.yaml` for more details.
+
+## Sandbox VM generation
+
+Vagrant is a useful tool for times when you would like to capture the state of the VM and
+publish it as a sandbox for others to use. We provide a wrapper script for this that works
+with VirtualBox. Here's how to run it:
+
+```
+./create-sandboxes.sh [temp work dir] [version]
+```
+If you don't specify the temp work dir the current directory is going to be used, the version
+by default is 1.0.
+
+Once the script is done running you will have vagrant-bigtop-v<version>.box and
+vagrant-bigtop-v<version>.ova VM images available to you under the temp work dir.
http://git-wip-us.apache.org/repos/asf/bigtop/blob/170df6d6/provisioner/vagrant/create-sandboxes.sh
----------------------------------------------------------------------
diff --git a/provisioner/vagrant/create-sandboxes.sh b/provisioner/vagrant/create-sandboxes.sh
new file mode 100755
index 0000000..e90e821
--- /dev/null
+++ b/provisioner/vagrant/create-sandboxes.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -ex
+WORKDIR=${1:-$0.work.dir}
+echo "Cleaning $WORKDIR"
+rm -rf $WORKDIR
+mkdir -p $WORKDIR
+VERSION=${2:-1.0}
+NAME="vagrant-bigtop-v$VERSION"
+# clean everything
+vagrant destroy -f
+vagrant up
+vagrant package --output $WORKDIR/${NAME}.box
+vagrant halt
+vmname=`VBoxManage list vms | cut -d \" -f 2 | grep vagrant_bigtop`
+VBoxManage export $vmname --output=$WORKDIR/${NAME}.ova --ovf20
+vagrant destroy -f
[2/3] bigtop git commit: BIGTOP-2710. qfs debian build is breaking
because it is confused about Maven home
Posted by rv...@apache.org.
BIGTOP-2710. qfs debian build is breaking because it is confused about Maven home
Project: http://git-wip-us.apache.org/repos/asf/bigtop/repo
Commit: http://git-wip-us.apache.org/repos/asf/bigtop/commit/4be5b98e
Tree: http://git-wip-us.apache.org/repos/asf/bigtop/tree/4be5b98e
Diff: http://git-wip-us.apache.org/repos/asf/bigtop/diff/4be5b98e
Branch: refs/heads/master
Commit: 4be5b98e6a4f931437897137dc10a2a36c7adc67
Parents: 170df6d
Author: Roman Shaposhnik <rv...@apache.org>
Authored: Wed Mar 22 11:50:41 2017 -0700
Committer: Roman Shaposhnik <rv...@apache.org>
Committed: Wed Mar 22 12:49:48 2017 -0700
----------------------------------------------------------------------
bigtop-packages/src/common/qfs/do-component-build | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/bigtop/blob/4be5b98e/bigtop-packages/src/common/qfs/do-component-build
----------------------------------------------------------------------
diff --git a/bigtop-packages/src/common/qfs/do-component-build b/bigtop-packages/src/common/qfs/do-component-build
index 3284cd3..738f011 100644
--- a/bigtop-packages/src/common/qfs/do-component-build
+++ b/bigtop-packages/src/common/qfs/do-component-build
@@ -83,6 +83,9 @@ command -v mvn > /dev/null 2>&1 || {
PYTHON3_PATH=${PYTHON3_PATH:-}
+# Workaround for BIGTOP-2710
+sed -ie '/which/!s#mvn #mvn -Duser.home=${HOME} #g' src/java/javabuild.sh
+
# Build and install QFS into the build/release directory
make
@@ -111,8 +114,8 @@ get_hadoop_qfs_profile() {
HADOOP_QFS_PROFILE=$(get_hadoop_qfs_profile)
cd src/java
-mvn -Dqfs.release.version=${QFS_VERSION} --projects qfs-access package
-mvn -P $HADOOP_QFS_PROFILE -Dqfs.release.version=${QFS_VERSION} \
+mvn -Duser.home=${HOME} -Dqfs.release.version=${QFS_VERSION} --projects qfs-access package
+mvn -P $HADOOP_QFS_PROFILE -Duser.home=${HOME} -Dqfs.release.version=${QFS_VERSION} \
-Dhadoop.release.version=${HADOOP_VERSION} -Dtest.build.data=/tmp \
package
cd ../..
[3/3] bigtop git commit: BIGTOP-2711. fix rat problems
Posted by rv...@apache.org.
BIGTOP-2711. fix rat problems
Project: http://git-wip-us.apache.org/repos/asf/bigtop/repo
Commit: http://git-wip-us.apache.org/repos/asf/bigtop/commit/c1cd696f
Tree: http://git-wip-us.apache.org/repos/asf/bigtop/tree/c1cd696f
Diff: http://git-wip-us.apache.org/repos/asf/bigtop/diff/c1cd696f
Branch: refs/heads/master
Commit: c1cd696fe405e7b622d9ab14440053a79f825112
Parents: 4be5b98
Author: Roman Shaposhnik <rv...@apache.org>
Authored: Wed Mar 22 12:17:50 2017 -0700
Committer: Roman Shaposhnik <rv...@apache.org>
Committed: Wed Mar 22 12:50:14 2017 -0700
----------------------------------------------------------------------
bigtop-packages/src/common/ambari/ambari-server.svc | 14 ++++++++++++++
bigtop-packages/src/common/ambari/ambari.defaults | 14 ++++++++++++++
build.gradle | 1 +
docker/sandbox/README.md | 15 +++++++++++++++
4 files changed, 44 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/bigtop/blob/c1cd696f/bigtop-packages/src/common/ambari/ambari-server.svc
----------------------------------------------------------------------
diff --git a/bigtop-packages/src/common/ambari/ambari-server.svc b/bigtop-packages/src/common/ambari/ambari-server.svc
index e69de29..ae1e83e 100644
--- a/bigtop-packages/src/common/ambari/ambari-server.svc
+++ b/bigtop-packages/src/common/ambari/ambari-server.svc
@@ -0,0 +1,14 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
http://git-wip-us.apache.org/repos/asf/bigtop/blob/c1cd696f/bigtop-packages/src/common/ambari/ambari.defaults
----------------------------------------------------------------------
diff --git a/bigtop-packages/src/common/ambari/ambari.defaults b/bigtop-packages/src/common/ambari/ambari.defaults
index e69de29..ae1e83e 100644
--- a/bigtop-packages/src/common/ambari/ambari.defaults
+++ b/bigtop-packages/src/common/ambari/ambari.defaults
@@ -0,0 +1,14 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
http://git-wip-us.apache.org/repos/asf/bigtop/blob/c1cd696f/build.gradle
----------------------------------------------------------------------
diff --git a/build.gradle b/build.gradle
index 48a1fde..08c2216 100644
--- a/build.gradle
+++ b/build.gradle
@@ -118,6 +118,7 @@ rat {
"bigtop-packages/src/charm/**/*.json",
"bigtop-packages/src/charm/**/*.svg",
"bigtop-packages/src/charm/**/*.yaml",
+ "bigtop-packages/src/charm/**/resources/**/*.txt",
"bigtop-deploy/juju/**/*.yaml",
"bigtop-tests/cloud-weather-report/**/*.yaml",
/* Misc individual files */
http://git-wip-us.apache.org/repos/asf/bigtop/blob/c1cd696f/docker/sandbox/README.md
----------------------------------------------------------------------
diff --git a/docker/sandbox/README.md b/docker/sandbox/README.md
index 89c9e5e..711903c 100644
--- a/docker/sandbox/README.md
+++ b/docker/sandbox/README.md
@@ -1,3 +1,18 @@
+Licensed to the Apache Software Foundation (ASF) under one or more
+contributor license agreements. See the NOTICE file distributed with
+this work for additional information regarding copyright ownership.
+The ASF licenses this file to You under the Apache License, Version 2.0
+(the "License"); you may not use this file except in compliance with
+the License. You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
# Bigtop Docker Sandbox
A tool to build and run big data pseudo cluster using Docker.