You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@usergrid.apache.org by sn...@apache.org on 2014/02/12 15:20:37 UTC
git commit: Rename PriamCluster to AWSCluster,
and move out of Core Persistence as it is no longer needed for
testing there.
Updated Branches:
refs/heads/two-dot-o e578bb639 -> 56277504a
Rename PriamCluster to AWSCluster, and move out of Core Persistence as it is no longer needed for testing there.
Project: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/commit/56277504
Tree: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/tree/56277504
Diff: http://git-wip-us.apache.org/repos/asf/incubator-usergrid/diff/56277504
Branch: refs/heads/two-dot-o
Commit: 56277504a8e932493cd1159e33855db2b3069101
Parents: e578bb6
Author: Dave Johnson <dm...@apigee.com>
Authored: Wed Feb 12 09:03:51 2014 -0500
Committer: Dave Johnson <dm...@apigee.com>
Committed: Wed Feb 12 09:03:51 2014 -0500
----------------------------------------------------------------------
stack/awscluster/README.md | 84 +++++
stack/awscluster/assembly.xml | 29 ++
stack/awscluster/aws-sample.properties | 2 +
stack/awscluster/cassandra-cf.json | 315 +++++++++++++++++++
stack/awscluster/pom.xml | 139 ++++++++
.../src/main/dist/conf/Priam.properties | 21 ++
.../src/main/dist/init_instance/create_raid0.sh | 32 ++
.../main/dist/init_instance/init_instance.sh | 54 ++++
.../dist/init_instance/install_cassandra.sh | 56 ++++
.../dist/init_instance/install_elasticsearch.sh | 24 ++
.../dist/init_instance/install_oraclejdk.sh | 31 ++
.../src/main/dist/lib/priam-agent-1.2.24.jar | Bin 0 -> 33020 bytes
.../dist/lib/priam-cass-extensions-1.2.24.jar | Bin 0 -> 5478 bytes
.../src/main/dist/webapps/priam-web-1.2.24.war | Bin 0 -> 31272443 bytes
.../src/main/groovy/configure_cassandra.groovy | 113 +++++++
.../main/groovy/configure_elasticsearch.groovy | 47 +++
.../src/main/groovy/configure_priam.groovy | 41 +++
.../src/main/groovy/registry_clear.groovy | 20 ++
.../src/main/groovy/registry_list.groovy | 24 ++
.../src/main/groovy/registry_register.groovy | 38 +++
.../src/main/groovy/tag_instance.groovy | 28 ++
.../src/main/groovy/wait_for_instances.groovy | 44 +++
22 files changed, 1142 insertions(+)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/README.md
----------------------------------------------------------------------
diff --git a/stack/awscluster/README.md b/stack/awscluster/README.md
new file mode 100644
index 0000000..1f3f7f5
--- /dev/null
+++ b/stack/awscluster/README.md
@@ -0,0 +1,84 @@
+AWS Cluster
+===
+
+*UNDER CONSTRUCTION*
+
+This project provides a AWS Cloud Formation template that launches a complete Usergrid installation
+on Amazon EC2. Currently it launches a set of instances running Cassanrda, Priam and Elastic Search.
+
+Two parts:
+
+1) A Maven assembly that builds an installer tarball and uploads it to S3.
+The tarball includes scripts to install Oracle JDK, Cassandra and Priam on EC2 instance.
+
+2) A CloudFormation script that creates an autoscaling cluster, security group and some
+number of EC2 instances, The template includes a CloudInit script that runs scipts from
+the installer tarball to configure and start Cassandra on each node.
+
+
+Getting set-up
+---
+To setup our AWS account to use PriamCluster here's what you need to do:
+
+* __Create an AWS EC2 key pair__. You will need this PEM file to login to your Cassandra instances.
+
+* __Create an AWS S3 bucket__ for the PriamCluster install bundle. Create an S3 bucket in your
+account with the name `ug-cloudformation-priam`.
+
+* __Upload the Oracle Java JDK__ to that same S3 bucket. Best practice is to use the Oracle Java u
+JDK with Cassandra, so you must upload the JDK to the S3 bucket that we created above. The JDK must
+be named `jdk-7u45-linux-x64.gz`.
+
+* __Create AWS SimpleDB domain for Priam__ properties and instance registration. It's best if you
+use a SimpleDB client (e.g. SDBNavigator for Chrome) to create these domains. You must create two
+domains named `PriamProperties` and `InstanceIdentity`. Just create the domains and Priam will create
+the attributes.
+
+* __Create an aws.properties file__ with your AWS credentials in the same directory as this
+README file. The file is git-ignored so you don't have to worry about accidentally committing it.
+
+* __Deploy this the PriamCluster assembly__ by running the Maven command `mvn depoy` in the same
+directory as this README file.
+
+
+Launching a new stack
+---
+Login to AWS Console and create a new CloudFormation stack. On the first screen, pick a short
+and simple name for your stack and choose the option to upload a template and upload
+the `cassandra-cf.json` file in this very directory.
+
+On the next screen, enter the number of servers you wish to create, the replication factor and
+the instance type you wish to use. Check the "I acknowlege that this will create IAM resources"
+and then click Next, Next, Next to take defaults and start the stack.
+
+Watch the EC2 console and see your instances come up. They will be tagged with the stack name
+that you provided.
+
+
+Exploring a Cassandra node
+---
+Take a look at your new cluster. Login to one of the instances and take a look at the Cassandra
+setup via `nodetool ring` and by looking at the following logs locations:
+
+ /var/log/usergrid-bootstrap.log - log created as instance was created
+
+ /var/log/cassandra - the Cassandra log files
+
+
+How things work
+---
+Here's what happens when the stack is started.
+
+CloudFormation reads the `cassanrda-cf.json` template and uses starts the EC2 instances that it
+specifies. When each instance comes up CloudFormation runs the CloudInit script specified in
+`cassandra-cf.json` and that script downloads the PriamCluster, sets up some environment scripts
+and calls the `init_instance/init_cass.sh` script to complete the setup.
+
+The `init_instance/init_cass.sh` calls `install_oraclejdk.sh` to download the JDK from S3 and
+install it. The script then installs Tomcat and Cassandra. Next it uses Groovy scripts to configure
+Priam and Cassandra and wait for other Cassandra nodes to come alive.
+
+When Tomcat starts up, Priam comes to life. Priam reads it's SimpleDB tables, writes a new
+cassandra.yaml file and with the right seends and initial token starts Cassandra.
+
+
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/assembly.xml
----------------------------------------------------------------------
diff --git a/stack/awscluster/assembly.xml b/stack/awscluster/assembly.xml
new file mode 100644
index 0000000..b5406dd
--- /dev/null
+++ b/stack/awscluster/assembly.xml
@@ -0,0 +1,29 @@
+<assembly>
+ <id>any</id>
+ <formats>
+ <format>tar.gz</format>
+ </formats>
+
+ <includeBaseDirectory>false</includeBaseDirectory>
+
+ <fileSets>
+
+ <fileSet>
+ <outputDirectory>.</outputDirectory>
+ <directory>${project.basedir}/src/main/dist</directory>
+ <includes>
+ <include>**</include>
+ </includes>
+ </fileSet>
+
+ <fileSet>
+ <outputDirectory>./scripts</outputDirectory>
+ <directory>./src/main/groovy</directory>
+ <includes>
+ <include>**</include>
+ </includes>
+ </fileSet>
+
+ </fileSets>
+
+</assembly>
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/aws-sample.properties
----------------------------------------------------------------------
diff --git a/stack/awscluster/aws-sample.properties b/stack/awscluster/aws-sample.properties
new file mode 100644
index 0000000..c3ade61
--- /dev/null
+++ b/stack/awscluster/aws-sample.properties
@@ -0,0 +1,2 @@
+accesskey=<YourServiceAccessKey>
+secretkey=<YourServiceSecretKey>
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/cassandra-cf.json
----------------------------------------------------------------------
diff --git a/stack/awscluster/cassandra-cf.json b/stack/awscluster/cassandra-cf.json
new file mode 100644
index 0000000..a28cf48
--- /dev/null
+++ b/stack/awscluster/cassandra-cf.json
@@ -0,0 +1,315 @@
+{
+ "AWSTemplateFormatVersion": "2010-09-09",
+ "Description": "Cassandra Test Stack",
+ "Parameters": {
+ "ReleaseBucket": {
+ "Description": "S3 Bucket where Priam-Cluster assembly is to be found.",
+ "Type": "String",
+ "Default": "ug-cloudformation-priam"
+ },
+ "KeyPair": {
+ "Description": "EC2 key pair to be use for SSH access",
+ "Type": "String",
+ "Default": "ug-cloudformation"
+ },
+ "CassClusterName": {
+ "Description": "Name to be used for Cassandra cluster.",
+ "Type": "String",
+ "Default": "usergrid"
+ },
+ "CassNumServers": {
+ "Description": "Number of Cass servers to start.",
+ "Type": "Number",
+ "Default": "3",
+ "MinValue": "3"
+ },
+ "CassInstanceType": {
+ "Description": "Instance type for Cass servers",
+ "Type": "String",
+ "Default": "m1.xlarge",
+ "AllowedValues": ["m1.small", "m1.medium", "m1.large", "m1.xlarge", "m3.xlarge", "m3.2xlarge"],
+ "ConstraintDescription": "must be valid instance type."
+ },
+ "CassReplicationFactor": {
+ "Description": "Cassandra replication factor",
+ "Type": "Number",
+ "Default": "2",
+ "MinValue": "1"
+ },
+ "NotificationEmail": {
+ "Description": "Email for notifications",
+ "Type": "String",
+ "Default": "dmjohnson+cf@apigee.com"
+ }
+ },
+ "Mappings": {
+ "AWSInstanceType2Arch": {
+ "m1.small": {"Arch": "64"},
+ "m1.medium": {"Arch": "64"},
+ "m1.large": {"Arch": "64"},
+ "m1.xlarge": {"Arch": "64"}
+ },
+ "AWSRegionArch2AMI": {
+ "ap-southeast-2": {"64": "ami-db8611e1"},
+ "us-east-1": {"64": "ami-e864da81"},
+ "us-west-2": {"64": "ami-50f97060"}
+ }
+ },
+ "Resources": {
+ "CassUser": {
+ "Type": "AWS::IAM::User",
+ "Properties": {
+ "Path": "/",
+ "Policies": [{
+ "PolicyName": "root",
+ "PolicyDocument": {"Statement": [{
+ "Effect": "Allow",
+ "Action": "*",
+ "Resource": "*"
+ }]}
+ }]
+ }
+ },
+ "CassKey": {
+ "Type": "AWS::IAM::AccessKey",
+ "Properties": {
+ "UserName": {"Ref": "CassUser"}
+ }
+ },
+ "CassAutoScalingLaunchConfiguration": {
+ "Type": "AWS::AutoScaling::LaunchConfiguration",
+ "Properties": {
+ "UserData": {
+ "Fn::Base64": {"Fn::Join": ["", [
+
+ "#!/bin/bash -ex\n",
+
+ "# CASSANDRA NODE STARTUP \n",
+
+ "exec >/var/log/usergrid-bootstrap.log 2>&1\n",
+ "\n",
+ "mkdir -p /usr/share/usergrid\n",
+ "\n",
+
+ "# create script that sets our environment variables\n",
+ "cat >/etc/profile.d/usergrid-env.sh <<EOF\n",
+ "alias sudo='sudo -E'\n",
+ "\n",
+ "export TYPE=cass\n",
+ "export STACK_NAME=", {"Ref": "AWS::StackName"}, "\n",
+ "\n",
+ "export PUBLIC_HOSTNAME=`(curl -s http://169.254.169.254/latest/meta-data/public-hostname)`\n",
+ "export INTERNAL_HOSTNAME=`(curl http://169.254.169.254/latest/meta-data/local-ipv4)`\n",
+ "\n",
+ "export EC2_INSTANCE_ID=`ec2metadata --instance-id`\n",
+ "export EC2_REGION=", {"Ref": "AWS::Region"}, "\n",
+ "export EC2_URL=https://ec2.amazonaws.com/\n",
+ "\n",
+ "export CASS_SECURITY_GROUP_NAME=", {"Ref": "CassSecurityGroup"}, "\n",
+ "\n",
+ "export JVM_EXTRA_OPTS=-javaagent:/usr/share/cassandra/lib/priam-cass-extensions-1.2.24.jar", "\n",
+ "\n",
+ "export CASSANDRA_CLUSTER_NAME=", {"Ref": "CassClusterName"}, "\n",
+ "export CASSANDRA_NUM_SERVERS=", {"Ref": "CassNumServers"}, "\n",
+ "export CASSANDRA_KEYSPACE_NAME=usergrid", "\n",
+ "export CASSANDRA_REPLICATION_FACTOR=", {"Ref": "CassReplicationFactor"}, "\n",
+ "\n",
+ "export RELEASE_BUCKET=", {"Ref": "ReleaseBucket"}, "\n",
+ "\n",
+ "EOF\n",
+ "\n",
+
+ "# put AWS creds in environment\n",
+ "cat >/etc/profile.d/aws-credentials.sh <<EOF\n",
+ "export AWS_ACCESS_KEY=", {"Ref": "CassKey"}, "\n",
+ "export AWS_SECRET_KEY=", {"Fn::GetAtt": ["CassKey", "SecretAccessKey"]}, "\n",
+ "EOF\n",
+ "\n",
+
+ "# put AWS creds Priam's config file\n",
+ "cat >/etc/awscredential.properties <<EOF\n",
+ "AWSACCESSID=", {"Ref": "CassKey"}, "\n",
+ "AWSKEY=", {"Fn::GetAtt": ["CassKey", "SecretAccessKey"]}, "\n",
+ "EOF\n",
+ "\n",
+
+ "# setup s3cmd (will be installed by init script) \n",
+ "cat >/etc/s3cfg <<EOF\n",
+ "access_key=", {"Ref": "CassKey"}, "\n",
+ "secret_key=", {"Fn::GetAtt": ["CassKey", "SecretAccessKey"]}, "\n",
+ "EOF\n",
+ "chmod 644 /etc/s3cfg\n",
+ "ln -s /etc/s3cfg ~ubuntu/.s3cfg\n",
+ "ln -s /etc/s3cfg ~root/.s3cfg\n",
+ "\n",
+
+ "# download usergrid and init script bundle from S3\n",
+ "apt-get update\n",
+ "apt-get -y install s3cmd\n",
+ "cd /usr/share/usergrid\n",
+ "s3cmd --config=/etc/s3cfg get s3://", {"Ref": "ReleaseBucket"}, "/priamcluster-1.0-SNAPSHOT-any.tar.gz\n",
+ "tar xvf priamcluster-1.0-SNAPSHOT-any.tar.gz\n",
+ "rm -fr priamcluster-1.0-SNAPSHOT-any.tar.gz\n",
+ "chmod 755 ./init_instance/*.sh\n",
+ "cd ./init_instance\n",
+
+ "# init as a Cassandra node \n",
+ "sh ./init_instance.sh\n"
+
+ ]]}
+ },
+ "KeyName": {"Ref": "KeyPair"},
+ "ImageId": {
+ "Fn::FindInMap": [
+ "AWSRegionArch2AMI",
+ {
+ "Ref": "AWS::Region"
+ },
+ {
+ "Fn::FindInMap": [
+ "AWSInstanceType2Arch",
+ {
+ "Ref": "CassInstanceType"
+ },
+ "Arch"
+ ]
+ }
+ ]
+ },
+ "InstanceType": {
+ "Ref": "CassInstanceType"
+ },
+ "IamInstanceProfile": {
+ "Ref": "RootInstanceProfile"
+ },
+ "SecurityGroups": [{"Ref": "CassSecurityGroup"}]
+ }
+ },
+ "CassAutoScalingGroup": {
+ "Type": "AWS::AutoScaling::AutoScalingGroup",
+ "Version": "2009-05-15",
+ "Properties": {
+ "AvailabilityZones": {
+ "Fn::GetAZs": ""
+ },
+ "LaunchConfigurationName": {
+ "Ref": "CassAutoScalingLaunchConfiguration"
+ },
+ "MinSize": {"Ref": "CassNumServers"},
+ "MaxSize": {"Ref": "CassNumServers"},
+ "NotificationConfiguration": {
+ "TopicARN": {"Ref": "NotificationTopic"},
+ "NotificationTypes": [
+ "autoscaling:EC2_INSTANCE_LAUNCH",
+ "autoscaling:EC2_INSTANCE_LAUNCH_ERROR",
+ "autoscaling:EC2_INSTANCE_TERMINATE",
+ "autoscaling:EC2_INSTANCE_TERMINATE_ERROR"
+ ]
+ }
+ }
+ },
+ "NotificationTopic": {
+ "Type": "AWS::SNS::Topic",
+ "Properties": {
+ "Subscription": [{
+ "Endpoint": {"Ref": "NotificationEmail"},
+ "Protocol": "email"
+ }]
+ }
+ },
+ "RootRole": {
+ "Type": "AWS::IAM::Role",
+ "Properties": {
+ "AssumeRolePolicyDocument": {
+ "Statement": [
+ {
+ "Effect": "Allow",
+ "Principal": {
+ "Service": [
+ "ec2.amazonaws.com"
+ ]
+ },
+ "Action": [
+ "sts:AssumeRole"
+ ]
+ }
+ ]
+ },
+ "Path": "/"
+ }
+ },
+ "RolePolicies": {
+ "Type": "AWS::IAM::Policy",
+ "Properties": {
+ "PolicyName": "root",
+ "PolicyDocument": {
+ "Statement": [
+ {
+ "Effect": "Allow",
+ "Action": "*",
+ "Resource": "*"
+ }
+ ]
+ },
+ "Roles": [
+ {
+ "Ref": "RootRole"
+ }
+ ]
+ }
+ },
+ "RootInstanceProfile": {
+ "Type": "AWS::IAM::InstanceProfile",
+ "Properties": {
+ "Path": "/",
+ "Roles": [
+ {
+ "Ref": "RootRole"
+ }
+ ]
+ }
+ },
+ "CassSecurityGroup": {
+ "Type": "AWS::EC2::SecurityGroup",
+ "Properties": {
+ "GroupDescription": "Database Machines",
+ "SecurityGroupIngress": [{
+ "IpProtocol": "tcp",
+ "FromPort": "22",
+ "ToPort": "22",
+ "CidrIp": "0.0.0.0/0"
+ }]
+ }
+ },
+ "CassSecurityGroupPort7000": {
+ "Type": "AWS::EC2::SecurityGroupIngress",
+ "Properties": {
+ "GroupName": {"Ref": "CassSecurityGroup"},
+ "IpProtocol": "tcp",
+ "FromPort": "7000",
+ "ToPort": "7000",
+ "SourceSecurityGroupName": {"Ref": "CassSecurityGroup"}
+ }
+ },
+ "CassSecurityGroupPort9160": {
+ "Type": "AWS::EC2::SecurityGroupIngress",
+ "Properties": {
+ "GroupName": {"Ref": "CassSecurityGroup"},
+ "IpProtocol": "tcp",
+ "FromPort": "9160",
+ "ToPort": "9160",
+ "SourceSecurityGroupName": {"Ref": "CassSecurityGroup"}
+ }
+ },
+ "CassSecurityGroupPort9200": {
+ "Type": "AWS::EC2::SecurityGroupIngress",
+ "Properties": {
+ "GroupName": {"Ref": "CassSecurityGroup"},
+ "IpProtocol": "tcp",
+ "FromPort": "9200",
+ "ToPort": "9200",
+ "SourceSecurityGroupName": {"Ref": "CassSecurityGroup"}
+ }
+ }
+ }
+}
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/pom.xml
----------------------------------------------------------------------
diff --git a/stack/awscluster/pom.xml b/stack/awscluster/pom.xml
new file mode 100644
index 0000000..358f24e
--- /dev/null
+++ b/stack/awscluster/pom.xml
@@ -0,0 +1,139 @@
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <artifactId>priamcluster</artifactId>
+ <groupId>org.usergrid</groupId>
+ <version>1.0-SNAPSHOT</version>
+
+ <name>priamcluster</name>
+ <description>Creates bundle for initializing a Cassandra and Priam cluster via CloudFormation</description>
+
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <aws.s3.release-bucket>ug-cloudformation-priam</aws.s3.release-bucket>
+ </properties>
+
+ <build>
+
+ <plugins>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-assembly-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>make-assembly</id>
+ <phase>package</phase>
+ <goals>
+ <goal>single</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <descriptors>
+ <descriptor>assembly.xml</descriptor>
+ </descriptors>
+ <tarLongFileMode>gnu</tarLongFileMode>
+ </configuration>
+ </plugin>
+
+ <plugin>
+ <groupId>org.codehaus.mojo</groupId>
+ <artifactId>exec-maven-plugin</artifactId>
+ <version>1.2</version>
+ <executions>
+ <execution>
+ <phase>deploy</phase>
+ <goals>
+ <goal>java</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <includeProjectDependencies>true</includeProjectDependencies>
+ <includePluginDependencies>true</includePluginDependencies>
+ <classpathScope>runtime</classpathScope>
+ <executableDependency>
+ <groupId>net.java.dev.jets3t</groupId>
+ <artifactId>jets3t</artifactId>
+ </executableDependency>
+ <mainClass>org.jets3t.apps.synchronize.Synchronize</mainClass>
+ <arguments>
+ <argument>--nodelete</argument>
+ <argument>--properties</argument>
+ <argument>${project.basedir}/aws.properties</argument>
+ <argument>UP</argument>
+ <argument>${aws.s3.release-bucket}</argument>
+ <argument>${project.build.directory}/${project.build.finalName}-any.tar.gz</argument>
+ </arguments>
+ </configuration>
+ <dependencies>
+ <dependency>
+ <groupId>net.java.dev.jets3t</groupId>
+ <artifactId>jets3t</artifactId>
+ <version>0.8.0</version>
+ </dependency>
+ </dependencies>
+ </plugin>
+
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-deploy-plugin</artifactId>
+ <version>2.8.1</version>
+ <configuration>
+ <skip>true</skip>
+ </configuration>
+ </plugin>
+ <plugin>
+ <artifactId>maven-compiler-plugin</artifactId>
+ <version>2.3.2</version>
+ <dependencies>
+ <dependency>
+ <groupId>org.codehaus.groovy</groupId>
+ <artifactId>groovy-eclipse-compiler</artifactId>
+ <version>2.6.0-01</version>
+ </dependency>
+ </dependencies>
+ <configuration>
+ <compilerId>groovy-eclipse-compiler</compilerId>
+ </configuration>
+ </plugin>
+ <plugin>
+ <artifactId>groovy-eclipse-compiler</artifactId>
+ <groupId>org.codehaus.groovy</groupId>
+ <version>2.6.0-01</version>
+ <extensions>true</extensions>
+ </plugin>
+ </plugins>
+
+ </build>
+
+ <dependencies>
+
+ <dependency>
+ <groupId>net.java.dev.jets3t</groupId>
+ <artifactId>synchronize</artifactId>
+ <version>0.8.0</version>
+ </dependency>
+
+ <dependency>
+ <groupId>net.java.dev.jets3t</groupId>
+ <artifactId>jets3t</artifactId>
+ <version>0.8.0</version>
+ </dependency>
+
+ <dependency>
+ <groupId>com.amazonaws</groupId>
+ <artifactId>aws-java-sdk</artifactId>
+ <version>1.6.7</version>
+ </dependency>
+ <dependency>
+ <artifactId>groovy-all</artifactId>
+ <groupId>org.codehaus.groovy</groupId>
+ <version>2.0.5</version>
+ </dependency>
+ </dependencies>
+
+</project>
+
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/conf/Priam.properties
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/conf/Priam.properties b/stack/awscluster/src/main/dist/conf/Priam.properties
new file mode 100644
index 0000000..e327526
--- /dev/null
+++ b/stack/awscluster/src/main/dist/conf/Priam.properties
@@ -0,0 +1,21 @@
+
+# priam settings to be added to SimpleDB by configure_priam.groovy
+
+priam.clustername=usergrid2
+
+priam.cass.home=/etc/cassandra
+priam.cass.startscript=/etc/init.d/cassandra start
+priam.cass.stopscript=/etc/init.d/cassandra stop
+priam.yamlLocation=/etc/cassandra/cassandra.yaml
+
+priam.commitlog.location=/mnt/data/cassandra/commitlog
+priam.data.location=/mnt/data/cassandra/data
+priam.cache.location=/mnt/data/cassandra/saved_caches
+
+priam.s3.base_dir=priam-backup
+priam.s3.bucket=priam-cassandra-archive
+priam.backup.incremental.enable=false
+
+priam.partitioner=org.apache.cassandra.dht.RandomPartitioner
+
+priam.zones.available=us-east-1a,us-east-1b,us-east-1c,us-east-1d
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/init_instance/create_raid0.sh
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/init_instance/create_raid0.sh b/stack/awscluster/src/main/dist/init_instance/create_raid0.sh
new file mode 100644
index 0000000..401fcf4
--- /dev/null
+++ b/stack/awscluster/src/main/dist/init_instance/create_raid0.sh
@@ -0,0 +1,32 @@
+#!/bin/bash
+
+# install postfix first and prevent it from prompting the user
+debconf-set-selections <<< "postfix postfix/mailname string your.hostname.com"
+debconf-set-selections <<< "postfix postfix/main_mailer_type string 'Internet Site'"
+apt-get install -y postfix
+
+# install mdadm RAID controller
+apt-get -y install mdadm
+
+# m1.xlarge has four 414GB disks, but only /dev/xvdb is mounted
+# unmount /dev/xvdb so we can use it in our setup
+umount /mnt
+
+# create & format partition on each of our four disks
+(echo o; echo n; echo p; echo 1; echo ; echo; echo w) | fdisk /dev/xvdb
+(echo o; echo n; echo p; echo 1; echo ; echo; echo w) | fdisk /dev/xvdc
+(echo o; echo n; echo p; echo 1; echo ; echo; echo w) | fdisk /dev/xvdd
+(echo o; echo n; echo p; echo 1; echo ; echo; echo w) | fdisk /dev/xvde
+
+# create striped RAID0 device with our four disks
+mdadm --create --verbose /dev/md0 --level=stripe \
+--raid-devices=4 /dev/xvdb1 /dev/xvdc1 /dev/xvdd1 /dev/xvde1
+
+# save config
+mdadm --detail --scan | tee /etc/mdadm/mdadm.conf
+
+# create, mount and save disk to stab
+mkfs.ext4 -b 4096 -E stride=32,stripe-width=128 -L Stripe /dev/md0
+mkdir -p /mnt
+echo "/dev/md0 /mnt auto defaults,nobootwait,noatime 0 2" | tee /etc/fstab
+mount /mnt
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/init_instance/init_instance.sh
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/init_instance/init_instance.sh b/stack/awscluster/src/main/dist/init_instance/init_instance.sh
new file mode 100644
index 0000000..552df78
--- /dev/null
+++ b/stack/awscluster/src/main/dist/init_instance/init_instance.sh
@@ -0,0 +1,54 @@
+#!/bin/bash
+echo "${HOSTNAME}" > /etc/hostname
+echo "127.0.0.1 ${HOSTNAME}" >> /etc/hosts
+hostname `cat /etc/hostname`
+
+echo "US/Eastern" > /etc/timezone
+dpkg-reconfigure -f noninteractive tzdata
+
+# Build environment for scripts
+. /etc/profile.d/aws-credentials.sh
+. /etc/profile.d/usergrid-env.sh
+
+cd /usr/share/usergrid/init_instance
+./create_raid0.sh
+
+# Install the easy stuff
+PKGS="ntp unzip groovy tomcat7 curl"
+apt-get update
+apt-get -y install ${PKGS}
+/etc/init.d/tomcat7 stop
+
+# Install AWS Java SDK and get it into the Groovy classpath
+curl http://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip > /tmp/aws-sdk-java.zip
+cd /usr/share/
+unzip /tmp/aws-sdk-java.zip
+mkdir -p /home/ubuntu/.groovy/lib
+cp /usr/share/aws-java-sdk-*/third-party/*/*.jar /home/ubuntu/.groovy/lib
+cp /usr/share/aws-java-sdk-*/lib/* /home/ubuntu/.groovy/lib
+# except for evil stax
+rm /home/ubuntu/.groovy/lib/stax*
+ln -s /home/ubuntu/.groovy /root/.groovy
+
+cd /usr/share/usergrid/init_instance
+./install_oraclejdk.sh
+
+# Install and stop Cassandra
+cd /usr/share/usergrid/init_instance
+./install_cassandra.sh
+
+# Install and start ElasticSearch
+cd /usr/share/usergrid/init_instance
+./install_elasticsearch.sh
+/etc/init.d/elasticsearch start
+
+# Starting Tomcat starts Priam which starts Priam
+/etc/init.d/tomcat7 restart
+
+# Priam consistently craps out on first run
+# making this ugly kludge necessary
+sleep 90
+/etc/init.d/tomcat7 restart
+
+cd /usr/share/usergrid/scripts
+groovy tag_instance.groovy
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/init_instance/install_cassandra.sh
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/init_instance/install_cassandra.sh b/stack/awscluster/src/main/dist/init_instance/install_cassandra.sh
new file mode 100644
index 0000000..5bf7d04
--- /dev/null
+++ b/stack/awscluster/src/main/dist/init_instance/install_cassandra.sh
@@ -0,0 +1,56 @@
+#!/bin/bash
+
+# Install and stop Cassandra
+pushd /etc/apt/sources.list.d
+
+cat >> cassandra.sources.list << EOF
+deb http://www.apache.org/dist/cassandra/debian 12x main
+EOF
+apt-get update
+apt-get -y install libcap2
+apt-get --force-yes -y install cassandra
+/etc/init.d/cassandra stop
+
+mkdir -p /mnt/data/cassandra
+chown cassandra /mnt/data/cassandra
+
+# Wait for other instances to start up
+cd /usr/share/usergrid/scripts
+groovy registry_register.groovy
+groovy wait_for_instances.groovy
+
+# Configure Priam
+cd /usr/share/usergrid/scripts
+groovy configure_cassandra.groovy > /etc/cassandra/cassandra.yaml
+groovy configure_priam.groovy
+
+# Copy Priam extension into Cassandra and Priam WAR into Tomcat
+rm -rf /var/lib/tomcat7/webapps/*
+mkdir -p /usr/share/cassandra/lib
+cp /usr/share/usergrid/lib/priam-cass-extensions-1.2.24.jar /usr/share/cassandra/lib
+cp /usr/share/usergrid/webapps/priam-web-1.2.24.war /var/lib/tomcat7/webapps/Priam.war
+
+# Make sure Priam via Tomcat can write to /etc/cassandra
+# TODO: do this without 777
+chmod -R 777 /etc/cassandra
+chmod 777 /etc/init.d/cassandra
+
+# Configure sudo for no passwords to please Priam
+cat >> /tmp/sudoers.new << EOF
+Defaults env_reset
+Defaults mail_badpass
+Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
+root ALL=(ALL:ALL) ALL
+%admin ALL=(ALL) ALL
+%sudo ALL=(ALL:ALL) NOPASSWD:ALL
+EOF
+visudo -c -f /tmp/sudoers.new
+if [ "$?" -eq "0" ]; then
+ cp /tmp/sudoers.new /etc/sudoers
+fi
+# Add tomat user to sudoers to please Priam
+adduser tomcat7 sudo
+/etc/init.d/sudo restart
+
+popd
+
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/init_instance/install_elasticsearch.sh
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/init_instance/install_elasticsearch.sh b/stack/awscluster/src/main/dist/init_instance/install_elasticsearch.sh
new file mode 100644
index 0000000..080fa9a
--- /dev/null
+++ b/stack/awscluster/src/main/dist/init_instance/install_elasticsearch.sh
@@ -0,0 +1,24 @@
+#!/bin/bash
+
+# Install and stop ElasticSearch
+pushd /etc/apt/sources.list.d
+
+cat >> elasticsearch.sources.list << EOF
+deb http://packages.elasticsearch.org/elasticsearch/1.0/debian stable main
+EOF
+apt-get update
+apt-get --force-yes -y install elasticsearch
+/etc/init.d/elasticsearch stop
+
+mkdir -p /mnt/data/elasticsearch
+chown elasticsearch /mnt/data/elasticsearch
+
+mkdir -p /mnt/log/elasticsearch
+chown elasticsearch /mnt/log/elasticsearch
+
+# Configure and restart ElasticSearch
+update-rc.d elasticsearch defaults 95 10
+cd /usr/share/usergrid/scripts
+groovy ./configure_elasticsearch.groovy > /etc/elasticsearch/elasticsearch.yml
+
+popd
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/init_instance/install_oraclejdk.sh
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/init_instance/install_oraclejdk.sh b/stack/awscluster/src/main/dist/init_instance/install_oraclejdk.sh
new file mode 100644
index 0000000..f81c5fa
--- /dev/null
+++ b/stack/awscluster/src/main/dist/init_instance/install_oraclejdk.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+pushd /tmp
+
+# Build environment for scripts
+. /etc/profile.d/aws-credentials.sh
+. /etc/profile.d/usergrid-env.sh
+
+echo ${RELEASE_BUCKET}
+
+# Get JDK from the release bucket
+s3cmd --config=/etc/s3cfg get s3://${RELEASE_BUCKET}/jdk-7u45-linux-x64.gz
+
+# Install it as they do here:
+# http://askubuntu.com/questions/56104/how-can-i-install-sun-oracles-proprietary-java-6-7-jre-or-jdk
+tar -xvf jdk-7u45-linux-x64.gz
+mkdir -p /usr/lib/jvm
+mv ./jdk1.7.0_45 /usr/lib/jvm/jdk1.7.0
+
+update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.7.0/bin/java" 2000
+update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.7.0/bin/javac" 2000
+update-alternatives --install "/usr/bin/javaws" "javaws" "/usr/lib/jvm/jdk1.7.0/bin/javaws" 2000
+
+chmod a+x /usr/bin/java
+chmod a+x /usr/bin/javac
+chmod a+x /usr/bin/javaws
+chown -R root:root /usr/lib/jvm/jdk1.7.0
+
+sudo rm /usr/lib/jvm/default-java
+sudo ln -s /usr/lib/jvm/jdk1.7.0 /usr/lib/jvm/default-java
+
+popd
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/lib/priam-agent-1.2.24.jar
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/lib/priam-agent-1.2.24.jar b/stack/awscluster/src/main/dist/lib/priam-agent-1.2.24.jar
new file mode 100644
index 0000000..21e6980
Binary files /dev/null and b/stack/awscluster/src/main/dist/lib/priam-agent-1.2.24.jar differ
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/lib/priam-cass-extensions-1.2.24.jar
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/lib/priam-cass-extensions-1.2.24.jar b/stack/awscluster/src/main/dist/lib/priam-cass-extensions-1.2.24.jar
new file mode 100644
index 0000000..77c4f7b
Binary files /dev/null and b/stack/awscluster/src/main/dist/lib/priam-cass-extensions-1.2.24.jar differ
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/dist/webapps/priam-web-1.2.24.war
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/dist/webapps/priam-web-1.2.24.war b/stack/awscluster/src/main/dist/webapps/priam-web-1.2.24.war
new file mode 100644
index 0000000..016eed4
Binary files /dev/null and b/stack/awscluster/src/main/dist/webapps/priam-web-1.2.24.war differ
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/configure_cassandra.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/configure_cassandra.groovy b/stack/awscluster/src/main/groovy/configure_cassandra.groovy
new file mode 100644
index 0000000..a71e071
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/configure_cassandra.groovy
@@ -0,0 +1,113 @@
+//
+// configure_cassandra.groovy
+//
+// Emits Cassandra config file based on environment and Cassandra node
+// registry in SimpleDB
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String hostName = (String)System.getenv().get("PUBLIC_HOSTNAME")
+def clusterName = (String)System.getenv().get("CASSANDRA_CLUSTER_NAME")
+String domain = stackName
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+// build seed list by listing all Cassandra nodes found in SimpleDB domain with our stackName
+def selectResult = sdbClient.select(new SelectRequest((String)"select * from ${domain}"))
+def seeds = ""
+def sep = ""
+for (item in selectResult.getItems()) {
+ def att = item.getAttributes().get(0)
+ if (att.getValue().equals(stackName)) {
+ seeds = "${seeds}${sep}${item.getName()}"
+ sep = ","
+ }
+}
+
+def cassandraConfig = """
+cluster_name: '${clusterName}'
+listen_address: ${hostName}
+seed_provider:
+ - class_name: org.apache.cassandra.locator.SimpleSeedProvider
+ parameters:
+ - seeds: "${seeds}"
+auto_bootstrap: true
+initial_token:
+hinted_handoff_enabled: true
+hinted_handoff_throttle_in_kb: 1024
+max_hints_delivery_threads: 2
+authenticator: org.apache.cassandra.auth.AllowAllAuthenticator
+authorizer: org.apache.cassandra.auth.AllowAllAuthorizer
+partitioner: org.apache.cassandra.dht.RandomPartitioner
+data_file_directories:
+ - /mnt/data/cassandra/data
+commitlog_directory: /mnt/data/cassandra/commitlog
+disk_failure_policy: stop
+key_cache_size_in_mb:
+key_cache_save_period: 14400
+row_cache_size_in_mb: 0
+row_cache_save_period: 0
+row_cache_provider: SerializingCacheProvider
+saved_caches_directory: /mnt/data/cassandra/saved_caches
+commitlog_sync: periodic
+commitlog_sync_period_in_ms: 10000
+commitlog_segment_size_in_mb: 32
+flush_largest_memtables_at: 0.75
+reduce_cache_sizes_at: 0.85
+reduce_cache_capacity_to: 0.6
+concurrent_reads: 32
+concurrent_writes: 32
+memtable_flush_queue_size: 4
+trickle_fsync: false
+trickle_fsync_interval_in_kb: 10240
+storage_port: 7000
+ssl_storage_port: 7001
+rpc_address: 0.0.0.0
+start_native_transport: false
+native_transport_port: 9042
+start_rpc: true
+rpc_port: 9160
+rpc_keepalive: true
+rpc_server_type: sync
+thrift_framed_transport_size_in_mb: 15
+thrift_max_message_length_in_mb: 16
+incremental_backups: false
+snapshot_before_compaction: false
+auto_snapshot: true
+column_index_size_in_kb: 64
+in_memory_compaction_limit_in_mb: 64
+multithreaded_compaction: false
+compaction_throughput_mb_per_sec: 16
+compaction_preheat_key_cache: true
+read_request_timeout_in_ms: 10000
+range_request_timeout_in_ms: 10000
+write_request_timeout_in_ms: 10000
+truncate_request_timeout_in_ms: 60000
+request_timeout_in_ms: 10000
+cross_node_timeout: false
+endpoint_snitch: SimpleSnitch
+dynamic_snitch_update_interval_in_ms: 100
+dynamic_snitch_reset_interval_in_ms: 600000
+dynamic_snitch_badness_threshold: 0.1
+request_scheduler: org.apache.cassandra.scheduler.NoScheduler
+index_interval: 128
+server_encryption_options:
+ internode_encryption: none
+ keystore: conf/.keystore
+ keystore_password: cassandra
+ truststore: conf/.truststore
+ truststore_password: cassandra
+client_encryption_options:
+ enabled: false
+ keystore: conf/.keystore
+ keystore_password: cassandra
+internode_compression: all
+"""
+
+println cassandraConfig
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/configure_elasticsearch.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/configure_elasticsearch.groovy b/stack/awscluster/src/main/groovy/configure_elasticsearch.groovy
new file mode 100644
index 0000000..6b90e17
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/configure_elasticsearch.groovy
@@ -0,0 +1,47 @@
+
+//
+// configure_cassandra.groovy
+//
+// Emits Cassandra config file based on environment and Cassandra node
+// registry in SimpleDB
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String hostName = (String)System.getenv().get("PUBLIC_HOSTNAME")
+def clusterName = (String)System.getenv().get("CASSANDRA_CLUSTER_NAME")
+String domain = stackName
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+// build seed list by listing all Cassandra nodes found in SimpleDB domain with our stackName
+def selectResult = sdbClient.select(new SelectRequest((String)"select * from ${domain}"))
+def seeds = ""
+def sep = ""
+for (item in selectResult.getItems()) {
+ def att = item.getAttributes().get(0)
+ if (att.getValue().equals(stackName)) {
+ seeds = "${seeds}${sep}\"${item.getName()}\""
+ sep = ","
+ }
+}
+
+def elasticSearchConfig = """
+cluster.name: usergrid2
+discovery.zen.ping.multicast.enabled: false
+discovery.zen.ping.unicast.hosts: [${seeds}]
+node:
+ name: ${hostName}
+network:
+ host: ${hostName}
+path:
+ logs: /mnt/log/elasticsearch
+ data: /mnt/data/elasticsearch
+"""
+
+println elasticSearchConfig
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/configure_priam.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/configure_priam.groovy b/stack/awscluster/src/main/groovy/configure_priam.groovy
new file mode 100644
index 0000000..22f4661
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/configure_priam.groovy
@@ -0,0 +1,41 @@
+//
+// configure_priam.groovy
+//
+// Put our property overrides into Priam's SimpleDB table
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String domain = "PriamProperties"
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String hostName = (String)System.getenv().get("PUBLIC_HOSTNAME")
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+// creates domain or no-op if it already exists
+sdbClient.createDomain(new CreateDomainRequest(domain))
+
+def props = new Properties()
+props.load(new FileInputStream("../conf/Priam.properties"))
+
+for (name in props.stringPropertyNames()) {
+
+ def value = props.getProperty(name)
+ def key = "${stackName}${name}"
+
+ def attrs = new ArrayList()
+ attrs.add(new ReplaceableAttribute("appId", stackName, true))
+ attrs.add(new ReplaceableAttribute("property", name, true))
+ attrs.add(new ReplaceableAttribute("value", value, true))
+
+ // this will set new or update existing attributes
+ def par = new PutAttributesRequest(domain, key, attrs)
+ sdbClient.putAttributes(par);
+}
+
+println "Configured Priam."
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/registry_clear.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/registry_clear.groovy b/stack/awscluster/src/main/groovy/registry_clear.groovy
new file mode 100644
index 0000000..5baa537
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/registry_clear.groovy
@@ -0,0 +1,20 @@
+//
+// registry_clear.groovy
+//
+// Deletes the Cassandra node registry
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String hostName = (String)System.getenv().get("PUBLIC_HOSTNAME")
+String domain = stackName
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+// TODO: this should only clear out items owned by our stack, i.e. those with the same stack name
+sdbClient.deleteDomain(new DeleteDomainRequest(domain))
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/registry_list.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/registry_list.groovy b/stack/awscluster/src/main/groovy/registry_list.groovy
new file mode 100644
index 0000000..9871a21
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/registry_list.groovy
@@ -0,0 +1,24 @@
+//
+// registry_list.groovy
+//
+// List contents of registry as debugging aid. Not used in any other scripts.
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String hostName = (String)System.getenv().get("PUBLIC_HOSTNAME")
+String domain = stackName
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+def selectResult = sdbClient.select(new SelectRequest((String)"select * from ${domain}"))
+
+for (item in selectResult.getItems()) {
+ def att = item.getAttributes().get(0)
+ println "${item.getName()} -> ${att.getName()} : ${att.getValue()}"
+}
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/registry_register.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/registry_register.groovy b/stack/awscluster/src/main/groovy/registry_register.groovy
new file mode 100644
index 0000000..739c04e
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/registry_register.groovy
@@ -0,0 +1,38 @@
+//
+// configure_usergrid.groovy
+//
+// Register this host machine as a Cassandra node in our stack.
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String hostName = (String)System.getenv().get("PUBLIC_HOSTNAME")
+String domain = stackName
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+// creates domain or no-op if it already exists
+sdbClient.createDomain(new CreateDomainRequest(domain))
+
+def gar = new GetAttributesRequest(domain, hostName);
+def response = sdbClient.getAttributes(gar);
+if (response.getAttributes().size() == 1) {
+ println "Already registered"
+ def attrs = response.getAttributes()
+ for (att in attrs) {
+ println("${hostName} -> ${att.getName()} : ${att.getValue()}")
+ }
+} else {
+ println "Registering..."
+ def stackAtt = new ReplaceableAttribute("stackname", stackName, true)
+ def attrs = new ArrayList()
+ attrs.add(stackAtt)
+ def par = new PutAttributesRequest(domain, hostName, attrs)
+ sdbClient.putAttributes(par);
+ println "Registraition done."
+}
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/tag_instance.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/tag_instance.groovy b/stack/awscluster/src/main/groovy/tag_instance.groovy
new file mode 100644
index 0000000..76c5e86
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/tag_instance.groovy
@@ -0,0 +1,28 @@
+//
+// tag_instance.groovy
+//
+// Tag instance so we can easily identify it in the EC2 console
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.ec2.*
+import com.amazonaws.services.ec2.model.*
+
+String type = (String)System.getenv().get("TYPE")
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String instanceId = (String)System.getenv().get("EC2_INSTANCE_ID")
+String stackName = (String)System.getenv().get("STACK_NAME")
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def ec2Client = new AmazonEC2Client(creds)
+
+def resources = new ArrayList()
+resources.add(instanceId)
+
+def tags = new ArrayList()
+def tag = "${stackName}-${type}-${instanceId}"
+tags.add(new Tag("Name", tag))
+
+ec2Client.createTags(new CreateTagsRequest(resources, tags))
+
+println "Tagged instance as ${tag}"
http://git-wip-us.apache.org/repos/asf/incubator-usergrid/blob/56277504/stack/awscluster/src/main/groovy/wait_for_instances.groovy
----------------------------------------------------------------------
diff --git a/stack/awscluster/src/main/groovy/wait_for_instances.groovy b/stack/awscluster/src/main/groovy/wait_for_instances.groovy
new file mode 100644
index 0000000..4f125b6
--- /dev/null
+++ b/stack/awscluster/src/main/groovy/wait_for_instances.groovy
@@ -0,0 +1,44 @@
+//
+// wait_for_cassandra.groovy
+//
+// Wait for enough Cassandra servers are up before proceding,
+// Enough means count greater than or equal to replication factor.
+//
+import com.amazonaws.auth.*
+import com.amazonaws.services.simpledb.*
+import com.amazonaws.services.simpledb.model.*
+
+String accessKey = (String)System.getenv().get("AWS_ACCESS_KEY")
+String secretKey = (String)System.getenv().get("AWS_SECRET_KEY")
+String stackName = (String)System.getenv().get("STACK_NAME")
+String domain = stackName
+
+//def replicationFactor = System.getenv().get("CASSANDRA_REPLICATION_FACTOR")
+def cassNumServers = System.getenv().get("CASSANDRA_NUM_SERVERS")
+
+def creds = new BasicAWSCredentials(accessKey, secretKey)
+def sdbClient = new AmazonSimpleDBClient(creds)
+
+println "Waiting..."
+
+def count = 0
+while (true) {
+ try {
+ def selectResult = sdbClient.select(new SelectRequest((String)"select * from ${domain}"))
+ for (item in selectResult.getItems()) {
+ def att = item.getAttributes().get(0)
+ if (att.getValue().equals(stackName)) {
+ count++
+ }
+ }
+ if (count >= cassNumServers || count > 300) {
+ break
+ }
+ } catch (Exception e) {
+ println "ERROR waiting for Casasndra ${e.getMessage()}, will continue waiting"
+ return
+ }
+ Thread.sleep(1000)
+}
+
+println "Waiting done."