You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@brooklyn.apache.org by Nakomis <gi...@git.apache.org> on 2014/06/26 17:57:43 UTC
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
GitHub user Nakomis opened a pull request:
https://github.com/apache/incubator-brooklyn/pull/25
Couchbase sync gateway
Resolves jira task BROOKLYN-9
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Nakomis/incubator-brooklyn couchbase-sync-gateway
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-brooklyn/pull/25.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #25
----
commit f9576bac4c82b29f5f0812b4f0d44393869a582e
Author: Martin Harris <gi...@nakomis.com>
Date: 2014-06-20T09:34:39Z
Copied in @ZaidM's work from https://github.com/ZaidM/brooklyn/tree/couchbase-sync-gateway1
commit b06812074916216634b72791718cc6ab7b9951b3
Author: Martin Harris <gi...@nakomis.com>
Date: 2014-06-25T15:44:11Z
Fixed bucket creation
commit 99dadcb267d5f13c5bb8f968cb81fa2dd836890e
Author: Martin Harris <gi...@nakomis.com>
Date: 2014-06-26T15:56:46Z
Fixed CouchbaseSyncGateway live tests
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701467
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayDriver.java ---
@@ -0,0 +1,9 @@
+package brooklyn.entity.nosql.couchbase;
+
+import brooklyn.entity.basic.SoftwareProcessDriver;
+
+public interface CouchbaseSyncGatewayDriver extends SoftwareProcessDriver {
+
+ public String getOsTag();
--- End diff --
Worth javadoc to say what format result will be - it's used in freemarker for building `CouchbaseSyncGateway.DOWNLOAD_URL` (hence me being ok with it being on the interface). That download url will have strict requirements for what format osTag needs to be in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14663845
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
--- End diff --
My personal rule of thumb is a `FIXME` is something that should be fixed before merged; a `TODO` is a suggestion of something that should be done in the future.
It looks like you've added synchronization. Should this comment be about doing it in a different way?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701347
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGateway.java ---
@@ -0,0 +1,61 @@
+package brooklyn.entity.nosql.couchbase;
+
+import brooklyn.config.ConfigKey;
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.ConfigKeys;
+import brooklyn.entity.basic.SoftwareProcess;
+import brooklyn.entity.proxying.ImplementedBy;
+import brooklyn.event.AttributeSensor;
+import brooklyn.event.basic.BasicAttributeSensorAndConfigKey;
+import brooklyn.event.basic.PortAttributeSensorAndConfigKey;
+import brooklyn.event.basic.Sensors;
+import brooklyn.util.flags.SetFromFlag;
+
+@ImplementedBy(CouchbaseSyncGatewayImpl.class)
+public interface CouchbaseSyncGateway extends SoftwareProcess {
+
+ @SetFromFlag("version")
+ ConfigKey<String> SUGGESTED_VERSION = ConfigKeys.newConfigKeyWithDefault(SoftwareProcess.SUGGESTED_VERSION,
+ "1.0-beta3.1");
+
+ @SetFromFlag("downloadUrl")
+ BasicAttributeSensorAndConfigKey<String> DOWNLOAD_URL = new BasicAttributeSensorAndConfigKey<String>(
+ SoftwareProcess.DOWNLOAD_URL, "http://packages.couchbase.com/releases/couchbase-sync-gateway/1.0-beta/couchbase-sync-gateway-community_${version}_${driver.osTag}");
+
+ @SetFromFlag("couchbaseServer")
+ ConfigKey<Entity> COUCHBASE_SERVER = ConfigKeys.newConfigKey(Entity.class, "couchbaseSyncGateway.couchbaseNode",
+ "Couchbase server node or cluster the sync gateway connects to");
+
+ @SetFromFlag("serverPool")
+ ConfigKey<String> COUCHBASE_SERVER_POOL = ConfigKeys.newStringConfigKey("couchbaseSyncGateway.serverPool",
+ "Couchbase Server pool name in which to find buckets", "default");
+
+ @SetFromFlag("couchbaseServerBucket")
+ ConfigKey<String> COUCHBASE_SERVER_BUCKET = ConfigKeys.newStringConfigKey("couchbaseSyncGateway.serverBucket",
+ "Name of the Couchbase bucket to use", "sync_gateway");
+
+ @SetFromFlag("couchbaseServerUrl")
+ ConfigKey<String> COUCHBASE_SERVER_URL = ConfigKeys.newStringConfigKey("couchbaseSyncGateway.couchbaseServerUrl",
--- End diff --
This config key isn't used anywhere. Is it needed? Or should it be deleted?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-48452750
@Nakomis finished reviewing this now - lots of great stuff; mostly minor comments. Ping me when it's ready for review again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701982
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
+ if (cbNode instanceof CouchbaseCluster) {
+ Optional<Entity> cbClusterNode = Iterables.tryFind(cbNode.getAttribute(CouchbaseCluster.GROUP_MEMBERS), new Predicate<Entity>() {
+
+ @Override
+ public boolean apply(@Nullable Entity entity) {
+ if (entity instanceof CouchbaseNode && Boolean.TRUE.equals(entity.getAttribute(CouchbaseNode.IS_IN_CLUSTER))) {
+ return true;
+ }
+ return false;
+ }
+ });
+ if (cbClusterNode.isPresent()) {
+ cbNode = cbClusterNode.get();
+ } else {
+ throw new IllegalArgumentException(format("The cluster %s does not contain any suitable Couchbase nodes to connect to..", cbNode.getId()));
+ }
+
+ }
+ String hostname = cbNode.getAttribute(CouchbaseNode.HOSTNAME);
+ String webPort = cbNode.getAttribute(CouchbaseNode.COUCHBASE_WEB_ADMIN_PORT).toString();
+
+
+ String username = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_USERNAME);
+ String password = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD);
+
+ String bucketName = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET);
+ String pool = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_POOL);
+ String pretty = entity.getConfig(CouchbaseSyncGateway.PRETTY) ? "-pretty" : "";
+ String verbose = entity.getConfig(CouchbaseSyncGateway.VERBOSE) ? "-verbose" : "";
+
+ String adminRestApiPort = entity.getConfig(CouchbaseSyncGateway.ADMIN_REST_API_PORT).iterator().next().toString();
+ String syncRestApiPort = entity.getConfig(CouchbaseSyncGateway.SYNC_REST_API_PORT).iterator().next().toString();
+
+ String serverWebAdminUrl = format("http://%s:%s@%s:%s", username, password, hostname, webPort);
+ String options = format("-url %s -bucket %s -adminInterface 0.0.0.0:%s -interface 0.0.0.0:%s -pool %s %s %s",
+ serverWebAdminUrl, bucketName, adminRestApiPort, syncRestApiPort, pool, pretty, verbose);
+
+ newScript(ImmutableMap.of("usePidFile", true), LAUNCHING)
+ .body.append(format("/opt/couchbase-sync-gateway/bin/sync_gateway %s ", options) + "> out.log 2> err.log < /dev/null &")
--- End diff --
This command looks like it will have launched a process. So should `stop` terminate that process? The docs seem to agree - http://developer.couchbase.com//mobile/develop/guides/sync-gateway/index.html says "runs an HTTP listener process". I didn't see any explicit stop CLI though. Maybe one just terminates it using the pid?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-53428152
can you add an example yaml including sync gateway? (and possibly also a downstream app)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701684
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
--- End diff --
Please add `// no-op` so folk know this is deliberate.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by Nakomis <gi...@git.apache.org>.
Github user Nakomis commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-48371211
I would normally do this, however a rebase tool that @ZaidM had used caused
a problem with the commit history, so there wasn't a sensible set of
commits that I could cherry-pick in to get what I needed, which is why I
did it this way
On 8 July 2014 17:28, Aled Sage <no...@github.com> wrote:
> @Nakomis <https://github.com/Nakomis> next time when you copy in @ZaidM
> <https://github.com/ZaidM> 's work, can you ensure it's his commit that
> appears in the history. For example, fetch his remote branch and then
> cherry-pick his commit.
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-48363923>
> .
>
--
Martin Harris
Lead Software Engineer
Cloudsoft Corporation Ltd
www.cloudsoftcorp.com
Mobile: +44 (0)7989 047-855
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-brooklyn/pull/25
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702346
--- Diff: software/nosql/src/test/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayEc2LiveTest.java ---
@@ -0,0 +1,114 @@
+package brooklyn.entity.nosql.couchbase;
+
+import java.util.List;
+import java.util.Map;
+
+import org.testng.annotations.Test;
+
+import brooklyn.entity.AbstractEc2LiveTest;
+import brooklyn.entity.group.DynamicCluster;
+import brooklyn.entity.proxying.EntitySpec;
+import brooklyn.entity.trait.Startable;
+import brooklyn.location.Location;
+import brooklyn.test.EntityTestUtils;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+
+@Test
+public class CouchbaseSyncGatewayEc2LiveTest extends AbstractEc2LiveTest {
+
+ @Override
+ protected void doTest(Location loc) throws Exception {
+ CouchbaseCluster cluster = app.createAndManageChild(EntitySpec.create(CouchbaseCluster.class)
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_USERNAME, "Administrator")
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD, "Password")
+ .configure(DynamicCluster.INITIAL_SIZE, 3)
+ .configure(CouchbaseCluster.CREATE_BUCKETS, (List<Map<String,Object>>)ImmutableList.of(
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "default",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11211
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "my_bucket",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11223
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "another",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11224
+ ))
+ )
+ );
+ CouchbaseSyncGateway gateway = app.createAndManageChild(EntitySpec.create(CouchbaseSyncGateway.class)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER, cluster)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET, "my_bucket")
+ );
+
+ app.start(ImmutableList.of(loc));
+
+ EntityTestUtils.assertAttributeEqualsEventually(gateway, Startable.SERVICE_UP, true);
+ }
+
+
+ // Supported operating systems
+
+ @Override
--- End diff --
If overriding, I think you need the test groups to be declared again. With the `@Test` annotation at the top, will these tests now be run as unit tests rather than just live tests?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by Nakomis <gi...@git.apache.org>.
Github user Nakomis commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14828879
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseNodeSshDriver.java ---
@@ -175,8 +201,83 @@ public void rebalance() {
.failOnNonZeroResultCode()
.execute();
entity.setAttribute(CouchbaseNode.REBALANCE_STATUS, "Rebalance Started");
+ // wait until the re-balance is complete
+ Repeater.create()
+ .every(Duration.millis(500))
+ .limitTimeTo(Duration.THIRTY_SECONDS)
+ .until(new Callable<Boolean>() {
+ @Override
+ public Boolean call() throws Exception {
+ for (String nodeHostName : CouchbaseNodeSshDriver.this.getNodeHostNames()) {
+ if (isNodeRebalancing(nodeHostName)) {
+ return true;
+ }
+ }
+ return false;
+ }
+ })
+ .run();
+ Repeater.create()
--- End diff --
The logic is reversed, so it until the node starts rebalancing, then waits for it to stop rebalancing
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14664590
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
+ //TODO: check for multiple bucket conflicts with port
+ List<Map<String, Object>> bucketsToCreate = getConfig(CREATE_BUCKETS);
+ Entity primaryNode = getPrimaryNode();
+
+ for (Map<String, Object> bucketMap : bucketsToCreate) {
+ String bucketName = bucketMap.containsKey("bucket") ? (String) bucketMap.get("bucket") : "default";
+ String bucketType = bucketMap.containsKey("bucket-type") ? (String) bucketMap.get("bucket-type") : "couchbase";
+ Integer bucketPort = bucketMap.containsKey("bucket-port") ? (Integer) bucketMap.get("bucket-port") : 11222;
+ Integer bucketRamSize = bucketMap.containsKey("bucket-ramsize") ? (Integer) bucketMap.get("bucket-ramsize") : 200;
+ Integer bucketReplica = bucketMap.containsKey("bucket-replica") ? (Integer) bucketMap.get("bucket-replica") : 1;
+
+ log.info("adding bucket: {} to primary node: {}", bucketName, primaryNode.getId());
+ createBucket(primaryNode, bucketName, bucketType, bucketPort, bucketRamSize, bucketReplica);
+ //TODO: add if bucket has been created.
+ }
+ }
+
+ public void createBucket(final Entity primaryNode, final String bucketName, final String bucketType, final Integer bucketPort, final Integer bucketRamSize, final Integer bucketReplica) {
+ DynamicTasks.queueIfPossible(TaskBuilder.<Void>builder().name("Creating bucket " + bucketName).body(
+ new Callable<Void>() {
+ @Override
+ public Void call() throws Exception {
+ DependentConfiguration.waitInTaskForAttributeReady(CouchbaseClusterImpl.this, CouchbaseCluster.BUCKET_CREATION_IN_PROGRESS, Predicates.equalTo(false));
+ if (CouchbaseClusterImpl.this.resetBucketCreation[0] != null) {
+ CouchbaseClusterImpl.this.resetBucketCreation[0].stop();
+ }
+ setAttribute(CouchbaseCluster.BUCKET_CREATION_IN_PROGRESS, true);
+
+ CouchbaseClusterImpl.this.resetBucketCreation[0] = HttpFeed.builder()
--- End diff --
An alternative to using the HttpFeed would be to use a `brooklyn.util.repeat.Repeater` to poll until success/fail.
It looks like `BUCKET_CREATION_IN_PROGRESS` is just used in this method, and unneccessarily by the caller of the createBuckets method (because it sequentially calls createBucket, which waits for bucket-creation to complete before returning).
Using the attribute to control synchronization feels dangerous - e.g. on brooklyn restart would anything reset it to `false`. Also, there is a race if two effectors execute concurrently - they both wait for it to be false, and then both set it to true. Better would be to use standard Java synchronization mechanisms in createBucket.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14664784
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseNodeSshDriver.java ---
@@ -175,8 +201,83 @@ public void rebalance() {
.failOnNonZeroResultCode()
.execute();
entity.setAttribute(CouchbaseNode.REBALANCE_STATUS, "Rebalance Started");
+ // wait until the re-balance is complete
+ Repeater.create()
+ .every(Duration.millis(500))
+ .limitTimeTo(Duration.THIRTY_SECONDS)
+ .until(new Callable<Boolean>() {
+ @Override
+ public Boolean call() throws Exception {
+ for (String nodeHostName : CouchbaseNodeSshDriver.this.getNodeHostNames()) {
+ if (isNodeRebalancing(nodeHostName)) {
+ return true;
+ }
+ }
+ return false;
+ }
+ })
+ .run();
+ Repeater.create()
--- End diff --
Is this code identical to that above, except for the different timeout?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by ZaidM <gi...@git.apache.org>.
Github user ZaidM commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14664057
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
--- End diff --
should be removed because this was fixed according to @Nakomis.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701665
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
--- End diff --
Does `stop` really not do anything? Please add a comment to say why.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701787
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
+ if (cbNode instanceof CouchbaseCluster) {
+ Optional<Entity> cbClusterNode = Iterables.tryFind(cbNode.getAttribute(CouchbaseCluster.GROUP_MEMBERS), new Predicate<Entity>() {
+
+ @Override
+ public boolean apply(@Nullable Entity entity) {
+ if (entity instanceof CouchbaseNode && Boolean.TRUE.equals(entity.getAttribute(CouchbaseNode.IS_IN_CLUSTER))) {
+ return true;
+ }
+ return false;
+ }
+ });
+ if (cbClusterNode.isPresent()) {
+ cbNode = cbClusterNode.get();
+ } else {
+ throw new IllegalArgumentException(format("The cluster %s does not contain any suitable Couchbase nodes to connect to..", cbNode.getId()));
+ }
+
+ }
+ String hostname = cbNode.getAttribute(CouchbaseNode.HOSTNAME);
+ String webPort = cbNode.getAttribute(CouchbaseNode.COUCHBASE_WEB_ADMIN_PORT).toString();
+
+
+ String username = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_USERNAME);
+ String password = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD);
+
+ String bucketName = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET);
+ String pool = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_POOL);
+ String pretty = entity.getConfig(CouchbaseSyncGateway.PRETTY) ? "-pretty" : "";
+ String verbose = entity.getConfig(CouchbaseSyncGateway.VERBOSE) ? "-verbose" : "";
+
+ String adminRestApiPort = entity.getConfig(CouchbaseSyncGateway.ADMIN_REST_API_PORT).iterator().next().toString();
--- End diff --
Instead use `entity.getAttribute(CouchbaseSyncGateway.ADMIN_REST_API_PORT)`.
If `ConfigToAttributes.apply(entity)` has previously been called (which it will have because this extends `SoftwareProcess`) then the attribute will have been set with the first free port in the range. However, it will be null if none of those were available.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14663754
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
+ //TODO: check for multiple bucket conflicts with port
--- End diff --
What do you mean by "with port"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702309
--- Diff: software/nosql/src/test/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayEc2LiveTest.java ---
@@ -0,0 +1,114 @@
+package brooklyn.entity.nosql.couchbase;
+
+import java.util.List;
+import java.util.Map;
+
+import org.testng.annotations.Test;
+
+import brooklyn.entity.AbstractEc2LiveTest;
+import brooklyn.entity.group.DynamicCluster;
+import brooklyn.entity.proxying.EntitySpec;
+import brooklyn.entity.trait.Startable;
+import brooklyn.location.Location;
+import brooklyn.test.EntityTestUtils;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+
+@Test
+public class CouchbaseSyncGatewayEc2LiveTest extends AbstractEc2LiveTest {
+
+ @Override
+ protected void doTest(Location loc) throws Exception {
+ CouchbaseCluster cluster = app.createAndManageChild(EntitySpec.create(CouchbaseCluster.class)
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_USERNAME, "Administrator")
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD, "Password")
+ .configure(DynamicCluster.INITIAL_SIZE, 3)
+ .configure(CouchbaseCluster.CREATE_BUCKETS, (List<Map<String,Object>>)ImmutableList.of(
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "default",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11211
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "my_bucket",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11223
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "another",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11224
+ ))
+ )
+ );
+ CouchbaseSyncGateway gateway = app.createAndManageChild(EntitySpec.create(CouchbaseSyncGateway.class)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER, cluster)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET, "my_bucket")
+ );
+
+ app.start(ImmutableList.of(loc));
+
+ EntityTestUtils.assertAttributeEqualsEventually(gateway, Startable.SERVICE_UP, true);
--- End diff --
Would be good to also check the buckets were created, if that's not too hard.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702145
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
+ if (cbNode instanceof CouchbaseCluster) {
+ Optional<Entity> cbClusterNode = Iterables.tryFind(cbNode.getAttribute(CouchbaseCluster.GROUP_MEMBERS), new Predicate<Entity>() {
+
+ @Override
+ public boolean apply(@Nullable Entity entity) {
+ if (entity instanceof CouchbaseNode && Boolean.TRUE.equals(entity.getAttribute(CouchbaseNode.IS_IN_CLUSTER))) {
+ return true;
+ }
+ return false;
+ }
+ });
+ if (cbClusterNode.isPresent()) {
+ cbNode = cbClusterNode.get();
+ } else {
+ throw new IllegalArgumentException(format("The cluster %s does not contain any suitable Couchbase nodes to connect to..", cbNode.getId()));
+ }
+
+ }
+ String hostname = cbNode.getAttribute(CouchbaseNode.HOSTNAME);
+ String webPort = cbNode.getAttribute(CouchbaseNode.COUCHBASE_WEB_ADMIN_PORT).toString();
+
+
+ String username = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_USERNAME);
+ String password = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD);
+
+ String bucketName = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET);
+ String pool = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_POOL);
+ String pretty = entity.getConfig(CouchbaseSyncGateway.PRETTY) ? "-pretty" : "";
+ String verbose = entity.getConfig(CouchbaseSyncGateway.VERBOSE) ? "-verbose" : "";
+
+ String adminRestApiPort = entity.getConfig(CouchbaseSyncGateway.ADMIN_REST_API_PORT).iterator().next().toString();
+ String syncRestApiPort = entity.getConfig(CouchbaseSyncGateway.SYNC_REST_API_PORT).iterator().next().toString();
+
+ String serverWebAdminUrl = format("http://%s:%s@%s:%s", username, password, hostname, webPort);
+ String options = format("-url %s -bucket %s -adminInterface 0.0.0.0:%s -interface 0.0.0.0:%s -pool %s %s %s",
+ serverWebAdminUrl, bucketName, adminRestApiPort, syncRestApiPort, pool, pretty, verbose);
+
+ newScript(ImmutableMap.of("usePidFile", true), LAUNCHING)
+ .body.append(format("/opt/couchbase-sync-gateway/bin/sync_gateway %s ", options) + "> out.log 2> err.log < /dev/null &")
+ .failOnNonZeroResultCode()
+ .execute();
+ }
+
+ @Override
+ public boolean isRunning() {
+ return newScript(MutableMap.of("usePidFile", true), CHECK_RUNNING).execute() == 0;
+ }
+
+ @Override
+ public void kill() {
+ newScript(MutableMap.of("usePidFile", true), KILLING).execute();
+ }
+
+ private List<String> installLinux(List<String> urls, String saveAs) {
+
+ String apt = chainGroup(
+ "which apt-get",
+ sudo("apt-get update"),
+ sudo(format("dpkg -i %s", saveAs)));
+
+ String yum = chainGroup(
+ "which yum",
+ sudo(format("rpm --install %s", saveAs)));
+
+ return ImmutableList.<String>builder()
+ .add(INSTALL_CURL)
+ .addAll(BashCommands.commandsToDownloadUrlsAs(urls, saveAs))
+ .add(alternatives(apt, yum))
--- End diff --
I thought our `BashCommands.installPackage(saveAs)` would do this? However, I don't see `dpkg` or `rpm` in that list (despite mention in the javadoc of `BashCommands.installExecutable`).
Longer term, would be good to get our `BashCommands.installPackage` to handle dpkg and rpm as well.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701324
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGateway.java ---
@@ -0,0 +1,61 @@
+package brooklyn.entity.nosql.couchbase;
+
+import brooklyn.config.ConfigKey;
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.ConfigKeys;
+import brooklyn.entity.basic.SoftwareProcess;
+import brooklyn.entity.proxying.ImplementedBy;
+import brooklyn.event.AttributeSensor;
+import brooklyn.event.basic.BasicAttributeSensorAndConfigKey;
+import brooklyn.event.basic.PortAttributeSensorAndConfigKey;
+import brooklyn.event.basic.Sensors;
+import brooklyn.util.flags.SetFromFlag;
+
+@ImplementedBy(CouchbaseSyncGatewayImpl.class)
+public interface CouchbaseSyncGateway extends SoftwareProcess {
+
+ @SetFromFlag("version")
+ ConfigKey<String> SUGGESTED_VERSION = ConfigKeys.newConfigKeyWithDefault(SoftwareProcess.SUGGESTED_VERSION,
+ "1.0-beta3.1");
+
+ @SetFromFlag("downloadUrl")
+ BasicAttributeSensorAndConfigKey<String> DOWNLOAD_URL = new BasicAttributeSensorAndConfigKey<String>(
+ SoftwareProcess.DOWNLOAD_URL, "http://packages.couchbase.com/releases/couchbase-sync-gateway/1.0-beta/couchbase-sync-gateway-community_${version}_${driver.osTag}");
+
+ @SetFromFlag("couchbaseServer")
+ ConfigKey<Entity> COUCHBASE_SERVER = ConfigKeys.newConfigKey(Entity.class, "couchbaseSyncGateway.couchbaseNode",
+ "Couchbase server node or cluster the sync gateway connects to");
+
+ @SetFromFlag("serverPool")
+ ConfigKey<String> COUCHBASE_SERVER_POOL = ConfigKeys.newStringConfigKey("couchbaseSyncGateway.serverPool",
+ "Couchbase Server pool name in which to find buckets", "default");
+
+ @SetFromFlag("couchbaseServerBucket")
+ ConfigKey<String> COUCHBASE_SERVER_BUCKET = ConfigKeys.newStringConfigKey("couchbaseSyncGateway.serverBucket",
+ "Name of the Couchbase bucket to use", "sync_gateway");
+
+ @SetFromFlag("couchbaseServerUrl")
+ ConfigKey<String> COUCHBASE_SERVER_URL = ConfigKeys.newStringConfigKey("couchbaseSyncGateway.couchbaseServerUrl",
--- End diff --
I'm tempted to say we should be using `ConfigKey<URI>` in places like this. But your code is following same pattern used elsewhere, so not suggesting you change it here. @grkvlt do you think we should start using `URI`? Changing old code will break backwards compatibility so hesitant to launch into that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14664735
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseNodeSshDriver.java ---
@@ -175,8 +201,83 @@ public void rebalance() {
.failOnNonZeroResultCode()
.execute();
entity.setAttribute(CouchbaseNode.REBALANCE_STATUS, "Rebalance Started");
+ // wait until the re-balance is complete
+ Repeater.create()
+ .every(Duration.millis(500))
+ .limitTimeTo(Duration.THIRTY_SECONDS)
+ .until(new Callable<Boolean>() {
+ @Override
+ public Boolean call() throws Exception {
+ for (String nodeHostName : CouchbaseNodeSshDriver.this.getNodeHostNames()) {
+ if (isNodeRebalancing(nodeHostName)) {
+ return true;
+ }
+ }
+ return false;
+ }
+ })
+ .run();
--- End diff --
This will return `false` if it timed out. You should check the result and at least log, rather than just continuing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701633
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayImpl.java ---
@@ -0,0 +1,66 @@
+package brooklyn.entity.nosql.couchbase;
+
+
+import brooklyn.config.render.RendererHints;
+import brooklyn.entity.basic.SoftwareProcessImpl;
+import brooklyn.event.feed.http.HttpFeed;
+import brooklyn.event.feed.http.HttpPollConfig;
+import brooklyn.event.feed.http.HttpValueFunctions;
+import brooklyn.location.access.BrooklynAccessUtils;
+
+import com.google.common.net.HostAndPort;
+
+public class CouchbaseSyncGatewayImpl extends SoftwareProcessImpl implements CouchbaseSyncGateway {
+
+ private HttpFeed httpFeed;
+
+ @Override
+ public Class<CouchbaseSyncGatewayDriver> getDriverInterface() {
+ return CouchbaseSyncGatewayDriver.class;
+ }
+
+ @Override
+ protected void connectSensors() {
+ super.connectSensors();
+ connectServiceUpIsRunning();
+ }
+
+ @Override
+ protected void connectServiceUpIsRunning() {
+
+
+ HostAndPort hp = BrooklynAccessUtils.getBrooklynAccessibleAddress(this,
+ getAttribute(CouchbaseSyncGateway.ADMIN_REST_API_PORT));
+
+ String managementUri = String.format("http://%s:%s",
+ hp.getHostText(), hp.getPort());
+
+ setAttribute(MANAGEMENT_URL, managementUri);
+
+ httpFeed = HttpFeed.builder()
+ .entity(this)
+ .period(200)
+ .baseUri(managementUri)
+ .poll(new HttpPollConfig<Boolean>(SERVICE_UP)
+ .onSuccess(HttpValueFunctions.responseCodeEquals(200)))
+ .build();
+
+ }
+
+ @Override
+ protected void disconnectSensors() {
+ super.disconnectSensors();
+ disconnectServiceUpIsRunning();
+ }
+
+ @Override
+ protected void disconnectServiceUpIsRunning() {
+ if (httpFeed != null) {
+ httpFeed.stop();
+ }
+ }
+
+ static {
+ RendererHints.register(MANAGEMENT_URL, new RendererHints.NamedActionWithUrl("Open"));
--- End diff --
If the `MANAGEMENT_URL` were of type `URL` or `URI` then we could do this rendering automatically. That's a pretty good reason for using the stronger typing, rather than Strings everywhere. @grkvlt i presume you agree? Is it easy to add the rendering hint automatically for any attribute of type URI/URL?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701547
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayImpl.java ---
@@ -0,0 +1,66 @@
+package brooklyn.entity.nosql.couchbase;
+
+
+import brooklyn.config.render.RendererHints;
+import brooklyn.entity.basic.SoftwareProcessImpl;
+import brooklyn.event.feed.http.HttpFeed;
+import brooklyn.event.feed.http.HttpPollConfig;
+import brooklyn.event.feed.http.HttpValueFunctions;
+import brooklyn.location.access.BrooklynAccessUtils;
+
+import com.google.common.net.HostAndPort;
+
+public class CouchbaseSyncGatewayImpl extends SoftwareProcessImpl implements CouchbaseSyncGateway {
+
+ private HttpFeed httpFeed;
+
+ @Override
+ public Class<CouchbaseSyncGatewayDriver> getDriverInterface() {
+ return CouchbaseSyncGatewayDriver.class;
+ }
+
+ @Override
+ protected void connectSensors() {
+ super.connectSensors();
+ connectServiceUpIsRunning();
+ }
+
+ @Override
+ protected void connectServiceUpIsRunning() {
+
+
+ HostAndPort hp = BrooklynAccessUtils.getBrooklynAccessibleAddress(this,
+ getAttribute(CouchbaseSyncGateway.ADMIN_REST_API_PORT));
+
+ String managementUri = String.format("http://%s:%s",
+ hp.getHostText(), hp.getPort());
+
+ setAttribute(MANAGEMENT_URL, managementUri);
+
+ httpFeed = HttpFeed.builder()
+ .entity(this)
+ .period(200)
+ .baseUri(managementUri)
+ .poll(new HttpPollConfig<Boolean>(SERVICE_UP)
+ .onSuccess(HttpValueFunctions.responseCodeEquals(200)))
--- End diff --
Do we also need `.onFailureOrException(Functions.constant(false)))`? I'm not sure off-hand.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by ZaidM <gi...@git.apache.org>.
Github user ZaidM commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14663863
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
+ //TODO: check for multiple bucket conflicts with port
--- End diff --
the bucket port to be specified in couchbase. It is part of initialising a bucket.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by fatuhoku <gi...@git.apache.org>.
Github user fatuhoku commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-68060279
Any updates for the YAML example?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701180
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
+ if (cbNode instanceof CouchbaseCluster) {
+ Optional<Entity> cbClusterNode = Iterables.tryFind(cbNode.getAttribute(CouchbaseCluster.GROUP_MEMBERS), new Predicate<Entity>() {
--- End diff --
Could use `Predicates.and(Predicates.instanceOf(CouchbaseNode.class), EntityPredicates.attributeEqualTo(CouchbaseNode.IS_IN_CLUSTER, true)`, rather than declaring a new predicate.
For general background reading, see https://code.google.com/p/guava-libraries/wiki/FunctionalExplained#Caveats and also the discussion at https://groups.google.com/forum/#!msg/jclouds-dev/f-3QY6EKum4/UVyjHTjT8WIJ (in particular Adrian Cole's "summarize my view" e-mail and Tim Peierls' reply).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14663634
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -24,23 +26,36 @@
import brooklyn.event.AttributeSensor;
import brooklyn.event.SensorEvent;
import brooklyn.event.SensorEventListener;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.event.feed.http.HttpFeed;
+import brooklyn.event.feed.http.HttpPollConfig;
+import brooklyn.event.feed.http.HttpValueFunctions;
+import brooklyn.event.feed.http.JsonFunctions;
import brooklyn.location.Location;
import brooklyn.policy.PolicySpec;
import brooklyn.util.collections.MutableSet;
+import brooklyn.util.guava.Functionals;
+import brooklyn.util.task.DynamicTasks;
+import brooklyn.util.task.TaskBuilder;
import brooklyn.util.task.Tasks;
import brooklyn.util.text.ByteSizeStrings;
+import brooklyn.util.text.Strings;
import brooklyn.util.time.Time;
import com.google.common.base.Function;
import com.google.common.base.Optional;
import com.google.common.base.Preconditions;
+import com.google.common.base.Predicates;
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
+import com.google.gson.JsonArray;
+import com.google.gson.JsonElement;
public class CouchbaseClusterImpl extends DynamicClusterImpl implements CouchbaseCluster {
private static final Logger log = LoggerFactory.getLogger(CouchbaseClusterImpl.class);
private final Object mutex = new Object[0];
+ private final HttpFeed[] resetBucketCreation = new HttpFeed[]{null};
--- End diff --
Why an array of size one? It looks like you're using it like an `AtomicReference<HttpFeed>` with get/set calls.
This field deserves a few lines of comments for what it's for - presumably it's allowing one to tell when the bucket creation has completed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-48363923
@Nakomis next time when you copy in @ZaidM 's work, can you ensure it's his commit that appears in the history. For example, fetch his remote branch and then cherry-pick his commit.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14663893
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
+ //TODO: check for multiple bucket conflicts with port
+ List<Map<String, Object>> bucketsToCreate = getConfig(CREATE_BUCKETS);
+ Entity primaryNode = getPrimaryNode();
+
+ for (Map<String, Object> bucketMap : bucketsToCreate) {
+ String bucketName = bucketMap.containsKey("bucket") ? (String) bucketMap.get("bucket") : "default";
+ String bucketType = bucketMap.containsKey("bucket-type") ? (String) bucketMap.get("bucket-type") : "couchbase";
+ Integer bucketPort = bucketMap.containsKey("bucket-port") ? (Integer) bucketMap.get("bucket-port") : 11222;
+ Integer bucketRamSize = bucketMap.containsKey("bucket-ramsize") ? (Integer) bucketMap.get("bucket-ramsize") : 200;
+ Integer bucketReplica = bucketMap.containsKey("bucket-replica") ? (Integer) bucketMap.get("bucket-replica") : 1;
+
+ log.info("adding bucket: {} to primary node: {}", bucketName, primaryNode.getId());
+ createBucket(primaryNode, bucketName, bucketType, bucketPort, bucketRamSize, bucketReplica);
+ //TODO: add if bucket has been created.
--- End diff --
What does this `TODO` mean?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#issuecomment-53839726
i have some fixes for this but they now depend on #131 so i'll wait for that to be merged then also merge this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702401
--- Diff: software/nosql/src/test/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayEc2LiveTest.java ---
@@ -0,0 +1,114 @@
+package brooklyn.entity.nosql.couchbase;
+
+import java.util.List;
+import java.util.Map;
+
+import org.testng.annotations.Test;
+
+import brooklyn.entity.AbstractEc2LiveTest;
+import brooklyn.entity.group.DynamicCluster;
+import brooklyn.entity.proxying.EntitySpec;
+import brooklyn.entity.trait.Startable;
+import brooklyn.location.Location;
+import brooklyn.test.EntityTestUtils;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+
+@Test
+public class CouchbaseSyncGatewayEc2LiveTest extends AbstractEc2LiveTest {
+
+ @Override
+ protected void doTest(Location loc) throws Exception {
+ CouchbaseCluster cluster = app.createAndManageChild(EntitySpec.create(CouchbaseCluster.class)
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_USERNAME, "Administrator")
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD, "Password")
+ .configure(DynamicCluster.INITIAL_SIZE, 3)
+ .configure(CouchbaseCluster.CREATE_BUCKETS, (List<Map<String,Object>>)ImmutableList.of(
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "default",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11211
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "my_bucket",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11223
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "another",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11224
+ ))
+ )
+ );
+ CouchbaseSyncGateway gateway = app.createAndManageChild(EntitySpec.create(CouchbaseSyncGateway.class)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER, cluster)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET, "my_bucket")
+ );
+
+ app.start(ImmutableList.of(loc));
+
+ EntityTestUtils.assertAttributeEqualsEventually(gateway, Startable.SERVICE_UP, true);
+ }
+
+
+ // Supported operating systems
+
+ @Override
+ public void test_Ubuntu_12_0() throws Exception {
+ super.test_Ubuntu_12_0();
+ }
+
+ @Override
+ public void test_Red_Hat_Enterprise_Linux_6() throws Exception {
+ super.test_Red_Hat_Enterprise_Linux_6();
+ }
+
+ @Override
+ public void test_CentOS_6_3() throws Exception {
+ super.test_CentOS_6_3();
+ }
+
+ // Unsupported operating systems
+
+ @Override
+ public void test_CentOS_5_6() throws Exception {
+ // Unsupported
+ // error: Failed dependencies:
+ // libc.so.6(GLIBC_2.7)(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libcrypto.so.10()(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libreadline.so.6()(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libssl.so.10()(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libstdc++.so.6(GLIBCXX_3.4.10)(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libstdc++.so.6(GLIBCXX_3.4.11)(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libstdc++.so.6(GLIBCXX_3.4.9)(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // libtinfo.so.5()(64bit) is needed by couchbase-server-2.5.1-1083.x86_64
+ // openssl >= 1.0.0 is needed by couchbase-server-2.5.1-1083.x86_64
+ // rpmlib(FileDigests) <= 4.6.0-1 is needed by couchbase-server-2.5.1-1083.x86_64
+ // rpmlib(PayloadIsXz) <= 5.2-1 is needed by couchbase-server-2.5.1-1083.x86_64
+ }
+
+ @Override
+ public void test_Debian_6() throws Exception {
+ // Unsupported
--- End diff --
Mark with `@Test(enabled=false)` so clear that it didn't run, rather than the test reporting success.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by ZaidM <gi...@git.apache.org>.
Github user ZaidM commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14664169
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseClusterImpl.java ---
@@ -315,6 +341,81 @@ public boolean isMemberInCluster(Entity e) {
return Optional.fromNullable(e.getAttribute(CouchbaseNode.IS_IN_CLUSTER)).or(false);
}
+ public void createBuckets() {
+ //FIXME: multiple buckets require synchronization/wait time (checks for port conflicts and exceeding ram size)
+ //TODO: check for multiple bucket conflicts with port
+ List<Map<String, Object>> bucketsToCreate = getConfig(CREATE_BUCKETS);
+ Entity primaryNode = getPrimaryNode();
+
+ for (Map<String, Object> bucketMap : bucketsToCreate) {
+ String bucketName = bucketMap.containsKey("bucket") ? (String) bucketMap.get("bucket") : "default";
+ String bucketType = bucketMap.containsKey("bucket-type") ? (String) bucketMap.get("bucket-type") : "couchbase";
+ Integer bucketPort = bucketMap.containsKey("bucket-port") ? (Integer) bucketMap.get("bucket-port") : 11222;
+ Integer bucketRamSize = bucketMap.containsKey("bucket-ramsize") ? (Integer) bucketMap.get("bucket-ramsize") : 200;
+ Integer bucketReplica = bucketMap.containsKey("bucket-replica") ? (Integer) bucketMap.get("bucket-replica") : 1;
+
+ log.info("adding bucket: {} to primary node: {}", bucketName, primaryNode.getId());
+ createBucket(primaryNode, bucketName, bucketType, bucketPort, bucketRamSize, bucketReplica);
+ //TODO: add if bucket has been created.
--- End diff --
probably to be added to a list of buckets that are online/active.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702387
--- Diff: software/nosql/src/test/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayEc2LiveTest.java ---
@@ -0,0 +1,114 @@
+package brooklyn.entity.nosql.couchbase;
+
+import java.util.List;
+import java.util.Map;
+
+import org.testng.annotations.Test;
+
+import brooklyn.entity.AbstractEc2LiveTest;
+import brooklyn.entity.group.DynamicCluster;
+import brooklyn.entity.proxying.EntitySpec;
+import brooklyn.entity.trait.Startable;
+import brooklyn.location.Location;
+import brooklyn.test.EntityTestUtils;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+
+@Test
+public class CouchbaseSyncGatewayEc2LiveTest extends AbstractEc2LiveTest {
+
+ @Override
+ protected void doTest(Location loc) throws Exception {
+ CouchbaseCluster cluster = app.createAndManageChild(EntitySpec.create(CouchbaseCluster.class)
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_USERNAME, "Administrator")
+ .configure(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD, "Password")
+ .configure(DynamicCluster.INITIAL_SIZE, 3)
+ .configure(CouchbaseCluster.CREATE_BUCKETS, (List<Map<String,Object>>)ImmutableList.of(
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "default",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11211
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "my_bucket",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11223
+ ),
+ (Map<String,Object>)ImmutableMap.<String, Object>of(
+ "bucket", "another",
+ "bucket-ramsize", 100,
+ "bucket-type", "couchbase",
+ "bucket-port", 11224
+ ))
+ )
+ );
+ CouchbaseSyncGateway gateway = app.createAndManageChild(EntitySpec.create(CouchbaseSyncGateway.class)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER, cluster)
+ .configure(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET, "my_bucket")
+ );
+
+ app.start(ImmutableList.of(loc));
+
+ EntityTestUtils.assertAttributeEqualsEventually(gateway, Startable.SERVICE_UP, true);
+ }
+
+
+ // Supported operating systems
+
+ @Override
+ public void test_Ubuntu_12_0() throws Exception {
+ super.test_Ubuntu_12_0();
+ }
+
+ @Override
+ public void test_Red_Hat_Enterprise_Linux_6() throws Exception {
+ super.test_Red_Hat_Enterprise_Linux_6();
+ }
+
+ @Override
+ public void test_CentOS_6_3() throws Exception {
+ super.test_CentOS_6_3();
+ }
+
+ // Unsupported operating systems
+
+ @Override
+ public void test_CentOS_5_6() throws Exception {
+ // Unsupported
--- End diff --
I like to include a `TODO` in comments like this.
I wonder if you use `apt-get` instead of `dpkg` whether this problem would go away.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14664973
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseNodeSshDriver.java ---
@@ -175,8 +201,83 @@ public void rebalance() {
.failOnNonZeroResultCode()
.execute();
entity.setAttribute(CouchbaseNode.REBALANCE_STATUS, "Rebalance Started");
+ // wait until the re-balance is complete
+ Repeater.create()
+ .every(Duration.millis(500))
+ .limitTimeTo(Duration.THIRTY_SECONDS)
+ .until(new Callable<Boolean>() {
+ @Override
+ public Boolean call() throws Exception {
+ for (String nodeHostName : CouchbaseNodeSshDriver.this.getNodeHostNames()) {
+ if (isNodeRebalancing(nodeHostName)) {
+ return true;
+ }
+ }
+ return false;
+ }
+ })
+ .run();
+ Repeater.create()
+ .every(Duration.FIVE_SECONDS)
+ .limitTimeTo(Duration.FIVE_MINUTES)
+ .until(new Callable<Boolean>() {
+ @Override
+ public Boolean call() throws Exception {
+ for (String nodeHostName : CouchbaseNodeSshDriver.this.getNodeHostNames()) {
+ if (isNodeRebalancing(nodeHostName)) {
+ return false;
+ }
+ }
+ return true;
+ }
+ })
+ .run();
+ log.info("rebalanced cluster via primary node {}", getEntity());
}
+ private Iterable<String> getNodeHostNames() throws URISyntaxException {
+ Function<JsonElement, Iterable<String>> getNodesAsList = new Function<JsonElement, Iterable<String>>() {
+ @Override public Iterable<String> apply(JsonElement input) {
+ if (input == null) {
+ return Collections.emptyList();
+ }
+ Collection<String> names = Lists.newArrayList();
+ JsonArray nodes = input.getAsJsonArray();
+ for (JsonElement element : nodes) {
+ // NOTE: the 'hostname' element also includes the port
+ names.add(element.getAsJsonObject().get("hostname").toString().replace("\"", ""));
+ }
+ return names;
+ }
+ };
+ HttpToolResponse nodesResponse = getAPIResponse(String.format("http://%s:%s/pools/nodes", getHostname(), getWebPort()));
+ return Functionals.chain(
+ HttpValueFunctions.jsonContents(),
+ JsonFunctions.walkN("nodes"),
+ getNodesAsList
+ ).apply(nodesResponse);
+ }
+
+ private boolean isNodeRebalancing(String nodeHostName) throws URISyntaxException {
+ HttpToolResponse response = getAPIResponse("http://" + nodeHostName + "/pools/nodes/rebalanceProgress");
+ if (response.getResponseCode() != 200) {
+ throw new IllegalStateException("failed to rebalance cluster: " + response);
+ }
+ return !HttpValueFunctions.jsonContents("status", String.class).apply(response).equals("none");
+ }
+
+ private HttpToolResponse getAPIResponse(String path) throws URISyntaxException {
--- End diff --
Instead of `path`, perhaps call it `uri` or some such. Path mean a particular part of the uri: http://docs.oracle.com/javase/7/docs/api/java/net/URI.html#getPath()
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702266
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
+ if (cbNode instanceof CouchbaseCluster) {
+ Optional<Entity> cbClusterNode = Iterables.tryFind(cbNode.getAttribute(CouchbaseCluster.GROUP_MEMBERS), new Predicate<Entity>() {
+
+ @Override
+ public boolean apply(@Nullable Entity entity) {
+ if (entity instanceof CouchbaseNode && Boolean.TRUE.equals(entity.getAttribute(CouchbaseNode.IS_IN_CLUSTER))) {
+ return true;
+ }
+ return false;
+ }
+ });
+ if (cbClusterNode.isPresent()) {
+ cbNode = cbClusterNode.get();
+ } else {
+ throw new IllegalArgumentException(format("The cluster %s does not contain any suitable Couchbase nodes to connect to..", cbNode.getId()));
+ }
+
+ }
+ String hostname = cbNode.getAttribute(CouchbaseNode.HOSTNAME);
+ String webPort = cbNode.getAttribute(CouchbaseNode.COUCHBASE_WEB_ADMIN_PORT).toString();
+
+
+ String username = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_USERNAME);
+ String password = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD);
+
+ String bucketName = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET);
+ String pool = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_POOL);
+ String pretty = entity.getConfig(CouchbaseSyncGateway.PRETTY) ? "-pretty" : "";
+ String verbose = entity.getConfig(CouchbaseSyncGateway.VERBOSE) ? "-verbose" : "";
+
+ String adminRestApiPort = entity.getConfig(CouchbaseSyncGateway.ADMIN_REST_API_PORT).iterator().next().toString();
+ String syncRestApiPort = entity.getConfig(CouchbaseSyncGateway.SYNC_REST_API_PORT).iterator().next().toString();
+
+ String serverWebAdminUrl = format("http://%s:%s@%s:%s", username, password, hostname, webPort);
+ String options = format("-url %s -bucket %s -adminInterface 0.0.0.0:%s -interface 0.0.0.0:%s -pool %s %s %s",
+ serverWebAdminUrl, bucketName, adminRestApiPort, syncRestApiPort, pool, pretty, verbose);
+
+ newScript(ImmutableMap.of("usePidFile", true), LAUNCHING)
+ .body.append(format("/opt/couchbase-sync-gateway/bin/sync_gateway %s ", options) + "> out.log 2> err.log < /dev/null &")
+ .failOnNonZeroResultCode()
+ .execute();
+ }
+
+ @Override
+ public boolean isRunning() {
+ return newScript(MutableMap.of("usePidFile", true), CHECK_RUNNING).execute() == 0;
+ }
+
+ @Override
+ public void kill() {
+ newScript(MutableMap.of("usePidFile", true), KILLING).execute();
+ }
+
+ private List<String> installLinux(List<String> urls, String saveAs) {
+
+ String apt = chainGroup(
+ "which apt-get",
+ sudo("apt-get update"),
+ sudo(format("dpkg -i %s", saveAs)));
+
+ String yum = chainGroup(
+ "which yum",
+ sudo(format("rpm --install %s", saveAs)));
--- End diff --
why use `rpm` after doing `which yum`? Looks like it should either be `which rpm` or use `yum` to install.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14700880
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
--- End diff --
This will never return if the cbNode is a `CouchbaseNode` rather than a cluster - but the description for `CouchbaseSyncGateway.COUCHBASE_SERVER` says it can be either.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14702239
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,158 @@
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
+ if (cbNode instanceof CouchbaseCluster) {
+ Optional<Entity> cbClusterNode = Iterables.tryFind(cbNode.getAttribute(CouchbaseCluster.GROUP_MEMBERS), new Predicate<Entity>() {
+
+ @Override
+ public boolean apply(@Nullable Entity entity) {
+ if (entity instanceof CouchbaseNode && Boolean.TRUE.equals(entity.getAttribute(CouchbaseNode.IS_IN_CLUSTER))) {
+ return true;
+ }
+ return false;
+ }
+ });
+ if (cbClusterNode.isPresent()) {
+ cbNode = cbClusterNode.get();
+ } else {
+ throw new IllegalArgumentException(format("The cluster %s does not contain any suitable Couchbase nodes to connect to..", cbNode.getId()));
+ }
+
+ }
+ String hostname = cbNode.getAttribute(CouchbaseNode.HOSTNAME);
+ String webPort = cbNode.getAttribute(CouchbaseNode.COUCHBASE_WEB_ADMIN_PORT).toString();
+
+
+ String username = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_USERNAME);
+ String password = cbNode.getConfig(CouchbaseNode.COUCHBASE_ADMIN_PASSWORD);
+
+ String bucketName = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_BUCKET);
+ String pool = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER_POOL);
+ String pretty = entity.getConfig(CouchbaseSyncGateway.PRETTY) ? "-pretty" : "";
+ String verbose = entity.getConfig(CouchbaseSyncGateway.VERBOSE) ? "-verbose" : "";
+
+ String adminRestApiPort = entity.getConfig(CouchbaseSyncGateway.ADMIN_REST_API_PORT).iterator().next().toString();
+ String syncRestApiPort = entity.getConfig(CouchbaseSyncGateway.SYNC_REST_API_PORT).iterator().next().toString();
+
+ String serverWebAdminUrl = format("http://%s:%s@%s:%s", username, password, hostname, webPort);
+ String options = format("-url %s -bucket %s -adminInterface 0.0.0.0:%s -interface 0.0.0.0:%s -pool %s %s %s",
+ serverWebAdminUrl, bucketName, adminRestApiPort, syncRestApiPort, pool, pretty, verbose);
+
+ newScript(ImmutableMap.of("usePidFile", true), LAUNCHING)
+ .body.append(format("/opt/couchbase-sync-gateway/bin/sync_gateway %s ", options) + "> out.log 2> err.log < /dev/null &")
+ .failOnNonZeroResultCode()
+ .execute();
+ }
+
+ @Override
+ public boolean isRunning() {
+ return newScript(MutableMap.of("usePidFile", true), CHECK_RUNNING).execute() == 0;
+ }
+
+ @Override
+ public void kill() {
+ newScript(MutableMap.of("usePidFile", true), KILLING).execute();
+ }
+
+ private List<String> installLinux(List<String> urls, String saveAs) {
+
+ String apt = chainGroup(
+ "which apt-get",
+ sudo("apt-get update"),
+ sudo(format("dpkg -i %s", saveAs)));
--- End diff --
Why not use `apt-get` to install? With `dpkg`, it won't install the required dependencies. See http://askubuntu.com/questions/309113/what-is-the-difference-between-dpkg-and-aptitude-apt-get
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by aledsage <gi...@git.apache.org>.
Github user aledsage commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r14701474
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewayImpl.java ---
@@ -0,0 +1,66 @@
+package brooklyn.entity.nosql.couchbase;
+
+
+import brooklyn.config.render.RendererHints;
+import brooklyn.entity.basic.SoftwareProcessImpl;
+import brooklyn.event.feed.http.HttpFeed;
+import brooklyn.event.feed.http.HttpPollConfig;
+import brooklyn.event.feed.http.HttpValueFunctions;
+import brooklyn.location.access.BrooklynAccessUtils;
+
+import com.google.common.net.HostAndPort;
+
+public class CouchbaseSyncGatewayImpl extends SoftwareProcessImpl implements CouchbaseSyncGateway {
+
+ private HttpFeed httpFeed;
+
+ @Override
+ public Class<CouchbaseSyncGatewayDriver> getDriverInterface() {
+ return CouchbaseSyncGatewayDriver.class;
+ }
+
+ @Override
+ protected void connectSensors() {
+ super.connectSensors();
+ connectServiceUpIsRunning();
+ }
+
+ @Override
+ protected void connectServiceUpIsRunning() {
+
--- End diff --
Delete blank lines
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-brooklyn pull request: Couchbase sync gateway
Posted by ahgittin <gi...@git.apache.org>.
Github user ahgittin commented on a diff in the pull request:
https://github.com/apache/incubator-brooklyn/pull/25#discussion_r16716038
--- Diff: software/nosql/src/main/java/brooklyn/entity/nosql/couchbase/CouchbaseSyncGatewaySshDriver.java ---
@@ -0,0 +1,176 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package brooklyn.entity.nosql.couchbase;
+
+import static brooklyn.util.ssh.BashCommands.INSTALL_CURL;
+import static brooklyn.util.ssh.BashCommands.alternatives;
+import static brooklyn.util.ssh.BashCommands.chainGroup;
+import static brooklyn.util.ssh.BashCommands.sudo;
+import static java.lang.String.format;
+
+import java.util.List;
+
+import javax.annotation.Nullable;
+
+import brooklyn.entity.Entity;
+import brooklyn.entity.basic.AbstractSoftwareProcessSshDriver;
+import brooklyn.entity.basic.Entities;
+import brooklyn.entity.basic.EntityLocal;
+import brooklyn.entity.drivers.downloads.DownloadResolver;
+import brooklyn.event.basic.DependentConfiguration;
+import brooklyn.location.OsDetails;
+import brooklyn.location.basic.SshMachineLocation;
+import brooklyn.util.collections.MutableMap;
+import brooklyn.util.ssh.BashCommands;
+import brooklyn.util.time.Duration;
+import brooklyn.util.time.Time;
+
+import com.google.common.base.Optional;
+import com.google.common.base.Predicate;
+import com.google.common.base.Predicates;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.Iterables;
+
+public class CouchbaseSyncGatewaySshDriver extends AbstractSoftwareProcessSshDriver implements CouchbaseSyncGatewayDriver {
+ public CouchbaseSyncGatewaySshDriver(EntityLocal entity, SshMachineLocation machine) {
+ super(entity, machine);
+ }
+
+ @Override
+ public void stop() {
+
+ }
+
+ @Override
+ public void install() {
+ //reference http://docs.couchbase.com/sync-gateway/#getting-started-with-sync-gateway
+ DownloadResolver resolver = Entities.newDownloader(this);
+ List<String> urls = resolver.getTargets();
+ String saveAs = resolver.getFilename();
+
+ OsDetails osDetails = getMachine().getMachineDetails().getOsDetails();
+
+ log.info("Installing couchbase-sync-gateway version: {}", getVersion());
+ if (osDetails.isLinux()) {
+ List<String> commands = installLinux(urls, saveAs);
+ newScript(INSTALLING)
+ .body.append(commands).execute();
+ }
+ }
+
+ @Override
+ public void customize() {
+
+ }
+
+ @Override
+ public void launch() {
+ Entity cbNode = entity.getConfig(CouchbaseSyncGateway.COUCHBASE_SERVER);
+ Entities.waitForServiceUp(cbNode, Duration.ONE_HOUR);
+ DependentConfiguration.waitInTaskForAttributeReady(cbNode, CouchbaseCluster.IS_CLUSTER_INITIALIZED, Predicates.equalTo(true));
+ // Even once the bucket has published its API URL, it can still take a couple of seconds for it to become available
+ Time.sleep(10 * 1000);
--- End diff --
can do `Time.sleep(Duration.TEN_SECONDS)`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---