You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Adrian Cole (JIRA)" <ji...@apache.org> on 2011/08/13 11:54:27 UTC
[jira] [Created] (WHIRR-361) refactor jclouds dependencies
refactor jclouds dependencies
-----------------------------
Key: WHIRR-361
URL: https://issues.apache.org/jira/browse/WHIRR-361
Project: Whirr
Issue Type: Improvement
Components: core
Affects Versions: 0.6.0
Reporter: Adrian Cole
Assignee: Adrian Cole
Fix For: 0.6.0
There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
* in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
* we've switched to SLF4J, yet haven't configured jclouds to use it
* especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086472#comment-13086472 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
@tom is this a WARN that happens once or repeatedly? does the actual maven execution fail, or are you referring to connection refused notices. Are you using aws-ec2 with the default properties in the project or overrides?
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-361:
------------------------------
Attachment: WHIRR-361.patch
I've created a patch from the changes Adrian suggested. Tested with ZK on EC2. I'm going to test HBase.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085346#comment-13085346 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
guys, would be easier to troubleshoot if we paste/attach stack traces and note the specific test cases or otherwise that fail w/parameters
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085530#comment-13085530 ]
Tom White commented on WHIRR-361:
---------------------------------
I can confirm that this fixes ZK launch. However, destroy doesn't work for the same reason, since DestroyClusterAction uses group names to select which instances to shutdown, so only the first in each group is selected. Not sure there's a workaround for this.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085429#comment-13085429 ]
Andrei Savu commented on WHIRR-361:
-----------------------------------
The same is true for CDH. All integration tests pass on Rackspace.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-361:
------------------------------
Priority: Blocker (was: Major)
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Attachment: WHIRR-361.patch
I've tested this on aws-ec2 for cassandra and zookeeper using the jclouds 1.1.1-SNAPSHOT version, which is the same as 1.1.0 except including the following fix: http://code.google.com/p/jclouds/issues/detail?id=660
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Attachment: WHIRR-361.patch
patch includes a parser fix for jclouds 1.1.0
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085528#comment-13085528 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
@tom thanks for the pointer.. I was looking for a .log file, and missed the checkstyle-result.xml!
@all thanks for testing
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085382#comment-13085382 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
HBase errors are related to a classpath conflict.
com.google.common.collect.ImmutableSet.copyOf(Ljava/util/Collection;) is in Guava, but not google-collections which it replaced some time back.
funny quote from google-collections homepage: "What you see here is ancient and unmaintained. Do not use it."
I'll hunt down the jar that has this in it.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085495#comment-13085495 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
Please figure out a way to get the attached ConfigureClusterAction passing checkstyle, as it took a while to refine, but maven isn't telling me what is wrong with it.
ok. logic is failing at the following redundant check:
// Check it's the correct cluster
if (!clusterSpec.getClusterName().equals(nodeMetadata.getGroup())) {
return false;
}
we are already know it is in the cluster from the latter check on id (taken from cluster map), so the above code is redundant.
For some reason in aws-ec2, the security group name isn't parsing properly which is why that check is failing. I'd recommend applying the attached version of ConfigureClusterAction, taking out the redundant cluster name check so that aws-ec2 can work without a patch on jclouds.
I've verified the solution I'm recommending works, but please do re-check. Also note, I am using the patch in WHIRR-341 so that I'm testing against a stable ubuntu ami.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-361:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I've just committed this. Nice cleanup!
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086444#comment-13086444 ]
Andrei Savu commented on WHIRR-361:
-----------------------------------
looks good to me. integration tests pass for zookeeper and all instances are terminated as expected.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085408#comment-13085408 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
whirr uses guava directly and also indirectly. since we use it directly, we should put it as a dependency in compile scope with a version that doesn't conflict with our dependencies. currently, r09 is best. if we explicitly call this out, then changing unrelated dependencies won't cause weird issues like this.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086627#comment-13086627 ]
Tom White commented on WHIRR-361:
---------------------------------
The test passed for me in the us-west-1 region (set using {{whirr.location-id=us-west-1}} in the property file).
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085310#comment-13085310 ]
Tom White commented on WHIRR-361:
---------------------------------
Are you able to run Hadoop and ZK integration tests? They are failing for me since the nodes are unable to connect to each other on the private addresses. Not sure what caused that.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085511#comment-13085511 ]
Tom White commented on WHIRR-361:
---------------------------------
The checkstyle problem is reported in core/target/checkstyle-result.xml. It's complaining about the format of "nodeId" - it should be "NODE_ID".
Running tests now.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086761#comment-13086761 ]
Andrei Savu commented on WHIRR-361:
-----------------------------------
Tested again and it seems like the integration tests for ZooKeeper & Cassandra pass as expected on ec2 us-east-1. I believe we experienced a brief service glitch today. +1 for committing WHIRR-341 & latest patch for WHIRR-361.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Attachment: WHIRR-361.patch
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085860#comment-13085860 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
jclouds is going to push out an emergency release for 1.1.1. I suggest we block on this as opposed to coding in the parser fix. The parser that is referenced is likely to change in jclouds 1.2.0, so packaging in a patch would make whirr 0.6.0 incompatible with jclouds 1.2.0.
http://groups.google.com/group/jclouds-dev/browse_thread/thread/74150d764cc70a02
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086779#comment-13086779 ]
Andrei Savu commented on WHIRR-361:
-----------------------------------
I've just committed the last patch.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085471#comment-13085471 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
testing zookeeper via mvn -Pintegration clean install -DargLine="-Dwhirr.test.provider=cloudservers-us ... works fine
testing with -Dwhirr.test.provider=aws-ec2 and patch for WHIRR-341 which uses ami-63be790a has the following behavior:
- Both nodes start and install zookeeper. However, it appears only one node seems to have scripts run on ConfigureClusterAction.doAction, so only one starts zookeeper in this case. This is visible in target/test-data/jclouds-compute.log and whirr.log
This is completely repeatable, but I haven't isolated why this is occurring
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085524#comment-13085524 ]
Tom White commented on WHIRR-361:
---------------------------------
I'm getting the same - nodeMetadata.getGroup() is returning null for the second node in the cluster.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu reopened WHIRR-361:
-------------------------------
I'm reopening this issue because on the current trunk I'm unable to run the HBase & CDH integration tests. I'm seeing a long list of Google Guice ProvisionException's (probably a classpath issue).
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086470#comment-13086470 ]
Tom White commented on WHIRR-361:
---------------------------------
I tried the latest patch and a ZK cluster starts OK (I get 'imok' back from each server), even though the integration test still fails in the same way as before. Since it works for others, this is not a blocker for release, although it would be nice to find out what's going on here.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Attachment: ConfigureClusterAction.java
much more useful logging, although I cannot find a way to get checkstyle to tell me what is wrong with the formatting
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085859#comment-13085859 ]
Tom White commented on WHIRR-361:
---------------------------------
Thanks for the patch Adrian. ZK starts up correctly (I can use a 4 letter word to interrogate both instances in a cluster), and shuts down correctly. The integration test is still failing for me however, with the following error repeating indefinitely:
{noformat}
2011-08-16 10:21:06,376 WARN [org.apache.zookeeper.ClientCnxn] (main-SendThread(ec2-107-20-63-131.compute-1.amazonaws.com:2181)) Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1078)
{noformat}
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085313#comment-13085313 ]
Andrei Savu commented on WHIRR-361:
-----------------------------------
Hadoop works for me on cloudservers & ec2 with the current trunk. Actually everything seems to be working fine and predictable on Rackspace. On EC2 I'm seeing a lot of strange failures and I'm starting to think this is somehow related to the automatically selected image. I'm planning to do more testing against a vanilla Ubuntu 10.04 LTS server but first we should fix the pom files so that we no longer get Guice exceptions in jclouds.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085391#comment-13085391 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
ahh this isn't about google collections, rather an old version of guava. something is designating guava-r05, as the following is in the mvn dependency:tree output
[INFO] org.apache.whirr:whirr-hbase:jar:0.6.0-incubating-SNAPSHOT
[INFO] +- org.apache.whirr:whirr-core:jar:0.6.0-incubating-SNAPSHOT:compile
[INFO] | +- ca.juliusdavies:not-yet-commons-ssl:jar:0.3.11:compile
[INFO] | +- org.jclouds:jclouds-compute:jar:1.1.0:compile
[INFO] | | +- org.jclouds:jclouds-scriptbuilder:jar:1.1.0:compile
[INFO] | | \- org.jclouds:jclouds-core:jar:1.1.0:compile
[INFO] | | +- net.oauth.core:oauth:jar:20100527:compile
[INFO] | | +- aopalliance:aopalliance:jar:1.0:compile
[INFO] | | +- com.sun.jersey:jersey-core:jar:1.1.5.1:compile
[INFO] | | +- com.google.inject.extensions:guice-assistedinject:jar:3.0:compile
[INFO] | | +- com.google.inject:guice:jar:3.0:compile
[INFO] | | +- javax.inject:javax.inject:jar:1:compile
[INFO] | | +- javax.annotation:jsr250-api:jar:1.0:compile
[INFO] | | +- com.google.code.gson:gson:jar:1.7.1:compile
[INFO] | | +- com.google.guava:guava:jar:r05:compile
--snip--
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Attachment: WHIRR-361-fixguava.patch
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Status: Patch Available (was: Reopened)
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-361:
------------------------------
Attachment: org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
This is the error log I get when trying to run the HBase integration tests. It's the same with maven2 & 3.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-361:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I can confirm this fixes the classpath problem. I've just committed the patch. HBase integration tests are now running and passing on cloudservers-us.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: WHIRR-361-fixguava.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Status: Patch Available (was: Open)
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086762#comment-13086762 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
+1
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-361:
------------------------------
Attachment: WHIRR-361.patch
tested patch using jclouds 1.1.1
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084579#comment-13084579 ]
Adrian Cole commented on WHIRR-361:
-----------------------------------
tested successfully BlobCacheTest with the following java args: -Dwhirr.test.provider=aws-ec2 -Dwhirr.test.identity=XXXX -Dwhirr.test.credential=XXX
2011-08-13 10:53:45,908 INFO [org.apache.whirr.util.BlobCache] (main) Created blob cache container '5x7h63enpifg' located in 'null'
2011-08-13 10:53:46,630 INFO [org.apache.whirr.util.BlobCache] (main) Uploading 'whirr5985646567714705282.txt' to '5x7h63enpifg' blob cache.
2011-08-13 10:53:52,873 INFO [org.apache.whirr.util.integration.BlobCacheTest] (main) (mkdir -p /tmp && cd /tmp && [ ! -f whirr5985646567714705282.txt ] && curl -C - -s -q -L --connect-timeout 10 --max-time 600 -X GET -s --retry 20 -H "Host: 5x7h63enpifg.s3.amazonaws.com" -H "Date: Sat, 13 Aug 2011 09:53:28 GMT" -H "Authorization: AWS 067PW7Z9P0FNH7JDPE82:eMRoiIqAe9u4I0/oh5uuzu7hApM=" https://5x7h63enpifg.s3.amazonaws.com/whirr5985646567714705282.txt >whirr5985646567714705282.txt)
2011-08-13 10:53:52,873 INFO [org.apache.whirr.util.BlobCache] (main) Removing blob cache '5x7h63enpifg'
2011-08-13 10:53:57,036 INFO [org.apache.whirr.util.BlobCache] (main) Created blob cache container 'ym26jg2ijmgz' located in 'null'
2011-08-13 10:53:57,394 INFO [org.apache.whirr.util.BlobCache] (main) Uploading 'whirr4809328296500028829.txt' to 'ym26jg2ijmgz' blob cache.
2011-08-13 10:55:56,399 INFO [org.apache.whirr.util.integration.BlobCacheTest] (main) (mkdir -p /tmp && cd /tmp && [ ! -f whirr4809328296500028829.txt ] && curl -C - -s -q -L --connect-timeout 10 --max-time 600 -X GET -s --retry 20 -H "Host: ym26jg2ijmgz.s3.amazonaws.com" -H "Date: Sat, 13 Aug 2011 09:55:45 GMT" -H "Authorization: AWS 067PW7Z9P0FNH7JDPE82:X1GLlfJtG6O+WHNFsoAN9yHg+7A=" https://ym26jg2ijmgz.s3.amazonaws.com/whirr4809328296500028829.txt >whirr4809328296500028829.txt)
2011-08-13 10:55:56,399 INFO [org.apache.whirr.util.BlobCache] (main) Removing blob cache 'ym26jg2ijmgz'
2011-08-13 10:56:18,436 INFO [org.apache.whirr.util.integration.BlobCacheTest] (main) Created temporary container 'ucnzfrosudrx'
2011-08-13 10:56:19,256 INFO [org.apache.whirr.util.integration.BlobCacheTest] (main) Removing temporary container 'ucnzfrosudrx'
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Fix For: 0.6.0
>
> Attachments: WHIRR-361.patch
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-361) refactor jclouds dependencies
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086473#comment-13086473 ]
Tom White commented on WHIRR-361:
---------------------------------
The WARN happens repeatedly and doesn't terminate. I am setting whirr.test.provider to "aws-ec2" and I have applied WHIRR-341.
> refactor jclouds dependencies
> -----------------------------
>
> Key: WHIRR-361
> URL: https://issues.apache.org/jira/browse/WHIRR-361
> Project: Whirr
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.6.0
> Reporter: Adrian Cole
> Assignee: Adrian Cole
> Priority: Blocker
> Fix For: 0.6.0
>
> Attachments: ConfigureClusterAction.java, WHIRR-361-fixguava.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, WHIRR-361.patch, org.apache.whirr.service.hbase.integration.HBase089ServiceTest.txt
>
> Original Estimate: 0.5h
> Remaining Estimate: 0.5h
>
> There are a few problems in our maven configuration, and a couple places where we aren't using the best jclouds configuration.
> * in our pom files, we needlessly declare transitive dependencies modules. this is unnecessary maintenance, as jclouds version/dependency configuration is scoped to whirr core
> * we've switched to SLF4J, yet haven't configured jclouds to use it
> * especially considering we are uploading large blobs, we should be using the jclouds EnterpriseConfigurationModule which handles file slicing much more effectively.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira