You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Loren J. Rittle" <lj...@motorola.com> on 2011/03/08 22:49:02 UTC
Issue with whirr-0.3.0-incubating and whirr-0.4.0-incubating
Hello,
While trying to start this cluster (pertaining portion of the properties file):
whirr.instance-templates=1 zk+nn+jt+hbase-master,2 zk+dn+tt+hbase-regionserver
whirr.provider=ec2
I received this message (which appears to hang the whirr
launch-cluster process):
Exception in thread "main" java.lang.IllegalStateException: The
permission '50.16.131.205/32-1-2181-2181' has already been authorized
on the specified group
at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.refineException(ParseAWSErrorFromXmlContent.java:106)
at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:86)
at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:70)
at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:201)
at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:166)
at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:135)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.jclouds.http.HttpResponseException: command: POST
https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with response:
HTTP/1.1 400 Bad Request; content: [The permission
'50.16.131.205/32-1-2181-2181' has already been authorized on the
specified group]
at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
... 9 more
I would be happy to provide more information if this is not a known issue.
It seems that this exception should be caught and ignored. Or,
redundant requests should not be made in the first place.
Regards,
Loren
--
Loren J. Rittle, Principal Staff Engineer, Motorola Mobility (IL93)
ljrittle@{motorola.com, acm.org, gcc.gnu.org, freebsd.org, gmail.com}
Re: Issue with whirr-0.3.0-incubating and whirr-0.4.0-incubating
Posted by Tom White <to...@gmail.com>.
On Tue, Mar 8, 2011 at 4:21 PM, Loren J. Rittle <lj...@motorola.com> wrote:
> Hi Tom,
>
> I confess that I have been tearing down many clusters manually (since
> no .whirr/cluster directory is created when it hangs ;-) but I have
> been careful to completely remove all keys, groups and instances (via
> the AWS management console) that were created by whirr.
>
> If I remove zookeeper from the second set of machines, then I do not
> see the issue. I tried the version posted multiple times (at least 3
> times across version .3 and .4) and have seen the error log posted
> each time.
I think this is the problem: you can't have a role split across two
template groups. In your configuration you have
whirr.instance-templates=1 zk+nn+jt+hbase-master,2 zk+dn+tt+hbase-regionserver
which splits the ZooKeeper ensemble across two groups. Until
https://issues.apache.org/jira/browse/WHIRR-249 is fixed, this won't
work.
As a workaround you can use more instances, e.g. by putting the
ZooKeeper ensemble on their own nodes. Alternatively you could use a
single node ZooKeeper cluster for a small cluster like this.
Cheers,
Tom
>
> Regards,
> Loren
>
> On Tue, Mar 8, 2011 at 5:49 PM, Tom White <to...@gmail.com> wrote:
>> Hi Loren,
>>
>> I see this occasionally when a cluster hasn't been torn down properly
>> (e.g. the instances have been terminated manually but the EC2 security
>> groups are still there). I opened
>> https://issues.apache.org/jira/browse/WHIRR-249 for this case. Do you
>> think there may be other cases causing this?
>>
>> Thanks,
>> Tom
>>
>> On Tue, Mar 8, 2011 at 1:49 PM, Loren J. Rittle <lj...@motorola.com> wrote:
>>> Hello,
>>>
>>> While trying to start this cluster (pertaining portion of the properties file):
>>>
>>> whirr.instance-templates=1 zk+nn+jt+hbase-master,2 zk+dn+tt+hbase-regionserver
>>> whirr.provider=ec2
>>>
>>> I received this message (which appears to hang the whirr
>>> launch-cluster process):
>>>
>>> Exception in thread "main" java.lang.IllegalStateException: The
>>> permission '50.16.131.205/32-1-2181-2181' has already been authorized
>>> on the specified group
>>> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.refineException(ParseAWSErrorFromXmlContent.java:106)
>>> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:86)
>>> at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:70)
>>> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:201)
>>> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:166)
>>> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:135)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>>> at java.lang.Thread.run(Thread.java:619)
>>> Caused by: org.jclouds.http.HttpResponseException: command: POST
>>> https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with response:
>>> HTTP/1.1 400 Bad Request; content: [The permission
>>> '50.16.131.205/32-1-2181-2181' has already been authorized on the
>>> specified group]
>>> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
>>> ... 9 more
>>>
>>> I would be happy to provide more information if this is not a known issue.
>>>
>>> It seems that this exception should be caught and ignored. Or,
>>> redundant requests should not be made in the first place.
>>>
>>> Regards,
>>> Loren
>>> --
>>> Loren J. Rittle, Principal Staff Engineer, Motorola Mobility (IL93)
>>> ljrittle@{motorola.com, acm.org, gcc.gnu.org, freebsd.org, gmail.com}
>>>
>>
>
>
>
> --
> Loren J. Rittle, Principal Staff Engineer, Motorola Mobility (IL93)
> ljrittle@{motorola.com, acm.org, gcc.gnu.org, freebsd.org, gmail.com}
>
Re: Issue with whirr-0.3.0-incubating and whirr-0.4.0-incubating
Posted by "Loren J. Rittle" <lj...@motorola.com>.
Hi Tom,
I confess that I have been tearing down many clusters manually (since
no .whirr/cluster directory is created when it hangs ;-) but I have
been careful to completely remove all keys, groups and instances (via
the AWS management console) that were created by whirr.
If I remove zookeeper from the second set of machines, then I do not
see the issue. I tried the version posted multiple times (at least 3
times across version .3 and .4) and have seen the error log posted
each time.
Regards,
Loren
On Tue, Mar 8, 2011 at 5:49 PM, Tom White <to...@gmail.com> wrote:
> Hi Loren,
>
> I see this occasionally when a cluster hasn't been torn down properly
> (e.g. the instances have been terminated manually but the EC2 security
> groups are still there). I opened
> https://issues.apache.org/jira/browse/WHIRR-249 for this case. Do you
> think there may be other cases causing this?
>
> Thanks,
> Tom
>
> On Tue, Mar 8, 2011 at 1:49 PM, Loren J. Rittle <lj...@motorola.com> wrote:
>> Hello,
>>
>> While trying to start this cluster (pertaining portion of the properties file):
>>
>> whirr.instance-templates=1 zk+nn+jt+hbase-master,2 zk+dn+tt+hbase-regionserver
>> whirr.provider=ec2
>>
>> I received this message (which appears to hang the whirr
>> launch-cluster process):
>>
>> Exception in thread "main" java.lang.IllegalStateException: The
>> permission '50.16.131.205/32-1-2181-2181' has already been authorized
>> on the specified group
>> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.refineException(ParseAWSErrorFromXmlContent.java:106)
>> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:86)
>> at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:70)
>> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:201)
>> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:166)
>> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:135)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>> at java.lang.Thread.run(Thread.java:619)
>> Caused by: org.jclouds.http.HttpResponseException: command: POST
>> https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with response:
>> HTTP/1.1 400 Bad Request; content: [The permission
>> '50.16.131.205/32-1-2181-2181' has already been authorized on the
>> specified group]
>> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
>> ... 9 more
>>
>> I would be happy to provide more information if this is not a known issue.
>>
>> It seems that this exception should be caught and ignored. Or,
>> redundant requests should not be made in the first place.
>>
>> Regards,
>> Loren
>> --
>> Loren J. Rittle, Principal Staff Engineer, Motorola Mobility (IL93)
>> ljrittle@{motorola.com, acm.org, gcc.gnu.org, freebsd.org, gmail.com}
>>
>
--
Loren J. Rittle, Principal Staff Engineer, Motorola Mobility (IL93)
ljrittle@{motorola.com, acm.org, gcc.gnu.org, freebsd.org, gmail.com}
Re: Issue with whirr-0.3.0-incubating and whirr-0.4.0-incubating
Posted by Tom White <to...@gmail.com>.
Hi Loren,
I see this occasionally when a cluster hasn't been torn down properly
(e.g. the instances have been terminated manually but the EC2 security
groups are still there). I opened
https://issues.apache.org/jira/browse/WHIRR-249 for this case. Do you
think there may be other cases causing this?
Thanks,
Tom
On Tue, Mar 8, 2011 at 1:49 PM, Loren J. Rittle <lj...@motorola.com> wrote:
> Hello,
>
> While trying to start this cluster (pertaining portion of the properties file):
>
> whirr.instance-templates=1 zk+nn+jt+hbase-master,2 zk+dn+tt+hbase-regionserver
> whirr.provider=ec2
>
> I received this message (which appears to hang the whirr
> launch-cluster process):
>
> Exception in thread "main" java.lang.IllegalStateException: The
> permission '50.16.131.205/32-1-2181-2181' has already been authorized
> on the specified group
> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.refineException(ParseAWSErrorFromXmlContent.java:106)
> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:86)
> at org.jclouds.http.handlers.DelegatingErrorHandler.handleError(DelegatingErrorHandler.java:70)
> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.shouldContinue(BaseHttpCommandExecutorService.java:201)
> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:166)
> at org.jclouds.http.internal.BaseHttpCommandExecutorService$HttpResponseCallable.call(BaseHttpCommandExecutorService.java:135)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: org.jclouds.http.HttpResponseException: command: POST
> https://ec2.us-east-1.amazonaws.com/ HTTP/1.1 failed with response:
> HTTP/1.1 400 Bad Request; content: [The permission
> '50.16.131.205/32-1-2181-2181' has already been authorized on the
> specified group]
> at org.jclouds.aws.handlers.ParseAWSErrorFromXmlContent.handleError(ParseAWSErrorFromXmlContent.java:75)
> ... 9 more
>
> I would be happy to provide more information if this is not a known issue.
>
> It seems that this exception should be caught and ignored. Or,
> redundant requests should not be made in the first place.
>
> Regards,
> Loren
> --
> Loren J. Rittle, Principal Staff Engineer, Motorola Mobility (IL93)
> ljrittle@{motorola.com, acm.org, gcc.gnu.org, freebsd.org, gmail.com}
>