You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whirr.apache.org by "Marc de Palol (JIRA)" <ji...@apache.org> on 2011/08/30 17:49:37 UTC
[jira] [Created] (WHIRR-378) Auth fail when creating a cluster from
an EC2 instance
Auth fail when creating a cluster from an EC2 instance
------------------------------------------------------
Key: WHIRR-378
URL: https://issues.apache.org/jira/browse/WHIRR-378
Project: Whirr
Issue Type: Bug
Components: service/hadoop
Affects Versions: 0.6.0
Reporter: Marc de Palol
There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
The user in the virtual machine is new and with valid .ssh keys.
The hadoop config file is (omitting commented lines):
whirr.cluster-name=hadoop
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
whirr.provider=aws-ec2
whirr.identity=****
whirr.credential=****
whirr.hardware-id=c1.xlarge
whirr.image-id=us-east-1/ami-da0cf8b3
whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
On Wed, Jan 4, 2012 at 2:21 AM, David Dundas <da...@textingly.com> wrote:
> Oh I understand now. I definitely misread your post. Ok then what is the
> difference between
> whirr.cluster-user and whirr.login-user?
>
whirr.login-user is only used for bootstrap. As part of the bootstrap
script Whirr is creating a new user
on the remote machine named whirr.cluster-user. This is the user you should
use for accessing the
remote machines.
The problem is that this mechanism is still brittle and it fails badly if
the remote user already exists.
>
> Do I need to create ssh key for the cluster user as well?
>
Whirr is automatically using / installing the ones you specify in the
.properties file.
>
> Should I run the command as the cluster user or the command user?
>
Run the command as any user if you specify a value for whirr.cluster-user.
>
> Can the login-user and cluster-user be the same?
>
No - as explained above.
>
> Thanks!
>
> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com> wrote:
>
>>
>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com> wrote:
>>
>>> * have you added something like whirr.cluster-user=whirr to
>>> hadoop.properties?
>>> >>> No I havent - should I be?
>>>
>>
>> Yes!! Whirr is trying to create the ec2-user on the remote machine and it
>> fails because that user already exists.
>>
>> I have created the following two related JIRA issues:
>> * https://issues.apache.org/jira/browse/WHIRR-463
>> * https://issues.apache.org/jira/browse/WHIRR-464
>>
>> As I said already in a comment my advice is to add:
>>
>> whirr.cluster-user=whirr (to hadoop.properties)
>>
>> and you should be able to login using something like:
>>
>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>
>> -- Andrei
>>
>>
>
>
>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
On Thu, Jan 5, 2012 at 8:44 PM, David Dundas <da...@textingly.com> wrote:
> Ok sounds good. Is there a command to add additional nodes to the same
> cluster?
Nope. We are working on this for 0.8.0.
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by David Dundas <da...@textingly.com>.
Ok sounds good. Is there a command to add additional nodes to the same
cluster?
Seeing as that's broken perhaps the easiest work around is to launch the
cluster, then add nodes to it to run the job?
On Thu, Jan 5, 2012 at 11:25 AM, Andrei Savu <sa...@gmail.com> wrote:
> David I will give it a try later today.
>
> -- Andrei
>
> On Wed, Jan 4, 2012 at 10:50 PM, David Dundas <da...@textingly.com> wrote:
>
>> Actually it seems that this command returns the same error
>>
>> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 2
>> hadoop-datanode+hadoop-tasktracker
>>
>> but this one doesn't
>>
>> whirr.instance-templates=1
>> hadoop-namenode+hadoop-jobtracker+hadoop-datanode+hadoop-tasktracker
>>
>> It looks like it's breaking when there are multiple groupings of
>> instances.
>>
>>
>>
>> On Wed, Jan 4, 2012 at 3:41 PM, Andrei Savu <sa...@gmail.com>wrote:
>>
>>> Let me know if a similar recipe for Hadoop works with Whirr 0.7.0 - it
>>> seems like the trunk is a bit broken now.
>>>
>>> -- Andrei Savu
>>>
>>>
>>> On Wed, Jan 4, 2012 at 10:30 PM, David Dundas <da...@textingly.com>wrote:
>>>
>>>> Ok that recipe works. It also looks like something in my original
>>>> whir.instance-templates
>>>>
>>>> May have been causing an error as well.
>>>>
>>>> Thanks!
>>>>
>>>> On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <sa...@gmail.com>wrote:
>>>>
>>>>> I have done some testing and I have been able to start a fully
>>>>> functional ZooKeeper cluster from a VM also running inside the Amazon Cloud
>>>>> using a recipe like this:
>>>>>
>>>>> whirr.cluster-name=zookeeper-asavu-from-ec2
>>>>> whirr.instance-templates=3 zookeeper
>>>>> whirr.cluster-user=asavu
>>>>> whirr.provider=aws-ec2
>>>>> whirr.identity=${env:AWS_ACCESS_KEY_ID}
>>>>> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
>>>>>
>>>>> Hadoop fails with the following exception but this is unrelated to
>>>>> this discussion:
>>>>>
>>>>> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
>>>>> 8021] for [184.72.197.142/32]
>>>>> Unable to start the cluster. Terminating all nodes.
>>>>> java.lang.NullPointerException
>>>>> at
>>>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
>>>>> at
>>>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
>>>>> at
>>>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
>>>>> at
>>>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
>>>>> at
>>>>> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
>>>>> at
>>>>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
>>>>> at
>>>>> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
>>>>> at
>>>>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
>>>>> at
>>>>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
>>>>> at org.apache.whirr.cli.Main.run(Main.java:64)
>>>>> at org.apache.whirr.cli.Main.main(Main.java:97)
>>>>>
>>>>> I am now running the integration tests on my machine to make sure that
>>>>> everything still works fine.
>>>>>
>>>>> -- Andrei Savu / andreisavu.ro
>>>>>
>>>>> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>>
>>>>>> I will give it a try now and get back to you with more feedback.
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com>wrote:
>>>>>>
>>>>>>> Also just added whir as the cluster-user, and it looks like I'm
>>>>>>> still getting the same errors.
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com>wrote:
>>>>>>>
>>>>>>>> Oh I understand now. I definitely misread your post. Ok then what
>>>>>>>> is the difference between
>>>>>>>> whirr.cluster-user and whirr.login-user?
>>>>>>>>
>>>>>>>> Do I need to create ssh key for the cluster user as well?
>>>>>>>>
>>>>>>>> Should I run the command as the cluster user or the command user?
>>>>>>>>
>>>>>>>> Can the login-user and cluster-user be the same?
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>>>>>>>
>>>>>>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>>>>>>> hadoop.properties?
>>>>>>>>>> >>> No I havent - should I be?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine
>>>>>>>>> and it fails because that user already exists.
>>>>>>>>>
>>>>>>>>> I have created the following two related JIRA issues:
>>>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>>>>>>
>>>>>>>>> As I said already in a comment my advice is to add:
>>>>>>>>>
>>>>>>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>>>>>>
>>>>>>>>> and you should be able to login using something like:
>>>>>>>>>
>>>>>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>>>>>>
>>>>>>>>> -- Andrei
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
David I will give it a try later today.
-- Andrei
On Wed, Jan 4, 2012 at 10:50 PM, David Dundas <da...@textingly.com> wrote:
> Actually it seems that this command returns the same error
>
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 2
> hadoop-datanode+hadoop-tasktracker
>
> but this one doesn't
>
> whirr.instance-templates=1
> hadoop-namenode+hadoop-jobtracker+hadoop-datanode+hadoop-tasktracker
>
> It looks like it's breaking when there are multiple groupings of
> instances.
>
>
>
> On Wed, Jan 4, 2012 at 3:41 PM, Andrei Savu <sa...@gmail.com> wrote:
>
>> Let me know if a similar recipe for Hadoop works with Whirr 0.7.0 - it
>> seems like the trunk is a bit broken now.
>>
>> -- Andrei Savu
>>
>>
>> On Wed, Jan 4, 2012 at 10:30 PM, David Dundas <da...@textingly.com>wrote:
>>
>>> Ok that recipe works. It also looks like something in my original
>>> whir.instance-templates
>>>
>>> May have been causing an error as well.
>>>
>>> Thanks!
>>>
>>> On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <sa...@gmail.com>wrote:
>>>
>>>> I have done some testing and I have been able to start a fully
>>>> functional ZooKeeper cluster from a VM also running inside the Amazon Cloud
>>>> using a recipe like this:
>>>>
>>>> whirr.cluster-name=zookeeper-asavu-from-ec2
>>>> whirr.instance-templates=3 zookeeper
>>>> whirr.cluster-user=asavu
>>>> whirr.provider=aws-ec2
>>>> whirr.identity=${env:AWS_ACCESS_KEY_ID}
>>>> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
>>>>
>>>> Hadoop fails with the following exception but this is unrelated to this
>>>> discussion:
>>>>
>>>> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
>>>> 8021] for [184.72.197.142/32]
>>>> Unable to start the cluster. Terminating all nodes.
>>>> java.lang.NullPointerException
>>>> at
>>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
>>>> at
>>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
>>>> at
>>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
>>>> at
>>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
>>>> at
>>>> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
>>>> at
>>>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
>>>> at
>>>> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
>>>> at
>>>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
>>>> at
>>>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
>>>> at org.apache.whirr.cli.Main.run(Main.java:64)
>>>> at org.apache.whirr.cli.Main.main(Main.java:97)
>>>>
>>>> I am now running the integration tests on my machine to make sure that
>>>> everything still works fine.
>>>>
>>>> -- Andrei Savu / andreisavu.ro
>>>>
>>>> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>
>>>>> I will give it a try now and get back to you with more feedback.
>>>>>
>>>>>
>>>>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com>wrote:
>>>>>
>>>>>> Also just added whir as the cluster-user, and it looks like I'm still
>>>>>> getting the same errors.
>>>>>>
>>>>>>
>>>>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com>wrote:
>>>>>>
>>>>>>> Oh I understand now. I definitely misread your post. Ok then what is
>>>>>>> the difference between
>>>>>>> whirr.cluster-user and whirr.login-user?
>>>>>>>
>>>>>>> Do I need to create ssh key for the cluster user as well?
>>>>>>>
>>>>>>> Should I run the command as the cluster user or the command user?
>>>>>>>
>>>>>>> Can the login-user and cluster-user be the same?
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>>>>>>
>>>>>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>>>>>> hadoop.properties?
>>>>>>>>> >>> No I havent - should I be?
>>>>>>>>>
>>>>>>>>
>>>>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine
>>>>>>>> and it fails because that user already exists.
>>>>>>>>
>>>>>>>> I have created the following two related JIRA issues:
>>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>>>>>
>>>>>>>> As I said already in a comment my advice is to add:
>>>>>>>>
>>>>>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>>>>>
>>>>>>>> and you should be able to login using something like:
>>>>>>>>
>>>>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>>>>>
>>>>>>>> -- Andrei
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by David Dundas <da...@textingly.com>.
Actually it seems that this command returns the same error
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 2
hadoop-datanode+hadoop-tasktracker
but this one doesn't
whirr.instance-templates=1
hadoop-namenode+hadoop-jobtracker+hadoop-datanode+hadoop-tasktracker
It looks like it's breaking when there are multiple groupings of instances.
On Wed, Jan 4, 2012 at 3:41 PM, Andrei Savu <sa...@gmail.com> wrote:
> Let me know if a similar recipe for Hadoop works with Whirr 0.7.0 - it
> seems like the trunk is a bit broken now.
>
> -- Andrei Savu
>
>
> On Wed, Jan 4, 2012 at 10:30 PM, David Dundas <da...@textingly.com> wrote:
>
>> Ok that recipe works. It also looks like something in my original
>> whir.instance-templates
>>
>> May have been causing an error as well.
>>
>> Thanks!
>>
>> On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <sa...@gmail.com>wrote:
>>
>>> I have done some testing and I have been able to start a fully
>>> functional ZooKeeper cluster from a VM also running inside the Amazon Cloud
>>> using a recipe like this:
>>>
>>> whirr.cluster-name=zookeeper-asavu-from-ec2
>>> whirr.instance-templates=3 zookeeper
>>> whirr.cluster-user=asavu
>>> whirr.provider=aws-ec2
>>> whirr.identity=${env:AWS_ACCESS_KEY_ID}
>>> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
>>>
>>> Hadoop fails with the following exception but this is unrelated to this
>>> discussion:
>>>
>>> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
>>> 8021] for [184.72.197.142/32]
>>> Unable to start the cluster. Terminating all nodes.
>>> java.lang.NullPointerException
>>> at
>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
>>> at
>>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
>>> at
>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
>>> at
>>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
>>> at
>>> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
>>> at
>>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
>>> at
>>> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
>>> at
>>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
>>> at
>>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
>>> at org.apache.whirr.cli.Main.run(Main.java:64)
>>> at org.apache.whirr.cli.Main.main(Main.java:97)
>>>
>>> I am now running the integration tests on my machine to make sure that
>>> everything still works fine.
>>>
>>> -- Andrei Savu / andreisavu.ro
>>>
>>> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>
>>>> I will give it a try now and get back to you with more feedback.
>>>>
>>>>
>>>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com>wrote:
>>>>
>>>>> Also just added whir as the cluster-user, and it looks like I'm still
>>>>> getting the same errors.
>>>>>
>>>>>
>>>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com>wrote:
>>>>>
>>>>>> Oh I understand now. I definitely misread your post. Ok then what is
>>>>>> the difference between
>>>>>> whirr.cluster-user and whirr.login-user?
>>>>>>
>>>>>> Do I need to create ssh key for the cluster user as well?
>>>>>>
>>>>>> Should I run the command as the cluster user or the command user?
>>>>>>
>>>>>> Can the login-user and cluster-user be the same?
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>>>>>
>>>>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>>>>> hadoop.properties?
>>>>>>>> >>> No I havent - should I be?
>>>>>>>>
>>>>>>>
>>>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine
>>>>>>> and it fails because that user already exists.
>>>>>>>
>>>>>>> I have created the following two related JIRA issues:
>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>>>>
>>>>>>> As I said already in a comment my advice is to add:
>>>>>>>
>>>>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>>>>
>>>>>>> and you should be able to login using something like:
>>>>>>>
>>>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>>>>
>>>>>>> -- Andrei
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
Let me know if a similar recipe for Hadoop works with Whirr 0.7.0 - it
seems like the trunk is a bit broken now.
-- Andrei Savu
On Wed, Jan 4, 2012 at 10:30 PM, David Dundas <da...@textingly.com> wrote:
> Ok that recipe works. It also looks like something in my original
> whir.instance-templates
>
> May have been causing an error as well.
>
> Thanks!
>
> On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <sa...@gmail.com>wrote:
>
>> I have done some testing and I have been able to start a fully functional
>> ZooKeeper cluster from a VM also running inside the Amazon Cloud using a
>> recipe like this:
>>
>> whirr.cluster-name=zookeeper-asavu-from-ec2
>> whirr.instance-templates=3 zookeeper
>> whirr.cluster-user=asavu
>> whirr.provider=aws-ec2
>> whirr.identity=${env:AWS_ACCESS_KEY_ID}
>> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
>>
>> Hadoop fails with the following exception but this is unrelated to this
>> discussion:
>>
>> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
>> 8021] for [184.72.197.142/32]
>> Unable to start the cluster. Terminating all nodes.
>> java.lang.NullPointerException
>> at
>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
>> at
>> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
>> at
>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
>> at
>> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
>> at
>> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
>> at
>> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
>> at
>> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
>> at
>> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
>> at
>> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
>> at org.apache.whirr.cli.Main.run(Main.java:64)
>> at org.apache.whirr.cli.Main.main(Main.java:97)
>>
>> I am now running the integration tests on my machine to make sure that
>> everything still works fine.
>>
>> -- Andrei Savu / andreisavu.ro
>>
>> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <sa...@gmail.com>wrote:
>>
>>> I will give it a try now and get back to you with more feedback.
>>>
>>>
>>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com>wrote:
>>>
>>>> Also just added whir as the cluster-user, and it looks like I'm still
>>>> getting the same errors.
>>>>
>>>>
>>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com>wrote:
>>>>
>>>>> Oh I understand now. I definitely misread your post. Ok then what is
>>>>> the difference between
>>>>> whirr.cluster-user and whirr.login-user?
>>>>>
>>>>> Do I need to create ssh key for the cluster user as well?
>>>>>
>>>>> Should I run the command as the cluster user or the command user?
>>>>>
>>>>> Can the login-user and cluster-user be the same?
>>>>>
>>>>> Thanks!
>>>>>
>>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>>
>>>>>>
>>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>>>>
>>>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>>>> hadoop.properties?
>>>>>>> >>> No I havent - should I be?
>>>>>>>
>>>>>>
>>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine
>>>>>> and it fails because that user already exists.
>>>>>>
>>>>>> I have created the following two related JIRA issues:
>>>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>>>
>>>>>> As I said already in a comment my advice is to add:
>>>>>>
>>>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>>>
>>>>>> and you should be able to login using something like:
>>>>>>
>>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>>>
>>>>>> -- Andrei
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>
>>
>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by David Dundas <da...@textingly.com>.
Ok that recipe works. It also looks like something in my original
whir.instance-templates
May have been causing an error as well.
Thanks!
On Wed, Jan 4, 2012 at 10:48 AM, Andrei Savu <sa...@gmail.com> wrote:
> I have done some testing and I have been able to start a fully functional
> ZooKeeper cluster from a VM also running inside the Amazon Cloud using a
> recipe like this:
>
> whirr.cluster-name=zookeeper-asavu-from-ec2
> whirr.instance-templates=3 zookeeper
> whirr.cluster-user=asavu
> whirr.provider=aws-ec2
> whirr.identity=${env:AWS_ACCESS_KEY_ID}
> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
>
> Hadoop fails with the following exception but this is unrelated to this
> discussion:
>
> Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
> 8021] for [184.72.197.142/32]
> Unable to start the cluster. Terminating all nodes.
> java.lang.NullPointerException
> at
> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
> at
> org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
> at
> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
> at
> org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
> at
> org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
> at
> org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
> at
> org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
> at
> org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
> at
> org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
> at org.apache.whirr.cli.Main.run(Main.java:64)
> at org.apache.whirr.cli.Main.main(Main.java:97)
>
> I am now running the integration tests on my machine to make sure that
> everything still works fine.
>
> -- Andrei Savu / andreisavu.ro
>
> On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <sa...@gmail.com>wrote:
>
>> I will give it a try now and get back to you with more feedback.
>>
>>
>> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com> wrote:
>>
>>> Also just added whir as the cluster-user, and it looks like I'm still
>>> getting the same errors.
>>>
>>>
>>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com>wrote:
>>>
>>>> Oh I understand now. I definitely misread your post. Ok then what is
>>>> the difference between
>>>> whirr.cluster-user and whirr.login-user?
>>>>
>>>> Do I need to create ssh key for the cluster user as well?
>>>>
>>>> Should I run the command as the cluster user or the command user?
>>>>
>>>> Can the login-user and cluster-user be the same?
>>>>
>>>> Thanks!
>>>>
>>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>>
>>>>>
>>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>>>
>>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>>> hadoop.properties?
>>>>>> >>> No I havent - should I be?
>>>>>>
>>>>>
>>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine and
>>>>> it fails because that user already exists.
>>>>>
>>>>> I have created the following two related JIRA issues:
>>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>>
>>>>> As I said already in a comment my advice is to add:
>>>>>
>>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>>
>>>>> and you should be able to login using something like:
>>>>>
>>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>>
>>>>> -- Andrei
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
I have done some testing and I have been able to start a fully functional
ZooKeeper cluster from a VM also running inside the Amazon Cloud using a
recipe like this:
whirr.cluster-name=zookeeper-asavu-from-ec2
whirr.instance-templates=3 zookeeper
whirr.cluster-user=asavu
whirr.provider=aws-ec2
whirr.identity=${env:AWS_ACCESS_KEY_ID}
whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
Hadoop fails with the following exception but this is unrelated to this
discussion:
Authorizing firewall ingress to [us-east-1/i-e2187780] on ports [8020,
8021] for [184.72.197.142/32]
Unable to start the cluster. Terminating all nodes.
java.lang.NullPointerException
at
org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduceConfiguration(HadoopConfigurationBuilder.java:113)
at
org.apache.whirr.service.hadoop.HadoopConfigurationBuilder.buildMapReduce(HadoopConfigurationBuilder.java:71)
at
org.apache.whirr.service.hadoop.HadoopClusterActionHandler.createHadoopConfigFiles(HadoopClusterActionHandler.java:102)
at
org.apache.whirr.service.hadoop.HadoopClusterActionHandler.beforeConfigure(HadoopClusterActionHandler.java:88)
at
org.apache.whirr.service.ClusterActionHandlerSupport.beforeAction(ClusterActionHandlerSupport.java:53)
at
org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:100)
at
org.apache.whirr.ClusterController.configureServices(ClusterController.java:148)
at
org.apache.whirr.ClusterController.launchCluster(ClusterController.java:110)
at
org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
at org.apache.whirr.cli.Main.run(Main.java:64)
at org.apache.whirr.cli.Main.main(Main.java:97)
I am now running the integration tests on my machine to make sure that
everything still works fine.
-- Andrei Savu / andreisavu.ro
On Wed, Jan 4, 2012 at 12:23 PM, Andrei Savu <sa...@gmail.com> wrote:
> I will give it a try now and get back to you with more feedback.
>
>
> On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com> wrote:
>
>> Also just added whir as the cluster-user, and it looks like I'm still
>> getting the same errors.
>>
>>
>> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com> wrote:
>>
>>> Oh I understand now. I definitely misread your post. Ok then what is the
>>> difference between
>>> whirr.cluster-user and whirr.login-user?
>>>
>>> Do I need to create ssh key for the cluster user as well?
>>>
>>> Should I run the command as the cluster user or the command user?
>>>
>>> Can the login-user and cluster-user be the same?
>>>
>>> Thanks!
>>>
>>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>>
>>>>
>>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>>
>>>>> * have you added something like whirr.cluster-user=whirr to
>>>>> hadoop.properties?
>>>>> >>> No I havent - should I be?
>>>>>
>>>>
>>>> Yes!! Whirr is trying to create the ec2-user on the remote machine and
>>>> it fails because that user already exists.
>>>>
>>>> I have created the following two related JIRA issues:
>>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>>
>>>> As I said already in a comment my advice is to add:
>>>>
>>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>>
>>>> and you should be able to login using something like:
>>>>
>>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>>
>>>> -- Andrei
>>>>
>>>>
>>>
>>>
>>>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
I will give it a try now and get back to you with more feedback.
On Wed, Jan 4, 2012 at 2:22 AM, David Dundas <da...@textingly.com> wrote:
> Also just added whir as the cluster-user, and it looks like I'm still
> getting the same errors.
>
>
> On Tue, Jan 3, 2012 at 7:21 PM, David Dundas <da...@textingly.com> wrote:
>
>> Oh I understand now. I definitely misread your post. Ok then what is the
>> difference between
>> whirr.cluster-user and whirr.login-user?
>>
>> Do I need to create ssh key for the cluster user as well?
>>
>> Should I run the command as the cluster user or the command user?
>>
>> Can the login-user and cluster-user be the same?
>>
>> Thanks!
>>
>> On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com>wrote:
>>
>>>
>>> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com>wrote:
>>>
>>>> * have you added something like whirr.cluster-user=whirr to
>>>> hadoop.properties?
>>>> >>> No I havent - should I be?
>>>>
>>>
>>> Yes!! Whirr is trying to create the ec2-user on the remote machine and
>>> it fails because that user already exists.
>>>
>>> I have created the following two related JIRA issues:
>>> * https://issues.apache.org/jira/browse/WHIRR-463
>>> * https://issues.apache.org/jira/browse/WHIRR-464
>>>
>>> As I said already in a comment my advice is to add:
>>>
>>> whirr.cluster-user=whirr (to hadoop.properties)
>>>
>>> and you should be able to login using something like:
>>>
>>> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>>>
>>> -- Andrei
>>>
>>>
>>
>>
>>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by David Dundas <da...@textingly.com>.
Oh I understand now. I definitely misread your post. Ok then what is the
difference between
whirr.cluster-user and whirr.login-user?
Do I need to create ssh key for the cluster user as well?
Should I run the command as the cluster user or the command user?
Can the login-user and cluster-user be the same?
Thanks!
On Tue, Jan 3, 2012 at 7:04 PM, Andrei Savu <sa...@gmail.com> wrote:
>
> On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com> wrote:
>
>> * have you added something like whirr.cluster-user=whirr to
>> hadoop.properties?
>> >>> No I havent - should I be?
>>
>
> Yes!! Whirr is trying to create the ec2-user on the remote machine and it
> fails because that user already exists.
>
> I have created the following two related JIRA issues:
> * https://issues.apache.org/jira/browse/WHIRR-463
> * https://issues.apache.org/jira/browse/WHIRR-464
>
> As I said already in a comment my advice is to add:
>
> whirr.cluster-user=whirr (to hadoop.properties)
>
> and you should be able to login using something like:
>
> ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
>
> -- Andrei
>
>
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
On Wed, Jan 4, 2012 at 1:57 AM, David Dundas <da...@textingly.com> wrote:
> * have you added something like whirr.cluster-user=whirr to
> hadoop.properties?
> >>> No I havent - should I be?
>
Yes!! Whirr is trying to create the ec2-user on the remote machine and it
fails because that user already exists.
I have created the following two related JIRA issues:
* https://issues.apache.org/jira/browse/WHIRR-463
* https://issues.apache.org/jira/browse/WHIRR-464
As I said already in a comment my advice is to add:
whirr.cluster-user=whirr (to hadoop.properties)
and you should be able to login using something like:
ssh -i ~/.ssh/id_rsa_whirr whirr@remote-host-ip
-- Andrei
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by David Dundas <da...@textingly.com>.
* are you running Whirr from a local machine or from a VM inside Amazon?
What user?
>>> VM Inside Amazon. Using a user I created not ec2-user or root.
* have you added something like whirr.cluster-user=whirr to
hadoop.properties?
>>> No I havent - should I be?
Here's our properties file:
whirr.cluster-name=myhadoopcluster
whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1
hadoop-datanode+hadoop-tasktracker
whirr.provider=aws-ec2
whirr.identity=<KEY>
whirr.credential=<SECRET>
whirr.login-user=USER
whirr.hardware-id=m1.small
whirr.image-id=us-east-1/ami-6936fb00
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
~
On Tue, Jan 3, 2012 at 6:46 PM, Andrei Savu <sa...@gmail.com> wrote:
> Moving this on the email list.
>
> A few questions:
>
> * are you running Whirr from a local machine or from a VM inside Amazon?
> What user?
> * have you added something like whirr.cluster-user=whirr to
> hadoop.properties?
>
> -- Andrei Savu / andreisavu.ro
>
> On Wed, Jan 4, 2012 at 1:40 AM, Michael David (Commented) (JIRA) <
> jira@apache.org> wrote:
>
>>
>> [
>> https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179133#comment-13179133]
>>
>> Michael David commented on WHIRR-378:
>> -------------------------------------
>>
>> Hi Andrei - Just tried with 0.7.0 and am getting new errors:
>>
>> Instead of using the properties file, I tried to invoke keys in command
>> line instead using the tuorial here:
>> http://whirr.apache.org/docs/0.7.0/whirr-in-5-minutes.html
>>
>> ./whirr launch-cluster --config hadoop.properties --private-key-file
>> ~/.ssh/id_rsa_whirr --public-key-file ~/.ssh/id_rsa_whirr.pub
>>
>> It seems that I'm not getting a different set of auth errors:
>>
>> Dying because - net.schmizz.sshj.transport.TransportException: Broken
>> transport; encountered EOF
>> Dying because - net.schmizz.sshj.transport.TransportException: Broken
>> transport; encountered EOF
>> <<kex done>> woke to: net.schmizz.sshj.transport.TransportException:
>> Broken transport; encountered EOF
>> << (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx.xx:22) error
>> acquiring SSHClient(timeout=60000): Broken transport; encountered EOF
>> net.schmizz.sshj.transport.TransportException: Broken transport;
>> encountered EOF
>> at net.schmizz.sshj.transport.Reader.run(Reader.java:70)
>> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
>> publickey auth failed
>> << (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx:22) error
>> acquiring SSHClient(timeout=60000): exception not retryable
>> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
>> authentication methods
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
>> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
>> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
>> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
>> at
>> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:636)
>> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
>> failed
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
>> at
>> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
>> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
>> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
>> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
>>
>> << problem applying options to node(us-east-1/i-xxxxxe):
>> org.jclouds.rest.AuthorizationException:
>> (USER:rsa[fingerprint(RSA),sha1(SHA)]@xx.xx.xx:22)
>> (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring
>> SSHClient(timeout=60000): Exhausted available authentication methods
>> at
>> org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413)
>> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244)
>> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
>> at
>> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:636)
>> Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted
>> available authentication methods
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
>> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
>> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
>> ... 9 more
>> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
>> failed
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
>> at
>> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
>> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
>> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
>> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
>> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
>> publickey auth failed
>> << (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xxx.xx:22) error acquiring
>> SSHClient(timeout=60000): exception not retryable
>> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
>> authentication methods
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
>> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
>> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
>> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
>> at
>> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:636)
>> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
>> failed
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
>> at
>> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
>> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
>> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
>> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
>> << (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring
>> SSHClient(timeout=60000): Exhausted available authentication methods
>> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
>> authentication methods
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
>> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
>> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
>> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
>> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
>> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
>> at
>> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
>> at
>> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:636)
>> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
>> failed
>> at
>> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
>> at
>> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
>> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
>> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
>> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
>>
>> Thanks - really appreciate your speedy answers here!
>>
>>
>> > Auth fail when creating a cluster from an EC2 instance
>> > ------------------------------------------------------
>> >
>> > Key: WHIRR-378
>> > URL: https://issues.apache.org/jira/browse/WHIRR-378
>> > Project: Whirr
>> > Issue Type: Bug
>> > Components: service/hadoop
>> > Affects Versions: 0.6.0
>> > Reporter: Marc de Palol
>> > Assignee: Adrian Cole
>> > Priority: Critical
>> > Fix For: 0.8.0
>> >
>> > Attachments: run-test.sh, setup-whirr.sh
>> >
>> >
>> > There is a ssh auth problem when creating a hadoop cluster from an EC2
>> ubuntu instance.
>> > I've been using the same configuration file from an EC2 computer an a
>> physical one, everything works fine in the physical one, but I keep getting
>> this error in EC2:
>> > Running configuration script on nodes: [us-east-1/i-c7fde5a6,
>> us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
>> > <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
>> publickey auth failed
>> > <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
>> publickey auth failed
>> > The user in the virtual machine is new and with valid .ssh keys.
>> > The hadoop config file is (omitting commented lines):
>> > whirr.cluster-name=hadoop
>> > whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3
>> hadoop-datanode+hadoop-tasktracker
>> > whirr.provider=aws-ec2
>> > whirr.identity=****
>> > whirr.credential=****
>> > whirr.hardware-id=c1.xlarge
>> > whirr.image-id=us-east-1/ami-da0cf8b3
>> > whirr.location-id=us-east-1
>>
>> --
>> This message is automatically generated by JIRA.
>> If you think it was sent incorrectly, please contact your JIRA
>> administrators:
>> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
>> For more information on JIRA, see: http://www.atlassian.com/software/jira
>>
>>
>>
>
--
_______________________________
David Dundas
CEO - textingly
www.textingly.com
text DAVID to 52627
linkedin - http://www.linkedin.com/in/daviddundas
textingly twitter: twitter.com/textingly
my twitter: twitter.com/dundas
Re: [jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by Andrei Savu <sa...@gmail.com>.
Moving this on the email list.
A few questions:
* are you running Whirr from a local machine or from a VM inside Amazon?
What user?
* have you added something like whirr.cluster-user=whirr to
hadoop.properties?
-- Andrei Savu / andreisavu.ro
On Wed, Jan 4, 2012 at 1:40 AM, Michael David (Commented) (JIRA) <
jira@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179133#comment-13179133]
>
> Michael David commented on WHIRR-378:
> -------------------------------------
>
> Hi Andrei - Just tried with 0.7.0 and am getting new errors:
>
> Instead of using the properties file, I tried to invoke keys in command
> line instead using the tuorial here:
> http://whirr.apache.org/docs/0.7.0/whirr-in-5-minutes.html
>
> ./whirr launch-cluster --config hadoop.properties --private-key-file
> ~/.ssh/id_rsa_whirr --public-key-file ~/.ssh/id_rsa_whirr.pub
>
> It seems that I'm not getting a different set of auth errors:
>
> Dying because - net.schmizz.sshj.transport.TransportException: Broken
> transport; encountered EOF
> Dying because - net.schmizz.sshj.transport.TransportException: Broken
> transport; encountered EOF
> <<kex done>> woke to: net.schmizz.sshj.transport.TransportException:
> Broken transport; encountered EOF
> << (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx.xx:22) error
> acquiring SSHClient(timeout=60000): Broken transport; encountered EOF
> net.schmizz.sshj.transport.TransportException: Broken transport;
> encountered EOF
> at net.schmizz.sshj.transport.Reader.run(Reader.java:70)
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
> publickey auth failed
> << (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx:22) error
> acquiring SSHClient(timeout=60000): exception not retryable
> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
> authentication methods
> at
> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
> at
> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
> failed
> at
> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
> at
> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
>
> << problem applying options to node(us-east-1/i-xxxxxe):
> org.jclouds.rest.AuthorizationException:
> (USER:rsa[fingerprint(RSA),sha1(SHA)]@xx.xx.xx:22)
> (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring
> SSHClient(timeout=60000): Exhausted available authentication methods
> at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413)
> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244)
> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
> at
> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted
> available authentication methods
> at
> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
> ... 9 more
> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
> failed
> at
> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
> at
> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
> publickey auth failed
> << (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xxx.xx:22) error acquiring
> SSHClient(timeout=60000): exception not retryable
> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
> authentication methods
> at
> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
> at
> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
> failed
> at
> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
> at
> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
> << (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring
> SSHClient(timeout=60000): Exhausted available authentication methods
> net.schmizz.sshj.userauth.UserAuthException: Exhausted available
> authentication methods
> at
> net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
> at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
> at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
> at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
> at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
> at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
> at
> org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
> at
> org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth
> failed
> at
> net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
> at
> net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
> at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
> at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
> at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
>
> Thanks - really appreciate your speedy answers here!
>
>
> > Auth fail when creating a cluster from an EC2 instance
> > ------------------------------------------------------
> >
> > Key: WHIRR-378
> > URL: https://issues.apache.org/jira/browse/WHIRR-378
> > Project: Whirr
> > Issue Type: Bug
> > Components: service/hadoop
> > Affects Versions: 0.6.0
> > Reporter: Marc de Palol
> > Assignee: Adrian Cole
> > Priority: Critical
> > Fix For: 0.8.0
> >
> > Attachments: run-test.sh, setup-whirr.sh
> >
> >
> > There is a ssh auth problem when creating a hadoop cluster from an EC2
> ubuntu instance.
> > I've been using the same configuration file from an EC2 computer an a
> physical one, everything works fine in the physical one, but I keep getting
> this error in EC2:
> > Running configuration script on nodes: [us-east-1/i-c7fde5a6,
> us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> > <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
> publickey auth failed
> > <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException:
> publickey auth failed
> > The user in the virtual machine is new and with valid .ssh keys.
> > The hadoop config file is (omitting commented lines):
> > whirr.cluster-name=hadoop
> > whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3
> hadoop-datanode+hadoop-tasktracker
> > whirr.provider=aws-ec2
> > whirr.identity=****
> > whirr.credential=****
> > whirr.hardware-id=c1.xlarge
> > whirr.image-id=us-east-1/ami-da0cf8b3
> > whirr.location-id=us-east-1
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113110#comment-13113110 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
I just ran this on 0.7.0, and received a different error ( a transient one related to ec2 ssh keys [1], which ended up having us create a 4th node ).
However, things operated did come up without the ssh error you mentioned. How are your ssh keys setup? Do you have a ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub file in correct form? Are you destroying your cluster through whirr or elsewhere?
I'll also try with 0.6.0 now
-- snip --
Running configuration script on nodes: [us-east-1/i-5032a530, us-east-1/i-5232a532, us-east-1/i-5432a534]
Configuration script run completed
Running configuration script on nodes: [us-east-1/i-4c32a52c]
Configuration script run completed
Completed configuration of hadoop
Namenode web UI available at http://ec2-184-72-85-39.compute-1.amazonaws.com:50070
Jobtracker web UI available at http://ec2-184-72-85-39.compute-1.amazonaws.com:50030
Wrote Hadoop site file /Users/adriancole/.whirr/hadoop/hadoop-site.xml
Wrote Hadoop proxy script /Users/adriancole/.whirr/hadoop/hadoop-proxy.sh
Wrote instances file /Users/adriancole/.whirr/hadoop/instances
Started cluster of 4 instances
--snip --
[1] http://code.google.com/p/jclouds/issues/detail?id=696
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113116#comment-13113116 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
ok. setting up whirr on slicehost ubuntu natty per below [1]
note I had to change the referenced setup-whirr.sh to not use the incubating svn repo
java -jar target/compute-basics-jar-with-dependencies.jar slicehost APIKEY APIKEY mygroup add
java -jar target/compute-basics-jar-with-dependencies.jar slicehost APIKEY APIKEY mygroup run ~/Downloads/setup-whirr.sh
[1] http://mail-archives.apache.org/mod_mbox/whirr-dev/201108.mbox/%3CCAARMrHXi1ad5fkGvSYJRYJB7OsDmsOtzLgbR=0G9KNP-FdjBeA@mail.gmail.com%3E
NEXT STEP:
make a properties file and run whirr on this fresh install to see if there's an OS issue, or perhaps a setup one.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113113#comment-13113113 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
so, my machine I am testing on is os/x. I will try this from a cloud node running ubuntu in a minute.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-378) Auth fail when creating a
cluster from an EC2 instance
Posted by "Michael David (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179133#comment-13179133 ]
Michael David edited comment on WHIRR-378 at 1/3/12 11:44 PM:
--------------------------------------------------------------
Hi Andrei - Just tried with 0.7.0 and am getting new errors:
Instead of using the properties file, I tried to invoke keys inline instead like the tuorial here: http://whirr.apache.org/docs/0.7.0/whirr-in-5-minutes.html
./whirr launch-cluster --config hadoop.properties --private-key-file ~/.ssh/id_rsa_whirr --public-key-file ~/.ssh/id_rsa_whirr.pub
It seems that I'm getting a different set of auth errors:
Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
<<kex done>> woke to: net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
<< (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Broken transport; encountered EOF
net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
at net.schmizz.sshj.transport.Reader.run(Reader.java:70)
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): exception not retryable
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<< problem applying options to node(us-east-1/i-xxxxxe):
org.jclouds.rest.AuthorizationException: (USER:rsa[fingerprint(RSA),sha1(SHA)]@xx.xx.xx:22) (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Exhausted available authentication methods
at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
... 9 more
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xxx.xx:22) error acquiring SSHClient(timeout=60000): exception not retryable
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<< (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Exhausted available authentication methods
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
Thanks - really appreciate your speedy answers here!
was (Author: hackwtf):
Hi Andrei - Just tried with 0.7.0 and am getting new errors:
Instead of using the properties file, I tried to invoke keys in command line instead using the tuorial here: http://whirr.apache.org/docs/0.7.0/whirr-in-5-minutes.html
./whirr launch-cluster --config hadoop.properties --private-key-file ~/.ssh/id_rsa_whirr --public-key-file ~/.ssh/id_rsa_whirr.pub
It seems that I'm not getting a different set of auth errors:
Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
<<kex done>> woke to: net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
<< (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Broken transport; encountered EOF
net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
at net.schmizz.sshj.transport.Reader.run(Reader.java:70)
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): exception not retryable
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<< problem applying options to node(us-east-1/i-xxxxxe):
org.jclouds.rest.AuthorizationException: (USER:rsa[fingerprint(RSA),sha1(SHA)]@xx.xx.xx:22) (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Exhausted available authentication methods
at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
... 9 more
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xxx.xx:22) error acquiring SSHClient(timeout=60000): exception not retryable
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<< (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Exhausted available authentication methods
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
Thanks - really appreciate your speedy answers here!
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179077#comment-13179077 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
I'm not 100% sure for 0.5.0 but this is true for 0.6+. BTW you should upgrade to our latest stable release (0.7.0). See: http://www.apache.org/dyn/closer.cgi/whirr/
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-378) Auth fail when creating a cluster from
an EC2 instance
Posted by "Andrei Savu (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu updated WHIRR-378:
------------------------------
Priority: Critical (was: Major)
Fix Version/s: 0.8.0
Marking this as a critical issue for 0.8.0.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Michael David (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179133#comment-13179133 ]
Michael David commented on WHIRR-378:
-------------------------------------
Hi Andrei - Just tried with 0.7.0 and am getting new errors:
Instead of using the properties file, I tried to invoke keys in command line instead using the tuorial here: http://whirr.apache.org/docs/0.7.0/whirr-in-5-minutes.html
./whirr launch-cluster --config hadoop.properties --private-key-file ~/.ssh/id_rsa_whirr --public-key-file ~/.ssh/id_rsa_whirr.pub
It seems that I'm not getting a different set of auth errors:
Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
Dying because - net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
<<kex done>> woke to: net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
<< (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Broken transport; encountered EOF
net.schmizz.sshj.transport.TransportException: Broken transport; encountered EOF
at net.schmizz.sshj.transport.Reader.run(Reader.java:70)
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (USER:rsa[fingerprint(FINGERPRINT),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): exception not retryable
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<< problem applying options to node(us-east-1/i-xxxxxe):
org.jclouds.rest.AuthorizationException: (USER:rsa[fingerprint(RSA),sha1(SHA)]@xx.xx.xx:22) (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Exhausted available authentication methods
at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
... 9 more
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xxx.xx:22) error acquiring SSHClient(timeout=60000): exception not retryable
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
<< (USER:rsa[fingerprint(RSA),sha1(SHA1)]@xx.xx.xx:22) error acquiring SSHClient(timeout=60000): Exhausted available authentication methods
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
Thanks - really appreciate your speedy answers here!
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Paul Baclace (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109128#comment-13109128 ]
Paul Baclace commented on WHIRR-378:
------------------------------------
This is the age of "eventually consistent" which means the app level must deal with inconsistency... but "app" in this case could be jclouds or Whirr, I don't know yet.
This might be a pass-thru issue to jclouds since the timeouts and number of attempts is not settable [sic] in jclouds.
(I have also seen an error about a security group not existing but it was created moments earlier: this one I will post as a separate bug since the problem above can occur independently.)
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Chris Custine (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140313#comment-13140313 ]
Chris Custine commented on WHIRR-378:
-------------------------------------
I have run into this issue as well and have some quick observations. My problems all arose from using whirr.private-key-file and whirr.public-key-file with relative paths like "./key.priv" which seems to work fine with a single instance in the cluster. However when there is more than one instance, the relative paths seem to make some instances fail (but not all). If I add fully qualified paths to the keys, everything works just fine for all instances.
Highly unscientific, but I did not have time to debug further once I figured it out. Seems to be a bug somewhere, but I am not sure where. If I have time again I will look into it.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Paul Baclace (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108972#comment-13108972 ]
Paul Baclace commented on WHIRR-378:
------------------------------------
Here is the stack trace associated with the "problem applying options" error:
2011-09-17 03:38:37,260 DEBUG [jclouds.compute] (user thread 2) << socket [address=204.236.165.200, port=22] opened
2011-09-17 03:38:46,153 ERROR [jclouds.compute] (user thread 2) << problem applying options to node(us-west-1/i-85914fc2):
org.jclouds.rest.AuthorizationException: (ubuntu@204.236.165.200:22) (ubuntu@204.236.165.200:22) error acquiring SSHClient(ubuntu@204.236.165.200:22)
: Exhausted available authentication methods
at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:387)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:214)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:229)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:107)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMa
p.java:150)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMa
p.java:57)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:183)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:155)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:204)
... 9 more
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474) at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127) at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194851#comment-13194851 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
Joris I'm happy it works for you. We are planning to address this unexpected behaviour in 0.7.1.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13109135#comment-13109135 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
You are right. We need to implement automatic retry behavior in many places.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrei Savu resolved WHIRR-378.
-------------------------------
Resolution: Invalid
I will close this as invalid because most of the time this false problem is generated by invalid configuration.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104810#comment-13104810 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
do you know what the login user is for this image? I think jclouds default is to try "ubuntu". If not, you'd want to specify whirr.login-user
http://whirr.apache.org/docs/0.6.0/configuration-guide.html
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221106#comment-13221106 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
I've tried several sizes but all fail to deploy successfully. Its starting the nodes in EC2, but never getting past the setup (just hangs after the following):
Bootstrapping cluster
Configuring template
Configuring template
Starting 1 node(s) with roles [hadoop-datanode, hadoop-tasktracker]
Starting 1 node(s) with roles [hadoop-namenode, hadoop-jobtracker]
This is the first time (in my experience) that it just hangs w/o giving more stack trace errors.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-378) Auth fail when creating a
cluster from an EC2 instance
Posted by "Joris Poort (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221125#comment-13221125 ]
Joris Poort edited comment on WHIRR-378 at 3/2/12 6:15 PM:
-----------------------------------------------------------
Seems like it might have been defining "whirr.bootstrap-user" instead of "whirr.login-user" caused the issue. Seems odd, but right now I've tracked it down to the following: custom AMI (standard OS, private or public) in combination with definition of either ("bootstrap-user" or no "login-user" - not sure which one) specified causes the issue.
Ref: https://issues.apache.org/jira/browse/WHIRR-475
was (Author: gpoort):
Seems like it might have been defining "whirr.bootstrap-user" instead of "whirr.login-user" caused the issue. Seems odd, but right now I've tracked it down to the following: custom AMI (standard OS, private or public) in combination with definition of either ("bootstrap-user" or no "login-user" - not sure which one) specified causes the issue.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193588#comment-13193588 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Still having issues with this too here...
I was able to fix it in os/x through recreating keys but now having issues again on ubuntu vm on amazon. Regenerated keys etc. but still getting this error.
Right now using:
whirr.bootstrap-user=ubuntu
whirr.cluster-user=ubuntu
Andrei - anything else I can try?
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220876#comment-13220876 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
whirr.bootstrap-user is going to be available in 0.8.0. For 0.7.1 you have to use whirr.login-user. Is the cluster deployment failing? (sometimes errors like this can be misleading) Also make sure are you *not* using t1.micro.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Michael David (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179032#comment-13179032 ]
Michael David commented on WHIRR-378:
-------------------------------------
I am seeing the same SSH issue, and I am not running as root. I am running as the default user ec2-user.
It seems as if whirr is creating a new keypair despite the fact that I already have set one in my properties.
I've tried a number of AMIs from various tutorials, and can't seem to get it to ssh in.
whirr version: Whirr 0.5.0-cdh3u2
hadoop version: Hadoop 0.20.2-cdh3u2
I've spent hours trying to get to the bottom of this with no luck so far.
Thanks for any help!
Properties FILE:
--------------------
whirr.cluster-name=myhadoopcluster
whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,1 hadoop-datanode+hadoop-tasktracker
whirr.provider=aws-ec2
whirr.identity=<my actual key>
whirr.credential=<my secret key>
whirr.login-user=ec2-user
whirr.hardware-id=m1.small
whirr.image-id=us-east-1/ami-6936fb00
whirr.private-key-file=/home/ec2-user/.ssh/id_rsa
whirr.public-key-file=/home/ec2-user/.ssh/id_rsa.pub
whirr.hadoop-install-function=install_cdh_hadoop
whirr.hadoop-configure-function=configure_cdh_hadoop
LOGS:
--------------
[org.apache.whirr.actions.NodeStarter] (pool-5-thread-5) Starting 1 node(s) with roles [hadoop-datanode, hadoop-tasktracker]
2012-01-03 21:01:08,903 DEBUG [jclouds.compute] (pool-5-thread-5) >> running 1 node group(myhadoopcluster) location(us-east-1) image(us-east-1/ami-6936fb00) hardwareProfile(m1.small) options([groupIds=[], keyPair=null, noKeyPair=false, monitoringEnabled=false, placementGroup=null, noPlacementGroup=false, subnetId=null, userData=null, blockDeviceMappings=[]])
2012-01-03 21:01:08,904 DEBUG [jclouds.compute] (pool-5-thread-5) >> creating keyPair region(us-east-1) tag(myhadoopcluster)
2012-01-03 21:01:09,245 DEBUG [jclouds.compute] (pool-5-thread-5) << created keyPair(jclouds#myhadoopcluster#us-east-1#70)
2012-01-03 21:01:09,246 DEBUG [jclouds.compute] (pool-5-thread-5) >> running 1 instance region(us-east-1) zone(null) ami(ami-6936fb00) params({InstanceType=[m1.small], SecurityGroup.1=[jclouds#myhadoopcluster#us-east-1], KeyName=[jclouds#myhadoopcluster#us-east-1#70]})
2012-01-03 21:01:09,542 DEBUG [jclouds.compute] (pool-5-thread-5) << started instances(i-xxx)
2012-01-03 21:01:09,610 DEBUG [jclouds.compute] (pool-5-thread-5) << present instances(i-xxx8)
2012-01-03 21:01:24,697 DEBUG [jclouds.compute] (user thread 6) >> blocking on socket [address=02.xx.xx.xx, port=22] for 600000 seconds
2012-01-03 21:01:25,777 DEBUG [jclouds.compute] (user thread 6) << socket [address=01.xx.xx.xx, port=22] opened
2012-01-03 21:01:48,490 DEBUG [jclouds.compute] (user thread 6) >> blocking on socket [address=02.xx.xx.xx, port=22] for 600000 seconds
2012-01-03 21:01:48,493 DEBUG [jclouds.compute] (user thread 6) << socket [address=01.xx.xx.xx, port=22] opened
2012-01-03 21:01:54,292 ERROR [jclouds.compute] (user thread 6) << problem applying options to node(us-east-1/i-901170f2):
org.jclouds.ssh.SshException: ec2-user@xx.xx.xx.xx: Error connecting to session.
at org.jclouds.ssh.jsch.JschSshClient.propagate(JschSshClient.java:251)
at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:205)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:139)
at org.jclouds.compute.strategy.CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(CustomizeNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:55)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: com.jcraft.jsch.JSchException: Auth fail
at com.jcraft.jsch.Session.connect(Session.java:461)
at com.jcraft.jsch.Session.connect(Session.java:154)
at org.jclouds.ssh.jsch.JschSshClient.newSession(JschSshClient.java:246)
at org.jclouds.ssh.jsch.JschSshClient.connect(JschSshClient.java:185)
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113142#comment-13113142 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
OK I think this testing is done. We need more info before taking any more action, as this config is pretty expensive to test!
My guess based on the error, is that there is an issue in ~/.ssh/id_rsa or ~/.ssh/id_rsa.pub, something we can possibly validate prior to use. My suggestion is that the user generate new keys and try having whirr use them, especially as this problem is noted in both ec2 and rackspace.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13140523#comment-13140523 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
Is that behaviour happening also on a different machine? Can you tell us more about you setup? Thanks!
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-378) Auth fail when creating a
cluster from an EC2 instance
Posted by "Joris Poort (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221125#comment-13221125 ]
Joris Poort edited comment on WHIRR-378 at 3/2/12 6:15 PM:
-----------------------------------------------------------
Seems like it might have been defining "whirr.bootstrap-user" instead of "whirr.login-user" caused the issue. Seems odd, but right now I've tracked it down to the following: custom AMI (standard OS, private or public) in combination with definition of either ("bootstrap-user" or no "login-user" - not sure which one) specified causes the issue.
was (Author: gpoort):
Seems like it might have been defining "whirr.bootstrap-user" instead of whirr.login-user caused the issue. Seems odd, but right now I've tracked it down to the following: custom AMI (standard OS, private or public) in combination with definition of either ("bootstrap-user" or no "login-user" - not sure which one) specified causes the issue.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Paul Baclace (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108253#comment-13108253 ]
Paul Baclace commented on WHIRR-378:
------------------------------------
I see this issue too (in 0.6.0), as far as I can tell from the description, but the upshot is some nodes are deleted as dead on arrival and more nodes are allocated so the cluster is successfully created. BUT I am charged for 1 hour of time on each apparent DOA node.
One run and found that 2 out of 5 nodes were seemingly dead on arrival (I have many examples from the same day) That is a high failure rate, so I wonder whether it was a false positive DOA. A summary of the trimmed whirr.log below (last 3 digits of i-number):
1. starting 3 instances/nodes (fbe, fc0, fc2) at 3:37:19
2. problem with a node (fc2) at 3:38:46 or 87 sec. after node start
3. starting a new instance/node (01c) at 3:40:14
4. problem with a another node (01c) at 3:41:19, or 65sec after node start
5. start a new instance/node (040) at 3:41:22
6. delete nodes (01c, fc2) at 3:44:34
The most caused-by ssh error is "net.schmizz.sshj.userauth.UserAuthException: publickey auth failed".
It looks like the overall error "problem applying options to node" is occurring 10 seconds after opening the socket, so that node is alive to some extent and it does not appear to be an ssh timeout. That this happens about 1 minute after instance start makes me think there could be an implicit timer awaiting boot-up. (These instances are all using the same private ami from instance-store and no EBS volumes.)
The failed nodes appear to be deleted after sufficient nodes are started up, not when they are determined to be failed. Looking at billing records, I noticed that I *am* being charged for these failed nodes, so I think this is an important bug to fix.
-----whirr.log excerpt-------
03:37:19,043 DEBUG [jclouds.compute] << started instances([region=us-west-1, name=i-f9914fbe])
03:37:19,133 DEBUG [jclouds.compute] << present instances([region=us-west-1, name=i-f9914fbe])
03:37:19,332 DEBUG [jclouds.compute] << started instances([region=us-west-1, name=i-87914fc0],[region=us-west-1, name=i-85914fc2])
03:37:19,495 DEBUG [jclouds.compute] << present instances([region=us-west-1, name=i-87914fc0],[region=us-west-1, name=i-85914fc2])
03:38:46,153 ERROR [jclouds.compute] << problem applying options to node(us-west-1/i-85914fc2)
03:40:14,460 DEBUG [jclouds.compute] << started instances([region=us-west-1, name=i-5b8e501c])
03:40:14,547 DEBUG [jclouds.compute] << present instances([region=us-west-1, name=i-5b8e501c])
03:41:19,691 ERROR [jclouds.compute] << problem applying options to node(us-west-1/i-5b8e501c)
03:41:22,738 DEBUG [jclouds.compute] << started instances([region=us-west-1, name=i-078e5040])
03:41:22,831 DEBUG [jclouds.compute] << present instances([region=us-west-1, name=i-078e5040])
03:44:34,257 INFO [org.apache.whirr.actions.BootstrapClusterAction] Deleting failed node node us-west-1/i-5b8e501c
03:44:34,259 INFO [org.apache.whirr.actions.BootstrapClusterAction] Deleting failed node node us-west-1/i-85914fc2
03:46:27,948 INFO [org.apache.whirr.service.FileClusterStateStore] (main) Wrote instances file instances
The instances file ends up containing: i-f9914fbe i-87914fc0 i-078e5040
And not containing: i-5b8e501c i-85914fc2
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113137#comment-13113137 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
works fine. I'll update the issue with the commands I used to create the cluster.
should try the following in case your keys aren't working right:
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13194370#comment-13194370 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Thanks Andrei - I was using a custom AMI but your explanation/tips helped fix it, really appreciate it :).
Joris
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108985#comment-13108985 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
Do you think this is a timing issue? Maybe the keys are not installed on the remote machine when jclouds tries to connect using SSH.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104805#comment-13104805 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
I've been having the same issue - did you come up with a temporary workaround for this?
Cheers - Joris
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220740#comment-13220740 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Having some issues with this again - not sure if its from 0.7.1 but we just transitioned to this latest version.
2012-03-01 22:54:24,843 ERROR [net.schmizz.sshj.transport.TransportImpl] (reader) Dying because - java.net.SocketTimeoutException: Read timed out
2012-03-01 22:54:30,986 ERROR [net.schmizz.sshj.transport.TransportImpl] (reader) Dying because - java.net.SocketTimeoutException: Read timed out
I started with a working configuration (standard Ubuntu 10.04) then created a custom AMI (without changing anything) and tried to boot the cluster.
I'm using:
whirr.bootstrap-user=ubuntu
whirr.cluster-user=whirr
jclouds.ec2.ami-query=owner-id=[ID]
Any ideas on what I can try out?
Thanks in advance!
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13193697#comment-13193697 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
whirr.bootstrap-user should be != than whirr.cluster-user
Unless you are using a custom AMI most of the time you don't need to specify a value for whirr.bootstrap-user.
*Important* whirr.cluster-user should not already exist on the remote machine. Try something like whirr.cluster-user=whirr or something else unique.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Carlos González-Cadenas (Commented JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13165316#comment-13165316 ]
Carlos González-Cadenas commented on WHIRR-378:
-----------------------------------------------
Hello,
I'm running your setup and run scripts attached but I cannot successfully launch a cluster on EC2.
First I launch in US-EAST-1 a Natty image (ami-fd589594). Then I run the setup script and then the run-test script.
After creating the instances, whirr starts giving errors and retrying with no success (see below). Can you help us with this?
Thanks
Carlos
Running configuration script on nodes: [us-east-1/i-c70415a4, us-east-1/i-c50415a6, us-east-1/i-cb0415a8]
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
Dying because - java.net.SocketTimeoutException: Read timed out
Dying because - java.net.SocketTimeoutException: Read timed out
<<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
<< (ubuntu@50.16.72.110:22) error acquiring SSHClient(ubuntu@50.16.72.110:22): Exhausted available authentication methods
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:183)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:155)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:204)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:229)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:107)
at org.jclouds.compute.strategy.RunScriptOnNodeAndAddToGoodMapOrPutExceptionIntoBadMap.call(RunScriptOnNodeAndAddToGoodMapOrPutExceptionIntoBadMap.java:69)
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177298#comment-13177298 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
Christian please make sure you are not running Whirr as root.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104829#comment-13104829 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
huh. ok, well let's leave this issue in. I should have a chance to test this w/debugger soon.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-378) Auth fail when creating a
cluster from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113110#comment-13113110 ]
Adrian Cole edited comment on WHIRR-378 at 9/23/11 2:20 AM:
------------------------------------------------------------
I just ran this on 0.7.0, and received a different error ( a transient one related to ec2 ssh keys [1] ).
However, things operated did come up without the ssh error you mentioned. How are your ssh keys setup? Do you have a ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub file in correct form? Are you destroying your cluster through whirr or elsewhere?
I'll also try with 0.6.0 now
-- snip --
Running configuration script on nodes: [us-east-1/i-5032a530, us-east-1/i-5232a532, us-east-1/i-5432a534]
Configuration script run completed
Running configuration script on nodes: [us-east-1/i-4c32a52c]
Configuration script run completed
Completed configuration of hadoop
Namenode web UI available at http://ec2-184-72-85-39.compute-1.amazonaws.com:50070
Jobtracker web UI available at http://ec2-184-72-85-39.compute-1.amazonaws.com:50030
Wrote Hadoop site file /Users/adriancole/.whirr/hadoop/hadoop-site.xml
Wrote Hadoop proxy script /Users/adriancole/.whirr/hadoop/hadoop-proxy.sh
Wrote instances file /Users/adriancole/.whirr/hadoop/instances
Started cluster of 4 instances
--snip --
[1] http://code.google.com/p/jclouds/issues/detail?id=696
was (Author: adrian@jclouds.org):
I just ran this on 0.7.0, and received a different error ( a transient one related to ec2 ssh keys [1], which ended up having us create a 4th node ).
However, things operated did come up without the ssh error you mentioned. How are your ssh keys setup? Do you have a ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub file in correct form? Are you destroying your cluster through whirr or elsewhere?
I'll also try with 0.6.0 now
-- snip --
Running configuration script on nodes: [us-east-1/i-5032a530, us-east-1/i-5232a532, us-east-1/i-5432a534]
Configuration script run completed
Running configuration script on nodes: [us-east-1/i-4c32a52c]
Configuration script run completed
Completed configuration of hadoop
Namenode web UI available at http://ec2-184-72-85-39.compute-1.amazonaws.com:50070
Jobtracker web UI available at http://ec2-184-72-85-39.compute-1.amazonaws.com:50030
Wrote Hadoop site file /Users/adriancole/.whirr/hadoop/hadoop-site.xml
Wrote Hadoop proxy script /Users/adriancole/.whirr/hadoop/hadoop-proxy.sh
Wrote instances file /Users/adriancole/.whirr/hadoop/instances
Started cluster of 4 instances
--snip --
[1] http://code.google.com/p/jclouds/issues/detail?id=696
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Christian Mauceri (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177302#comment-13177302 ]
Christian Mauceri commented on WHIRR-378:
-----------------------------------------
Thanks Andrei :) Once again if I can help, let me know.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (WHIRR-378) Auth fail when creating a cluster from
an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole updated WHIRR-378:
------------------------------
Attachment: setup-whirr.sh
run-test.sh
scripts ran to setup and run the cluster with the parameters in this issue
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Christian Mauceri (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177246#comment-13177246 ]
Christian Mauceri commented on WHIRR-378:
-----------------------------------------
Hi, because it is quite important to me to launch Whirr from the clouds, I tried on Rackspace and I got a similar error, see below.
I do not know if it helps but apparently the issue is not restricted to ec2.
If this is not the place to post such information, please forgive me.
If I can help, let me know. I'm very frustrated, it is not possible to use Whirr from the clouds.
Cheers.
Scripts(ScriptBasedClusterAction.java:215)
at org.apache.whirr.actions.ScriptBasedClusterAction.doAction(ScriptBasedClusterAction.java:128)
at org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107)
at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:109)
at org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
at org.apache.whirr.cli.Main.run(Main.java:64)
at org.apache.whirr.cli.Main.main(Main.java:97)
Caused by: org.jclouds.rest.AuthorizationException:
(root:rsa[fingerprint(70:65:c2:67:ba:e4:6d:34:e7:85:36:d3:8f:e0:37:fb),sha1(f6:39:75:1f:08:46:5d:d8:b0:69:a9:0b:ed:a4:0d:2c:32:fd:ad:c8)]@50.56.237.81:22)
(root:rsa[fingerprint(70:65:c2:67:ba:e4:6d:34:e7:85:36:d3:8f:e0:37:fb),sha1(f6:39:75:1f:08:46:5d:d8:b0:69:a9:0b:ed:a4:0d:2c:32:fd:ad:c8)]@50.56.237.81:22)
error acquiring SSHClient(timeout=60000): Exhausted available
authentication methods
at org.jclouds.sshj.SshjSshClient.propagate(SshjSshClient.java:413)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:244)
at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:255)
at org.jclouds.compute.callables.RunScriptOnNodeAsInitScriptUsingSsh.call(RunScriptOnNodeAsInitScriptUsingSsh.java:89)
at org.jclouds.compute.internal.BaseComputeService.runScriptOnNode(BaseComputeService.java:612)
at org.apache.whirr.actions.ScriptBasedClusterAction$2.call(ScriptBasedClusterAction.java:190)
at org.apache.whirr.actions.ScriptBasedClusterAction$2.call(ScriptBasedClusterAction.java:178)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)
Caused by: net.schmizz.sshj.userauth.UserAuthException: Exhausted
available authentication methods
at net.schmizz.sshj.userauth.UserAuthImpl.authenticate(UserAuthImpl.java:114)
at net.schmizz.sshj.SSHClient.auth(SSHClient.java:204)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:304)
at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:323)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:199)
at org.jclouds.sshj.SshjSshClient$1.create(SshjSshClient.java:171)
at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:220)
... 10 more
Caused by: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
at net.schmizz.sshj.userauth.UserAuthImpl.handle(UserAuthImpl.java:157)
at net.schmizz.sshj.transport.TransportImpl.handle(TransportImpl.java:474)
at net.schmizz.sshj.transport.Decoder.decode(Decoder.java:127)
at net.schmizz.sshj.transport.Decoder.received(Decoder.java:195)
at net.schmizz.sshj.transport.Reader.run(Reader.java:72)
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Adrian Cole reassigned WHIRR-378:
---------------------------------
Assignee: Adrian Cole
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Adrian Cole (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113111#comment-13113111 ]
Adrian Cole commented on WHIRR-378:
-----------------------------------
run fine on 0.6.0 with the same properties as are in the issue. I suspect this is actually a content-related problem with your ssh keypair.
can you try creating a new key and using that instead? this is detailed here:
http://whirr.apache.org/docs/0.6.0/whirr-in-5-minutes.html
-- snip --
Running configuration script on nodes: [us-east-1/i-aa21b6ca, us-east-1/i-ac21b6cc, us-east-1/i-ae21b6ce]
Configuration script run completed
Running configuration script on nodes: [us-east-1/i-a421b6c4]
Configuration script run completed
Completed configuration of hadoop
Namenode web UI available at http://ec2-50-16-93-2.compute-1.amazonaws.com:50070
Jobtracker web UI available at http://ec2-50-16-93-2.compute-1.amazonaws.com:50030
Wrote Hadoop site file /Users/adriancole/.whirr/hadoop/hadoop-site.xml
Wrote Hadoop proxy script /Users/adriancole/.whirr/hadoop/hadoop-proxy.sh
Wrote instances file /Users/adriancole/.whirr/hadoop/instances
Started cluster of 4 instances
-- snip --
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Michael David (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179067#comment-13179067 ]
Michael David commented on WHIRR-378:
-------------------------------------
Thanks for this. I will give it a go. I thought that Whirr does NOT create a new user on the remote server. Does it?
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177322#comment-13177322 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
No worries. Let us know if you find any other issues.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (WHIRR-378) Auth fail when creating a
cluster from an EC2 instance
Posted by "Andrei Savu (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13220876#comment-13220876 ]
Andrei Savu edited comment on WHIRR-378 at 3/2/12 12:39 PM:
------------------------------------------------------------
whirr.bootstrap-user is going to be available in 0.8.0. For 0.7.1 you have to use whirr.login-user. Is the cluster deployment failing? (sometimes errors like this can be misleading) Also make sure you are *not* using t1.micro.
was (Author: savu.andrei):
whirr.bootstrap-user is going to be available in 0.8.0. For 0.7.1 you have to use whirr.login-user. Is the cluster deployment failing? (sometimes errors like this can be misleading) Also make sure are you *not* using t1.micro.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221125#comment-13221125 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Seems like it might have been defining "whirr.bootstrap-user" instead of whirr.login-user caused the issue. Seems odd, but right now I've tracked it down to the following: custom AMI (standard OS, private or public) in combination with definition of either ("bootstrap-user" or no "login-user" - not sure which one) specified causes the issue.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104904#comment-13104904 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Just tried this on rackspace, same issues. So it seems to be independent of the provider.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Andrei Savu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13179065#comment-13179065 ]
Andrei Savu commented on WHIRR-378:
-----------------------------------
Michael try to add:
{noformat}
whirr.cluster-user=something-different-than-ec2-user
{noformat}
You should be able to login using:
{noformat}
ssh -i /home/ec2-user/.ssh/id_rsa whirr.cluster-user@host
{noformat}
I am trying this workaround now.
I will create soon a patch to address the following issues:
* fail fast when running Whirr as root
* do not fail if a remote user with the same name as whirr.cluster-user already exists
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Christian Mauceri (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177305#comment-13177305 ]
Christian Mauceri commented on WHIRR-378:
-----------------------------------------
Andrei I was root I try again.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Christian Mauceri (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13177316#comment-13177316 ]
Christian Mauceri commented on WHIRR-378:
-----------------------------------------
Ho Andrei, I'm so ashamed and so happy. Thank you so much, it works on Rackspace ! Next step Mahout. Sorry for thr disturbance.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
> Assignee: Adrian Cole
> Priority: Critical
> Fix For: 0.8.0
>
> Attachments: run-test.sh, setup-whirr.sh
>
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104833#comment-13104833 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Ok, great thanks so much. Let me know if there's anything else you'd like me to test. I can recreate the issue repeatedly right now on my configuration. (very similar to original poster - EC2, Ubuntu, CDH)
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (WHIRR-378) Auth fail when creating a cluster
from an EC2 instance
Posted by "Joris Poort (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/WHIRR-378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104826#comment-13104826 ]
Joris Poort commented on WHIRR-378:
-----------------------------------
Thanks Adrian - tried this out, but unfortunately didn't make a difference for me.
I've had the same issue come up with a configuration that previously worked. I thought that it may be something on the Amazon side, but haven't figured it out.
> Auth fail when creating a cluster from an EC2 instance
> ------------------------------------------------------
>
> Key: WHIRR-378
> URL: https://issues.apache.org/jira/browse/WHIRR-378
> Project: Whirr
> Issue Type: Bug
> Components: service/hadoop
> Affects Versions: 0.6.0
> Reporter: Marc de Palol
>
> There is a ssh auth problem when creating a hadoop cluster from an EC2 ubuntu instance.
> I've been using the same configuration file from an EC2 computer an a physical one, everything works fine in the physical one, but I keep getting this error in EC2:
> Running configuration script on nodes: [us-east-1/i-c7fde5a6, us-east-1/i-c9fde5a8, us-east-1/i-cbfde5aa]
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> <<authenticated>> woke to: net.schmizz.sshj.userauth.UserAuthException: publickey auth failed
> The user in the virtual machine is new and with valid .ssh keys.
> The hadoop config file is (omitting commented lines):
> whirr.cluster-name=hadoop
> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,3 hadoop-datanode+hadoop-tasktracker
> whirr.provider=aws-ec2
> whirr.identity=****
> whirr.credential=****
> whirr.hardware-id=c1.xlarge
> whirr.image-id=us-east-1/ami-da0cf8b3
> whirr.location-id=us-east-1
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira