[jira] [Created] (JCLOUDS-1103) SSHJ public key authentication fails sometimes with "Exhausted available authentication methods" message

["Josef Cacek (JIRA)" <ji...@apache.org>]

Josef Cacek created JCLOUDS-1103:
------------------------------------

             Summary: SSHJ public key authentication fails sometimes with "Exhausted available authentication methods" message
                 Key: JCLOUDS-1103
                 URL: https://issues.apache.org/jira/browse/JCLOUDS-1103
             Project: jclouds
          Issue Type: Bug
    Affects Versions: 2.0.0
         Environment: We hit this on OpenStack, but IMO it's not related to the provider used.
            Reporter: Josef Cacek
            Priority: Critical


Connecting with SSHJ SshClient sometimes fails with "Exhausted available authentication methods" message.

It can be caused by the [issue#153|https://github.com/hierynomus/sshj/issues/153] reported on sshj GitHub.

In our tests we run following scenario on Openstack:
1. start a Node from an Image
2. make some checks (e.g. ports are open)
3. suspend() the Node and wait till its suspended
4. resume() the Node and wait till its ready (running and ports are open again)
5. get SSH client by calling
{code}
SshClient sshClient = getComputeServiceContext().utils().sshForNode().apply(nodeMetadata);
sshClient.connect();
{code}

Sometimes it fails to connect the SSH with exception:

{code}
08:27:48.255 [main] ERROR jclouds.ssh - << (cloud-user:rsa[fingerprint(11:50:0c:5f:c9:a4:bc:c0:85:1d:08:d9:9c:d5:b9:52),sha1(ff:aa:5a:8c:80:7f:04:b7:6c:90:b5:dd:91:c4:8e:ea:4f:c3:e7:9f)]@10.8.50.241:22) error acquiring {hostAndPort=10.8.50.241:22, loginUser=cloud-user, ssh=null, connectTimeout=60000, sessionTimeout=60000} (out of retries - max 1): Exhausted available authentication methods
net.schmizz.sshj.userauth.UserAuthException: Exhausted available authentication methods
    at net.schmizz.sshj.SSHClient.auth(SSHClient.java:217) ~[sshj-0.12.0.jar:na]
    at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:316) ~[sshj-0.12.0.jar:na]
    at net.schmizz.sshj.SSHClient.authPublickey(SSHClient.java:335) ~[sshj-0.12.0.jar:na]
    at org.jclouds.sshj.SSHClientConnection.create(SSHClientConnection.java:163) ~[jclouds-sshj-2.0.0-eapqe.4.jar:2.0.0-eapqe.4]
    at org.jclouds.sshj.SSHClientConnection.create(SSHClientConnection.java:49) ~[jclouds-sshj-2.0.0-eapqe.4.jar:2.0.0-eapqe.4]
    at org.jclouds.sshj.SshjSshClient.acquire(SshjSshClient.java:195) [jclouds-sshj-2.0.0-eapqe.4.jar:2.0.0-eapqe.4]
    at org.jclouds.sshj.SshjSshClient.connect(SshjSshClient.java:225) [jclouds-sshj-2.0.0-eapqe.4.jar:2.0.0-eapqe.4]
    at ...
{code}

My guess is about 5% of the scenario runs is failing.

We have also a retry-if-fails logic in our code to prevent situations when the OpenSSH server in the Node is not yet running. So it tries to get the SSH client 12-times with 5 seconds sleeps following every unsuccessful connect. These retries don't help to solve this issue.

We've tried to update SSHJ to latest released version {{0.15.0}}, but we still hit this issue.

When we used *JSCH* implementation (module {{JschSshClientModule}})  instead of the SSHJ (module {{SshjSshClientModule}})   we *didn't see this problem*.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)