You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by "Pavel Pohner (Jira)" <ji...@apache.org> on 2021/06/01 07:52:00 UTC
[jira] [Commented] (SSHD-1173) SFTP worker threads got stuck while
processing PUT methods against one specific SFTP server
[ https://issues.apache.org/jira/browse/SSHD-1173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17354912#comment-17354912 ]
Pavel Pohner commented on SSHD-1173:
------------------------------------
It might be the cause indeed! Thanks for your suggestion, I don't think I'm able to verify it directly though, as I have no access to the target server (it's owned by third party and used through our software which implements Mina, so I don't have credentials and if I understand correctly window size is negotiated after the authentication is done), but I will definitely try to look into it. Are there any "less probable" causes than this one, that are worth to check?
> SFTP worker threads got stuck while processing PUT methods against one specific SFTP server
> -------------------------------------------------------------------------------------------
>
> Key: SSHD-1173
> URL: https://issues.apache.org/jira/browse/SSHD-1173
> Project: MINA SSHD
> Issue Type: Question
> Affects Versions: 2.6.0
> Reporter: Pavel Pohner
> Priority: Critical
> Labels: SFTP, mina, sshd
>
> Hey guys,
> Recently I've encountered one remote SFTP server, that causes SFTP worker threads to enter TIMED_WAITING state, from which they do not recover. Please, take a look into this thread dump:
> {code:java}
> SFTP-worker-3 #2155 prio=5 os_prio=0 cpu=61692.09ms elapsed=1136151.86s tid=0x00007fe41c005800 nid=0x18808 in Object.wait() [0x00007fe145b1d000]SFTP-worker-3" #2155 prio=5 os_prio=0 cpu=61692.09ms elapsed=1136151.86s tid=0x00007fe41c005800 nid=0x18808 in Object.wait() [0x00007fe145b1d000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(java.base@11.0.10/Native Method) - waiting on <no object reference available> at org.apache.sshd.common.future.DefaultSshFuture.await0(DefaultSshFuture.java:69) - waiting to re-lock in wait() <0x00000006e3bbc420> (a org.apache.sshd.common.channel.IoWriteFutureImpl) at org.apache.sshd.common.future.AbstractSshFuture.verifyResult(AbstractSshFuture.java:109) at org.apache.sshd.common.io.AbstractIoWriteFuture.verify(AbstractIoWriteFuture.java:39) at org.apache.sshd.common.io.AbstractIoWriteFuture.verify(AbstractIoWriteFuture.java:30) at org.apache.sshd.common.future.VerifiableFuture.verify(VerifiableFuture.java:43) at org.apache.sshd.sftp.client.impl.DefaultSftpClient.send(DefaultSftpClient.java:292) at org.apache.sshd.sftp.client.impl.SftpOutputStreamAsync.flush(SftpOutputStreamAsync.java:210) at org.apache.sshd.sftp.client.impl.SftpOutputStreamAsync.write(SftpOutputStreamAsync.java:141) at java.io.InputStream.transferTo(java.base@11.0.10/InputStream.java:705) at com.mina.command.Put.replyInto(Put.java:55) at com.sftpserver.MinaSftpHandler.channelReadInternal(MinaSftpHandler.java:251) at com.sftpserver.MinaSftpHandler.lambda$channelRead$0(MinaSftpHandler.java:125) at com.sftpserver.MinaSftpHandler$$Lambda$392/0x0000000800764040.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(java.base@11.0.10/Executors.java:515) at java.util.concurrent.FutureTask.run(java.base@11.0.10/FutureTask.java:264) at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.10/ThreadPoolExecutor.java:1128) at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.10/ThreadPoolExecutor.java:628) at java.lang.Thread.run(java.base@11.0.10/Thread.java:834)
> {code}
> Seems like the thread is waiting for lock in org.apache.sshd.common.future.DefaultSshFuture.await0(DefaultSshFuture.java), I'm currently not sure what thread holds the lock and why it's not ever released.
> Also, I'm not able to reproduce this, but it constantly happens to all the PUT methods targeting one specific remote server (which I don't own), so I suppose there would be something odd with the remote server, but it doesn't mean, that we shouldn't be able to deal with that.
> Have you ever seen such behavior? Could it be Mina sshd's bug? (seems like verify() in org.apache.sshd.sftp.client.impl.DefaultSftpClient.send(DefaultSftpClient.java:292) is called without any timeout, which then defaults to LONG.MAX here: at org.apache.sshd.common.future.VerifiableFuture.verify(VerifiableFuture.java:43), shouldn't we have configurable timeout here? or what's the point of default timeout? what are we really waiting for in at org.apache.sshd.common.future.DefaultSshFuture.await0(DefaultSshFuture.java:69)?)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For additional commands, e-mail: dev-help@mina.apache.org