You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by GitBox <gi...@apache.org> on 2022/12/17 17:29:29 UTC
[GitHub] [mina-sshd] tomaswolf opened a new issue, #293: Performance problems in tests (NIO2, MINA, ubuntu)
tomaswolf opened a new issue, #293:
URL: https://github.com/apache/mina-sshd/issues/293
Looking through the GitHub build logs, there seems to be a problem with two tests on Linux, with the NIO2 or MINA transport back-ends. They show very poor performance:
**org.apache.sshd.sftp.client.SftpTransferTest**
```
Build NIO2 MINA Netty
ubuntu, java 8 ~300s ~300s ~70s
ubuntu, java 11 ~300s ~300s ~50s
ubuntu, java 17 ~300s ~300s ~50s
windows, java 8 ~62s ~67s ~66s
windows, java 11 ~49s ~49s ~43s
windows, java 17 ~43s ~44s ~45s
```
**org.apache.sshd.scp.client.ScpTest**
```
Build NIO2 MINA Netty
ubuntu, java 8 ~19s ~19s ~11s
ubuntu, java 11 ~19s ~19s ~11s
ubuntu, java 17 ~19s ~19s ~11s
windows, java 8 ~12s ~12s ~12s
windows, java 11 ~12s ~12s ~12s
windows, java 17 ~12s ~12s ~12s
```
These are the only tests for which I see such discrepancies. It looks like the code is doing something that doesn't play well with NIO2 or MINA on Linux.
It's not a new problem either; I could find this pattern also in the [build logs of 2022-10-25](https://github.com/apache/mina-sshd/actions/runs/3318464369/jobs/5482485034) (commit 7ad3eadef) and also in even older builds. Though e.g. in a [build from 2022-09-20](https://github.com/apache/mina-sshd/actions/runs/3089979040/jobs/4998289149) the differences are smaller (but still there; 120s instead of 300s, and 15s instead of 19s).
The windows times and the Linux/Netty times appear to be about normal and are what I also see for NIO2 or MINA when building locally on OS X: 12s for ScpTest and about a minute for the SftpTransferTest.
What is going on here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For additional commands, e-mail: dev-help@mina.apache.org
[GitHub] [mina-sshd] tomaswolf commented on issue #293: Performance problems in tests (NIO2 and MINA, Linux only)
Posted by GitBox <gi...@apache.org>.
tomaswolf commented on issue #293:
URL: https://github.com/apache/mina-sshd/issues/293#issuecomment-1374645896
The problem in SFTP is caused by excessively jumping back and forth in the file on the server side. It manifests only with NIO2 and MINA but not with Netty because timing is different and the files end up being read more or less sequentially, or the backwards jumps are within some OS buffer size. With NIO2 and MINA, more initial read requests appear to be handled before a backwards jump occurs, which then leads to more and larger backwards jumps if one keeps issuing asynchronous requests.
I did not analyze why the problem did not occur on Windows or locally on OS X; perhaps these OSes handle this better than Linux, or the request timings and their handling are more like in the Netty/Linux case.
I still have no idea why the problem occurs from a particular GitHub CI build on. Perhaps something in the Linux container used to run the tests changed. In any case I have not found any particular commit from which on the performance would have dropped markedly on Linux; the problem was apparent in all runs in my local CentOS 7 VM, no matter how far back I went.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For additional commands, e-mail: dev-help@mina.apache.org
[GitHub] [mina-sshd] tomaswolf commented on issue #293: Performance problems in tests (NIO2, MINA, ubuntu)
Posted by GitBox <gi...@apache.org>.
tomaswolf commented on issue #293:
URL: https://github.com/apache/mina-sshd/issues/293#issuecomment-1356404731
Partial problem cause: in the **SftpTransferTest**, the problem is the SFTP buffer size of 64kB in `testTransferIntegrityWithBufferLargerThanPacket`. That is 1kB more than the maximum the server will ever return; its setting for the maximum chunk to return is 63kB. This means the client fetches the missing 1kB synchronously, which makes transfer times explode to 120s when running this test locally on a CentOS 7 virtual machine, as opposed to about 14 seconds with the default SFTP buffer size. If the buffer size is 62kB, this test runs locally for me also fast in 10 seconds.
I had pointed that out before in [SSHD-1287](https://issues.apache.org/jira/browse/SSHD-1287). There is clearly room for improvements there.
This test is run twice during `SftpTransferTest`, and the overall run time of this test class is about 300s also locally on CentOS.
This does not explain yet:
* why there is no such problem on Windows, nor with Netty on Linux. (Or at least, why the time doesn't increase that much.)
* why the overall time for that test increased from 120s in September to 300s in December. Some bisecting is needed.
* where the far less marked but still noticeable difference in **ScpTest** comes from.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For additional commands, e-mail: dev-help@mina.apache.org
[GitHub] [mina-sshd] tomaswolf closed issue #293: Performance problems in tests (NIO2 and MINA, Linux only)
Posted by GitBox <gi...@apache.org>.
tomaswolf closed issue #293: Performance problems in tests (NIO2 and MINA, Linux only)
URL: https://github.com/apache/mina-sshd/issues/293
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For additional commands, e-mail: dev-help@mina.apache.org
[GitHub] [mina-sshd] tomaswolf commented on issue #293: Performance problems in tests (NIO2 and MINA, Linux only)
Posted by GitBox <gi...@apache.org>.
tomaswolf commented on issue #293:
URL: https://github.com/apache/mina-sshd/issues/293#issuecomment-1369168078
This is also reproducible for me in a CentOS 7 virtual machine running locally. From the logs I see that with NIO2 and MINA, there is a 40ms delay for getting these extra 1kB chunks, which isn't present in the Netty logs. But I have no idea yet where these 40ms come from.
Bisecting did not yield any insights.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@mina.apache.org
For additional commands, e-mail: dev-help@mina.apache.org