You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by "Anton Dmitriev (JIRA)" <ji...@apache.org> on 2018/08/30 12:13:00 UTC
[jira] [Created] (IGNITE-9432) Investigate ability to make
ScanQuery faster
Anton Dmitriev created IGNITE-9432:
--------------------------------------
Summary: Investigate ability to make ScanQuery faster
Key: IGNITE-9432
URL: https://issues.apache.org/jira/browse/IGNITE-9432
Project: Ignite
Issue Type: Improvement
Components: ml
Affects Versions: 2.7
Reporter: Anton Dmitriev
Fix For: 2.7
When we make ScanQuery via Binary Client Protocol it works very slowly in case we need to retrieve a big amount of objects (all objects in our case). The most slower part is a waiting for response from Apache Ignite. This also mentioned on devlist [http://apache-ignite-developers.2346864.n4.nabble.com/How-to-reduce-Scan-Query-execution-time-td34212.html].
To reproduce the problem we've prepared a small example: [slow-scan-query-reproducer|https://github.com/dmitrievanthony/slow-scan-query-reproducer]. In this example we creates a cache with 500 objects 1Mb each (on localhost), we make ScanQuery with different page sizes and calculate time between the moment when the request has been sent and the moment when the response is ready to be receive. The measurements are here:
{{Page size 5 Mb, waiting time 119.85 ± 6.72 ms}}
{{ Page size 10 Mb, waiting time 157.70 ± 15.35 ms}}
{{ Page size 20 Mb, waiting time 204.50 ± 19.18 ms}}
{{ Page size 50 Mb, waiting time 264.70 ± 22.30 ms}}
{{ Page size 100 Mb, waiting time 463.35 ± 17.12 ms}}
{{ Page size 150 Mb, waiting time 672.50 ± 21.98 ms}}
As result we spend ~4ms per every megabyte on _something_. It means that we will be able to achieve 250Mb/s throughput in best case. It's too slow, isn't it?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)