You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Vladimir Ozerov (JIRA)" <ji...@apache.org> on 2017/10/23 08:07:00 UTC
[jira] [Created] (IGNITE-6699) Optimize client-side data streamer
performance
Vladimir Ozerov created IGNITE-6699:
---------------------------------------
Summary: Optimize client-side data streamer performance
Key: IGNITE-6699
URL: https://issues.apache.org/jira/browse/IGNITE-6699
Project: Ignite
Issue Type: Task
Security Level: Public (Viewable by anyone)
Components: streaming
Affects Versions: 2.3
Reporter: Vladimir Ozerov
Fix For: 2.4
Currently if a user has several server nodes and a single client node with single thread pushing data to streamer, he will not be able to load data at maximum speed. On the other hand, if he start several data loading threads, throughput will increase.
One of root causes of this is bad data streamer design. Method {{IgniteDataStreamer.addData(K, V)}} returns new feature for every operation, this is too fine grained approach. Also it generates a lot of garbage and causes contention on streamer internals.
Proposed implementation flow:
1) Compare performance of {{addData(K, V)}} vs {{addData(Collection)}} methods from one thread in distributed environment. The latter should show considerably higher throughput.
2) Users should receive per-batch features, rather than per-key.
3) Try caching thread data in some collection until it is large enough to avoid contention and unnecessary allocations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)