You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Denis Chudov (Jira)" <ji...@apache.org> on 2022/04/05 09:23:00 UTC
[jira] [Comment Edited] (IGNITE-16760) Performance degradation of IgniteTables#tables after configuration changes
[ https://issues.apache.org/jira/browse/IGNITE-16760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17517327#comment-17517327 ]
Denis Chudov edited comment on IGNITE-16760 at 4/5/22 9:22 AM:
---------------------------------------------------------------
The growth of duration of IgniteTables#tables method happens due to growth of records count having prefix "table.tables" stored in distributed configuration. They need to be read for retrieving all tables' information. See {{TableManager#directTableIds}} which eventually calls {{DistributedConfigurationStorage#readAllLatest}}.
There are two problems:
- cursor which is created for {{readAllLatest}} retrieves records one by one, requesting the metastorage raft group on each {{next()}} call and receiving SingleEntryResponse, see processing of CursorNextCommand in MetaStorageListener. We can implement batch responses to reduce the count of network hops;
- for this particular case, there are too many records with tombstones that are read but seem to be useless. The cursor could skip them, possibly we can introduce an option for creating cursor on range which should skip tombstones.
Also, the impact of such problems can be reduced when metastorage compaction will be implemented.
was (Author: denis chudov):
The growth of duration of IgniteTables#tables method happens due to growth of records count having prefix "table.tables" stored in distributed configuration. They need to be read for retrieving all tables' information. See `TableManager#directTableIds` which eventually calls `DistributedConfigurationStorage#readAllLatest`.
There are two problems:
- cursor which is created for `readAllLatest` retrieves records one by one, requesting the metastorage raft group on each `next()` call and receiving `SingleEntryResponse`, see processing of CursorNextCommand in MetaStorageListener. We can implement batch responses to reduce the count of network hops;
- for this particular case, there are too many records with tombstones that are read but seem to be useless. The cursor could skip them, possibly we can introduce an option for creating cursor on range which should skip tombstones.
Also, the impact of such problems can be reduced when metastorage compaction will be implemented.
> Performance degradation of IgniteTables#tables after configuration changes
> --------------------------------------------------------------------------
>
> Key: IGNITE-16760
> URL: https://issues.apache.org/jira/browse/IGNITE-16760
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 3.0.0-alpha4
> Reporter: Taras Ledkov
> Priority: Major
> Labels: ignite-3
>
> Performance degradation of configuration changes:
> Steps to reproduce:
> 1. Start cluster with 3 nodes
> 2. Run in the loop
> {code}
> CREATE TABLE TEST(ID INTEGER PRIMARY KEY, V INTEGER)
> DROP TABLE TEST
> {code}
> On begin {{IgniteTables#tables}} takes ~ 0.7 sec.
> The time of the operation is grown.
> The time after ~100 iteration is about 20 sec.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)