You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Vladimir Ozerov (JIRA)" <ji...@apache.org> on 2018/05/24 09:52:00 UTC
[jira] [Created] (IGNITE-8597) Direct data load
Vladimir Ozerov created IGNITE-8597:
---------------------------------------
Summary: Direct data load
Key: IGNITE-8597
URL: https://issues.apache.org/jira/browse/IGNITE-8597
Project: Ignite
Issue Type: Task
Components: general
Reporter: Vladimir Ozerov
We should implement optimized data loading mode, which will bypass as much internal Ignite components as possible to improve data loading speed.
Raw design:
1) Direct data load must be performed in exclusive mode, i.e. nobody else are allowed to read or write to specific cache/table at this time. I.e. we need to implement distributed table/cache locks.
2) At first we should write to data pages directly skipping free lists, PK hash index and secondary indexes
3) Once loading is finished, we should rebuild free lists and indexes (bottom-up); external merge implementation will be required
4) We should distinguish between initial load and incremental load. The latter would require merge of indexes with previously available data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)