[jira] [Updated] (IGNITE-19395) Reduce write amplification for RocksDB partition storage

["Aleksandr Polovtsev (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IGNITE-19395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Aleksandr Polovtsev updated IGNITE-19395:
-----------------------------------------
    Fix Version/s: 3.0.0-beta2

> Reduce write amplification for RocksDB partition storage
> --------------------------------------------------------
>
>                 Key: IGNITE-19395
>                 URL: https://issues.apache.org/jira/browse/IGNITE-19395
>             Project: Ignite
>          Issue Type: Improvement
>            Reporter: Ivan Bessonov
>            Assignee: Aleksandr Polovtsev
>            Priority: Major
>              Labels: ignite-3
>             Fix For: 3.0.0-beta2
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently, the "commit" operation in rocksdb storage looks like this:
> {code:java}
> val data = db.read(writeIntentKey);
> db.remove(writeIntentKey);
> db.write(committedKey, data);{code}
> This is wasteful, we end up writing everything twice. There's another solution, we may add a level of indirection to the data:
> {code:java}
> // RowId index.
> [ TableId?? | PartId | RowId | Timestamp ] -> [ DataId ]
> [ TableId?? | PartId | RowId ] -> [ DataId | TxId | CommitTableId | CommitPartId ]
> // Data.
> [ DataId ] -> [ Payload ]{code}
> {{DataId}} must be unique. I don't like the idea of auto-incrementing key (we should always persist latest value), there must be another way.
> The main idea is that DataId doesn't change while committing the data, meaning that it can be generated using RowId and TxId.
> For example, {{RowId ++ beginTimestamp(TxId)}} seems like a unique value (with mandatory partition ID prefix and probably a table ID prefix)



--
This message was sent by Atlassian Jira
(v8.20.10#820010)