[jira] [Commented] (IOTDB-1930) MetadataIndex hash structure design

["Zesong Sun (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IOTDB-1930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17447726#comment-17447726 ] 

Zesong Sun commented on IOTDB-1930:
-----------------------------------

MetadataIndexNode turns into HashMap, whose key is the name of device or measurement, and the value is pair of offset and size:
{code:java}
Map <String, Pair<long, int>> pathToOffsetMap // name-> <offset, size> {code}
Although hash table structure is used, the tree structure is still used at the end of the file to avoid the problem of large I / O. However, in order to take advantage of hash table, N = 8192 index entries are recorded in each MetadataIndexNode, that is, n = 8192 entries can be stored at most in the map.

 

Here are more details in [design doc|https://cwiki.apache.org/confluence/display/IOTDB/Experiments+of+TsFile+MedatadaIndexTree#ExperimentsofTsFileMedatadaIndexTree-(V)MetadataIndexhashstructuredesign].

> MetadataIndex hash structure design
> -----------------------------------
>
>                 Key: IOTDB-1930
>                 URL: https://issues.apache.org/jira/browse/IOTDB-1930
>             Project: Apache IoTDB
>          Issue Type: Task
>            Reporter: Jialin Qiao
>            Assignee: Zesong Sun
>            Priority: Major
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> Design hash structure of MetadataIndex according to the inspiration from [Timon: A Timestamped Event Database for Efficient Telemetry Data Processing and Analytics|https://dl.acm.org/doi/abs/10.1145/3318464.3386136].[1]
> [1] Cao, Wei, et al. "Timon: A Timestamped Event Database for Efficient Telemetry Data Processing and Analytics." Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data. 2020.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)