[jira] [Created] (COMDEV-402) Apache IoTDB: Metadata (Schema) Storage Engine

["Xiangdong Huang (Jira)" <ji...@apache.org>]

Xiangdong Huang created COMDEV-402:
--------------------------------------

             Summary: Apache IoTDB: Metadata (Schema) Storage Engine
                 Key: COMDEV-402
                 URL: https://issues.apache.org/jira/browse/COMDEV-402
             Project: Community Development
          Issue Type: Task
          Components: GSoC/Mentoring ideas
            Reporter: Xiangdong Huang


Apache IoTDB [1] is an Open Source IoT database designed to meet the rigorous data, storage, and analytics requirements of large-scale Internet of Things (IoT) and Industrial Internet of Things (IIoT) applications. 

Different with traditional relational databases, IoTDB uses tree-based structure in memory to manage the schema (a.k.a, metadata), and use a Write-Ahead-Log-like file structure to persist the schema.

Now, each time series will take about 300 Bytes in memory. However, an IoTDB instance may manage more than 100 million time series, which may take more than 30GB memory.

Therefore, we'd like to re-design the schema management module.
1. File: Persist the tree on disk like a b-tree.
2. WAL: implement the WAL of the metadata. So we can update the tree on disk in batch, rather than one operation by one.
3. Cache: we may have no memory to load the whole tree. So a cache is needed and query from the tree on disk is needed.


What knowledge you need to know:
1. Java
2. Basic design idea about Database [2]



[1] https://iotdb.apache.org
[2] http://pages.cs.wisc.edu/~dbbook/openAccess/firstEdition/slides/pdfslides/mod2l1.pdf



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@community.apache.org
For additional commands, e-mail: dev-help@community.apache.org