You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Jialin Qiao <qj...@mails.tsinghua.edu.cn> on 2019/07/03 02:24:43 UTC
New directory structure
Hi,
The directory structure of IoTDB is in chaos, so I reorganize it in PR[1].
User configurations:
base_dir: base directory of IoTDB ("data" by default)
sys_dir: system data directory ("data/system" by default)
data_dirs: data file directories, which supports multiple directories ("data/data" by default, multiple directories are separated by ',")
wal_dir: directory of write-ahead-log ("data/wal" by default)
Under default configurations, the actual folders are like this:
data
data —————————————————————(data files)
sequence ——————————————————(folder of sequence data files)
storage_group_x ——————————————(storage group x)
{systemtime-version}.tsfile ————————(data file)
{name_of_TsFile}.resource ————————(resource file of corresponding tsfile)
unsequence —————————————————(folder of unsequence data files, which the same structure with "sequence" directory)
system
roles ————————————————————(information of roles)
users ————————————————————(information of users)
schema ———————————————————(system schema, including storage groups and timeseries)
storage_groups ———————————————(version file of each storage group)
wal
{name_of_storage_group}-{name_of_TsFile} ———(a folder for one data file)
wal{version} ————————————————(wal file for a memtable in corresponding data file)
[1] https://github.com/apache/incubator-iotdb/pull/217
Best,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
Re: Re: New directory structure
Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.
Hi,
Thanks for reminding, I will update the documents ASAP.
Best,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
> -----原始邮件-----
> 发件人: "Xiangdong Huang" <sa...@gmail.com>
> 发送时间: 2019-07-03 19:34:43 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: Re: New directory structure
>
> Hi,
>
> +1 for the new structure.
>
> Jialin, as Christofer said, I think we need review all documents before
> your branch can be merged..
>
> Best,
> -----------------------------------
> Xiangdong Huang
> School of Software, Tsinghua University
>
> 黄向东
> 清华大学 软件学院
>
>
> Christofer Dutz <ch...@c-ware.de> 于2019年7月3日周三 下午7:28写道：
>
> > Hi all,
> >
> > Please think of documenting this in any release notes as incompatible
> > change.
> >
> > Chris
> >
> > Holen Sie sichOutlook für Android<https://aka.ms/ghei36>
> >
> > ________________________________
> > From: Jialin Qiao <qj...@mails.tsinghua.edu.cn>
> > Sent: Wednesday, July 3, 2019 4:46:18 AM
> > To: dev@iotdb.apache.org
> > Subject: Re: New directory structure
> >
> > Hi,
> >
> > The following illustration is more clear:
> >
> > data —————————————————————————(base directory)
> > |├── data —————————————————————(data files)
> > │ ├── sequence ——————————————————(folder of sequence data files in each
> > storage group)
> > │ │ └── storage_group_x ——————————————(storage group x)
> > │ │ ├── {systemtime-version}.tsfile ————————(data file)
> > │ │ ├── {name_of_TsFile}.resource ————————(resource file of
> > corresponding tsfile)
> > │ └── unsequence —————————————————(folder of unsequence data files,
> > which the same structure with "sequence" directory)
> > |├── system
> > │ ├── roles ————————————————————(information of roles)
> > │ ├── schema ———————————————————(system schema, including storage groups
> > and timeseries)
> > │ ├── storage_groups ———————————————(version file of each storage
> > group)
> > │ └── users ————————————————————(information of users)
> > |└── wal
> > | ├── {name_of_storage_group}-{name_of_TsFile} ———(a folder for one
> > data file)
> > | └── wal{version} ————————————————(wal file for a memtable in
> > corresponding data file)
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
> >
> > > -----原始邮件-----
> > > 发件人: "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>
> > > 发送时间: 2019-07-03 10:24:43 (星期三)
> > > 收件人: dev@iotdb.apache.org
> > > 抄送:
> > > 主题: New directory structure
> > >
> > > Hi,
> > >
> > > The directory structure of IoTDB is in chaos, so I reorganize it in
> > PR[1].
> > >
> > > User configurations:
> > >
> > > base_dir: base directory of IoTDB ("data" by default)
> > > sys_dir: system data directory ("data/system" by default)
> > > data_dirs: data file directories, which supports multiple directories
> > ("data/data" by default, multiple directories are separated by ',")
> > > wal_dir: directory of write-ahead-log ("data/wal" by default)
> > >
> > > Under default configurations, the actual folders are like this:
> > >
> > > data
> > > data —————————————————————(data files)
> > > sequence ——————————————————(folder of sequence data files)
> > > storage_group_x ——————————————(storage group x)
> > > {systemtime-version}.tsfile ————————(data file)
> > > {name_of_TsFile}.resource ————————(resource file
> > of corresponding tsfile)
> > > unsequence —————————————————(folder of unsequence data
> > files, which the same structure with "sequence" directory)
> > > system
> > > roles ————————————————————(information of roles)
> > > users ————————————————————(information of users)
> > > schema ———————————————————(system schema, including storage
> > groups and timeseries)
> > > storage_groups ———————————————(version file of each
> > storage group)
> > > wal
> > > {name_of_storage_group}-{name_of_TsFile} ———(a folder for
> > one data file)
> > > wal{version} ————————————————(wal file for a memtable
> > in corresponding data file)
> > >
> > >
> > > [1] https://github.com/apache/incubator-iotdb/pull/217
> > >
> > > Best,
> > > --
> > > Jialin Qiao
> > > School of Software, Tsinghua University
> > >
> > > 乔嘉林
> > > 清华大学 软件学院
> >
Re: New directory structure
Posted by Xiangdong Huang <sa...@gmail.com>.
Hi,
+1 for the new structure.
Jialin, as Christofer said, I think we need review all documents before
your branch can be merged..
Best,
-----------------------------------
Xiangdong Huang
School of Software, Tsinghua University
黄向东
清华大学 软件学院
Christofer Dutz <ch...@c-ware.de> 于2019年7月3日周三 下午7:28写道：
> Hi all,
>
> Please think of documenting this in any release notes as incompatible
> change.
>
> Chris
>
> Holen Sie sichOutlook für Android<https://aka.ms/ghei36>
>
> ________________________________
> From: Jialin Qiao <qj...@mails.tsinghua.edu.cn>
> Sent: Wednesday, July 3, 2019 4:46:18 AM
> To: dev@iotdb.apache.org
> Subject: Re: New directory structure
>
> Hi,
>
> The following illustration is more clear:
>
> data —————————————————————————(base directory)
> |├── data —————————————————————(data files)
> │ ├── sequence ——————————————————(folder of sequence data files in each
> storage group)
> │ │ └── storage_group_x ——————————————(storage group x)
> │ │ ├── {systemtime-version}.tsfile ————————(data file)
> │ │ ├── {name_of_TsFile}.resource ————————(resource file of
> corresponding tsfile)
> │ └── unsequence —————————————————(folder of unsequence data files,
> which the same structure with "sequence" directory)
> |├── system
> │ ├── roles ————————————————————(information of roles)
> │ ├── schema ———————————————————(system schema, including storage groups
> and timeseries)
> │ ├── storage_groups ———————————————(version file of each storage
> group)
> │ └── users ————————————————————(information of users)
> |└── wal
> | ├── {name_of_storage_group}-{name_of_TsFile} ———(a folder for one
> data file)
> | └── wal{version} ————————————————(wal file for a memtable in
> corresponding data file)
>
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
>
> > -----原始邮件-----
> > 发件人: "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>
> > 发送时间: 2019-07-03 10:24:43 (星期三)
> > 收件人: dev@iotdb.apache.org
> > 抄送:
> > 主题: New directory structure
> >
> > Hi,
> >
> > The directory structure of IoTDB is in chaos, so I reorganize it in
> PR[1].
> >
> > User configurations:
> >
> > base_dir: base directory of IoTDB ("data" by default)
> > sys_dir: system data directory ("data/system" by default)
> > data_dirs: data file directories, which supports multiple directories
> ("data/data" by default, multiple directories are separated by ',")
> > wal_dir: directory of write-ahead-log ("data/wal" by default)
> >
> > Under default configurations, the actual folders are like this:
> >
> > data
> > data —————————————————————(data files)
> > sequence ——————————————————(folder of sequence data files)
> > storage_group_x ——————————————(storage group x)
> > {systemtime-version}.tsfile ————————(data file)
> > {name_of_TsFile}.resource ————————(resource file
> of corresponding tsfile)
> > unsequence —————————————————(folder of unsequence data
> files, which the same structure with "sequence" directory)
> > system
> > roles ————————————————————(information of roles)
> > users ————————————————————(information of users)
> > schema ———————————————————(system schema, including storage
> groups and timeseries)
> > storage_groups ———————————————(version file of each
> storage group)
> > wal
> > {name_of_storage_group}-{name_of_TsFile} ———(a folder for
> one data file)
> > wal{version} ————————————————(wal file for a memtable
> in corresponding data file)
> >
> >
> > [1] https://github.com/apache/incubator-iotdb/pull/217
> >
> > Best,
> > --
> > Jialin Qiao
> > School of Software, Tsinghua University
> >
> > 乔嘉林
> > 清华大学 软件学院
>
Re: New directory structure
Posted by Christofer Dutz <ch...@c-ware.de>.
Hi all,
Please think of documenting this in any release notes as incompatible change.
Chris
Holen Sie sichOutlook für Android<https://aka.ms/ghei36>
________________________________
From: Jialin Qiao <qj...@mails.tsinghua.edu.cn>
Sent: Wednesday, July 3, 2019 4:46:18 AM
To: dev@iotdb.apache.org
Subject: Re: New directory structure
Hi,
The following illustration is more clear:
data ―――――――――――――――――――――――――(base directory)
|├── data ―――――――――――――――――――――(data files)
│ ├── sequence ――――――――――――――――――(folder of sequence data files in each storage group)
│ │ └── storage_group_x ――――――――――――――(storage group x)
│ │ ├── {systemtime-version}.tsfile ――――――――(data file)
│ │ ├── {name_of_TsFile}.resource ――――――――(resource file of corresponding tsfile)
│ └── unsequence ―――――――――――――――――(folder of unsequence data files, which the same structure with "sequence" directory)
|├── system
│ ├── roles ――――――――――――――――――――(information of roles)
│ ├── schema ―――――――――――――――――――(system schema, including storage groups and timeseries)
│ ├── storage_groups ―――――――――――――――(version file of each storage group)
│ └── users ――――――――――――――――――――(information of users)
|└── wal
| ├── {name_of_storage_group}-{name_of_TsFile} ―――(a folder for one data file)
| └── wal{version} ――――――――――――――――(wal file for a memtable in corresponding data file)
Best,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
> -----原始邮件-----
> 发件人: "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>
> 发送时间: 2019-07-03 10:24:43 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: New directory structure
>
> Hi,
>
> The directory structure of IoTDB is in chaos, so I reorganize it in PR[1].
>
> User configurations:
>
> base_dir: base directory of IoTDB ("data" by default)
> sys_dir: system data directory ("data/system" by default)
> data_dirs: data file directories, which supports multiple directories ("data/data" by default, multiple directories are separated by ',")
> wal_dir: directory of write-ahead-log ("data/wal" by default)
>
> Under default configurations, the actual folders are like this:
>
> data
> data ―――――――――――――――――――――(data files)
> sequence ――――――――――――――――――(folder of sequence data files)
> storage_group_x ――――――――――――――(storage group x)
> {systemtime-version}.tsfile ――――――――(data file)
> {name_of_TsFile}.resource ――――――――(resource file of corresponding tsfile)
> unsequence ―――――――――――――――――(folder of unsequence data files, which the same structure with "sequence" directory)
> system
> roles ――――――――――――――――――――(information of roles)
> users ――――――――――――――――――――(information of users)
> schema ―――――――――――――――――――(system schema, including storage groups and timeseries)
> storage_groups ―――――――――――――――(version file of each storage group)
> wal
> {name_of_storage_group}-{name_of_TsFile} ―――(a folder for one data file)
> wal{version} ――――――――――――――――(wal file for a memtable in corresponding data file)
>
>
> [1] https://github.com/apache/incubator-iotdb/pull/217
>
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院
Re: New directory structure
Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.
Hi,
The following illustration is more clear:
data —————————————————————————(base directory)
|├── data —————————————————————(data files)
│ ├── sequence ——————————————————(folder of sequence data files in each storage group)
│ │ └── storage_group_x ——————————————(storage group x)
│ │ ├── {systemtime-version}.tsfile ————————(data file)
│ │ ├── {name_of_TsFile}.resource ————————(resource file of corresponding tsfile)
│ └── unsequence —————————————————(folder of unsequence data files, which the same structure with "sequence" directory)
|├── system
│ ├── roles ————————————————————(information of roles)
│ ├── schema ———————————————————(system schema, including storage groups and timeseries)
│ ├── storage_groups ———————————————(version file of each storage group)
│ └── users ————————————————————(information of users)
|└── wal
| ├── {name_of_storage_group}-{name_of_TsFile} ———(a folder for one data file)
| └── wal{version} ————————————————(wal file for a memtable in corresponding data file)
Best,
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
> -----原始邮件-----
> 发件人: "Jialin Qiao" <qj...@mails.tsinghua.edu.cn>
> 发送时间: 2019-07-03 10:24:43 (星期三)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: New directory structure
>
> Hi,
>
> The directory structure of IoTDB is in chaos, so I reorganize it in PR[1].
>
> User configurations:
>
> base_dir: base directory of IoTDB ("data" by default)
> sys_dir: system data directory ("data/system" by default)
> data_dirs: data file directories, which supports multiple directories ("data/data" by default, multiple directories are separated by ',")
> wal_dir: directory of write-ahead-log ("data/wal" by default)
>
> Under default configurations, the actual folders are like this:
>
> data
> data —————————————————————(data files)
> sequence ——————————————————(folder of sequence data files)
> storage_group_x ——————————————(storage group x)
> {systemtime-version}.tsfile ————————(data file)
> {name_of_TsFile}.resource ————————(resource file of corresponding tsfile)
> unsequence —————————————————(folder of unsequence data files, which the same structure with "sequence" directory)
> system
> roles ————————————————————(information of roles)
> users ————————————————————(information of users)
> schema ———————————————————(system schema, including storage groups and timeseries)
> storage_groups ———————————————(version file of each storage group)
> wal
> {name_of_storage_group}-{name_of_TsFile} ———(a folder for one data file)
> wal{version} ————————————————(wal file for a memtable in corresponding data file)
>
>
> [1] https://github.com/apache/incubator-iotdb/pull/217
>
> Best,
> --
> Jialin Qiao
> School of Software, Tsinghua University
>
> 乔嘉林
> 清华大学 软件学院