You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Dawei Liu <at...@163.com> on 2020/06/19 17:15:10 UTC
[Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
Dear community,
I'd like to share the weekly report for you this week.
[Big Event]
* We plan to hold online-discussion every 2-3 weeks to let contributors communicate more deeply.
* Maturity Evaluation draft of IoTDB is published, please have a review [1]
* We are voting the 0.10.0-RC4 now, please check and vote!
* A new feature: "show latest timeseries", for showing the latest active timeseries, i.e., ordering all timeseries according to how long the series has not written new data.
* The MTree is optimized
* Memory cost experiment, 20M time series metadata in MTree takes about 7G memory.
[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
[in-progress]
* Check point of MTree to accelerate the restart process when there are many timeseries. It takes about 2 minutes for 20M timeseries.
* Memory control of storage engine, query engine, flushing memtable, index of each tsfile
* Support time range Deletion
[bug fix]
[IOTDB-765] Fix failed to get upgrade.txt file when using HDFS storage
[IOTDB-769] [To rel/0.10] fix precision lost when using PLAIN for FLOAT/DOUBLE
[IOTDB-762] Fix set version IndexOutofBound
[IOTDB-772] The start-server.sh uses tool long java command in WinOS
Fix Restart deadlock bug
[IOTDB-761]cherry-picked from v0.10: flink-tsfile-connecto and flink-example dependency conflict: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
Have a great weekend.
Thanks
——
Dawei Liu
Re: Re:[Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
Posted by Yuyuan KANG <ky...@mails.tsinghua.edu.cn>.
Thanks for your sharing.
Best,
Yuyuan Kang
> -----原始邮件-----
> 发件人: "Dawei Liu" <at...@163.com>
> 发送时间: 2020-06-20 01:23:27 (星期六)
> 收件人: "dev@iotdb.apache.org" <de...@iotdb.apache.org>
> 抄送:
> 主题: Re:[Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
>
> 大家好，
> 很开心又一次与大家分享 IoTDB 社区周报。
>
>
> [本周大事件]
> * 我们计划每2-3周举行一次在线讨论，以便让更多的贡献者能够深入交流。
> * IoTDB 社区的成熟度评估(draft)已经发布，欢迎各位审阅[1].
> * 0.10.0-RC4 正在投票，欢迎检查并发表意见和建议!
> * 新增加了 "show latest timeseries"功能, 该功能按照timeseries中最后一个点的时间戳降序输出，可以更方便地知道哪些时序还在写数据.
> * 优化了 MTree
> * 分享一个内存开销的实验结果，在MTree中保存2千万的时序元数据大约需要 7G 内存.
>
> [1]https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
>
>
> [in-progress]
> * 新增 MTree 的检查点，加快当数据库中存在非常多的时序数据时的重启恢复速度。目前2千万数据大概需要花费2分钟。
> * 新增对存储引擎、查询引擎、memTable 以及每个TsFile的索引内存控制。
> * 新增基于时间范围删除数据的语句
>
>
>
>
> [bug fix]
> [IOTDB-765] 修复当使用HDFS存储时，获取 upgrade.txt 文件产生的错误
> [IOTDB-769] [To rel/0.10] 修复当使用PLAIN编码时，FLOAT/DOUBLE丢失精度的问题
> [IOTDB-762] 修复设置versionInfo时产生的 IndexOutofBound 错误
> [IOTDB-772] 优化 WinOS 下 start-server.sh 脚本中的 java 命令过长
> 修复重启时死锁的bug
> [IOTDB-761]cherry-picked from v0.10: 修复 flink-tsfile-connecto 和 flink-example工程中的依赖冲突: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
> Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
>
>
>
>
> 祝各位周末愉快
>
>
> Thanks
> -----
>
>
> Dawei Liu
>
>
>
>
> On 06/20/2020 01:15，Dawei Liu<at...@163.com> wrote：
> Dear community,
>
>
>
> I'd like to share the weekly report for you this week.
>
>
> [Big Event]
> * We plan to hold online-discussion every 2-3 weeks to let contributors communicate more deeply.
> * Maturity Evaluation draft of IoTDB is published, please have a review [1]
> * We are voting the 0.10.0-RC4 now, please check and vote!
> * A new feature: "show latest timeseries", for showing the latest active timeseries, i.e., ordering all timeseries according to how long the series has not written new data.
> * The MTree is optimized
> * Memory cost experiment, 20M time series metadata in MTree takes about 7G memory.
>
>
> [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
>
>
> [in-progress]
> * Check point of MTree to accelerate the restart process when there are many timeseries. It takes about 2 minutes for 20M timeseries.
> * Memory control of storage engine, query engine, flushing memtable, index of each tsfile
> * Support time range Deletion
>
>
> [bug fix]
> [IOTDB-765] Fix failed to get upgrade.txt file when using HDFS storage
> [IOTDB-769] [To rel/0.10] fix precision lost when using PLAIN for FLOAT/DOUBLE
> [IOTDB-762] Fix set version IndexOutofBound
> [IOTDB-772] The start-server.sh uses tool long java command in WinOS
> Fix Restart deadlock bug
> [IOTDB-761]cherry-picked from v0.10: flink-tsfile-connecto and flink-example dependency conflict: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
> Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
>
>
>
>
> Have a great weekend.
>
>
> Thanks
> ——
>
>
> Dawei Liu
>
Re: [Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
Posted by Jialin Qiao <qj...@mails.tsinghua.edu.cn>.
Thanks DaWei!
--
Jialin Qiao
School of Software, Tsinghua University
乔嘉林
清华大学 软件学院
> -----原始邮件-----
> 发件人: "孙泽嵩" <sz...@mails.tsinghua.edu.cn>
> 发送时间: 2020-06-20 07:07:44 (星期六)
> 收件人: dev@iotdb.apache.org
> 抄送:
> 主题: Re: [Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
>
> Hi Dawei,
>
> Welcome back!
>
> Thanks for ur sharing : )
>
>
> Best,
> -----------------------------------
> Zesong Sun
> School of Software, Tsinghua University
>
> 孙泽嵩
> 清华大学 软件学院
>
> > 2020年6月20日 01:23，Dawei Liu <at...@163.com> 写道：
> >
> > 大家好，
> > 很开心又一次与大家分享 IoTDB 社区周报。
> >
> >
> > [本周大事件]
> > * 我们计划每2-3周举行一次在线讨论，以便让更多的贡献者能够深入交流。
> > * IoTDB 社区的成熟度评估(draft)已经发布，欢迎各位审阅[1].
> > * 0.10.0-RC4 正在投票，欢迎检查并发表意见和建议!
> > * 新增加了 "show latest timeseries"功能, 该功能按照timeseries中最后一个点的时间戳降序输出，可以更方便地知道哪些时序还在写数据.
> > * 优化了 MTree
> > * 分享一个内存开销的实验结果，在MTree中保存2千万的时序元数据大约需要 7G 内存.
> >
> > [1]https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
> >
> >
> > [in-progress]
> > * 新增 MTree 的检查点，加快当数据库中存在非常多的时序数据时的重启恢复速度。目前2千万数据大概需要花费2分钟。
> > * 新增对存储引擎、查询引擎、memTable 以及每个TsFile的索引内存控制。
> > * 新增基于时间范围删除数据的语句
> >
> >
> >
> >
> > [bug fix]
> > [IOTDB-765] 修复当使用HDFS存储时，获取 upgrade.txt 文件产生的错误
> > [IOTDB-769] [To rel/0.10] 修复当使用PLAIN编码时，FLOAT/DOUBLE丢失精度的问题
> > [IOTDB-762] 修复设置versionInfo时产生的 IndexOutofBound 错误
> > [IOTDB-772] 优化 WinOS 下 start-server.sh 脚本中的 java 命令过长
> > 修复重启时死锁的bug
> > [IOTDB-761]cherry-picked from v0.10: 修复 flink-tsfile-connecto 和 flink-example工程中的依赖冲突: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
> > Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
> >
> >
> >
> >
> > 祝各位周末愉快
> >
> >
> > Thanks
> > -----
> >
> >
> > Dawei Liu
> >
> >
> >
> >
> > On 06/20/2020 01:15，Dawei Liu<at...@163.com> wrote：
> > Dear community,
> >
> >
> >
> > I'd like to share the weekly report for you this week.
> >
> >
> > [Big Event]
> > * We plan to hold online-discussion every 2-3 weeks to let contributors communicate more deeply.
> > * Maturity Evaluation draft of IoTDB is published, please have a review [1]
> > * We are voting the 0.10.0-RC4 now, please check and vote!
> > * A new feature: "show latest timeseries", for showing the latest active timeseries, i.e., ordering all timeseries according to how long the series has not written new data.
> > * The MTree is optimized
> > * Memory cost experiment, 20M time series metadata in MTree takes about 7G memory.
> >
> >
> > [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
> >
> >
> > [in-progress]
> > * Check point of MTree to accelerate the restart process when there are many timeseries. It takes about 2 minutes for 20M timeseries.
> > * Memory control of storage engine, query engine, flushing memtable, index of each tsfile
> > * Support time range Deletion
> >
> >
> > [bug fix]
> > [IOTDB-765] Fix failed to get upgrade.txt file when using HDFS storage
> > [IOTDB-769] [To rel/0.10] fix precision lost when using PLAIN for FLOAT/DOUBLE
> > [IOTDB-762] Fix set version IndexOutofBound
> > [IOTDB-772] The start-server.sh uses tool long java command in WinOS
> > Fix Restart deadlock bug
> > [IOTDB-761]cherry-picked from v0.10: flink-tsfile-connecto and flink-example dependency conflict: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
> > Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
> >
> >
> >
> >
> > Have a great weekend.
> >
> >
> > Thanks
> > ——
> >
> >
> > Dawei Liu
> >
>
Re: [Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
Posted by 孙泽嵩 <sz...@mails.tsinghua.edu.cn>.
Hi Dawei,
Welcome back!
Thanks for ur sharing : )
Best,
-----------------------------------
Zesong Sun
School of Software, Tsinghua University
孙泽嵩
清华大学 软件学院
> 2020年6月20日 01:23，Dawei Liu <at...@163.com> 写道：
>
> 大家好，
> 很开心又一次与大家分享 IoTDB 社区周报。
>
>
> [本周大事件]
> * 我们计划每2-3周举行一次在线讨论，以便让更多的贡献者能够深入交流。
> * IoTDB 社区的成熟度评估(draft)已经发布，欢迎各位审阅[1].
> * 0.10.0-RC4 正在投票，欢迎检查并发表意见和建议!
> * 新增加了 "show latest timeseries"功能, 该功能按照timeseries中最后一个点的时间戳降序输出，可以更方便地知道哪些时序还在写数据.
> * 优化了 MTree
> * 分享一个内存开销的实验结果，在MTree中保存2千万的时序元数据大约需要 7G 内存.
>
> [1]https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
>
>
> [in-progress]
> * 新增 MTree 的检查点，加快当数据库中存在非常多的时序数据时的重启恢复速度。目前2千万数据大概需要花费2分钟。
> * 新增对存储引擎、查询引擎、memTable 以及每个TsFile的索引内存控制。
> * 新增基于时间范围删除数据的语句
>
>
>
>
> [bug fix]
> [IOTDB-765] 修复当使用HDFS存储时，获取 upgrade.txt 文件产生的错误
> [IOTDB-769] [To rel/0.10] 修复当使用PLAIN编码时，FLOAT/DOUBLE丢失精度的问题
> [IOTDB-762] 修复设置versionInfo时产生的 IndexOutofBound 错误
> [IOTDB-772] 优化 WinOS 下 start-server.sh 脚本中的 java 命令过长
> 修复重启时死锁的bug
> [IOTDB-761]cherry-picked from v0.10: 修复 flink-tsfile-connecto 和 flink-example工程中的依赖冲突: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
> Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
>
>
>
>
> 祝各位周末愉快
>
>
> Thanks
> -----
>
>
> Dawei Liu
>
>
>
>
> On 06/20/2020 01:15，Dawei Liu<at...@163.com> wrote：
> Dear community,
>
>
>
> I'd like to share the weekly report for you this week.
>
>
> [Big Event]
> * We plan to hold online-discussion every 2-3 weeks to let contributors communicate more deeply.
> * Maturity Evaluation draft of IoTDB is published, please have a review [1]
> * We are voting the 0.10.0-RC4 now, please check and vote!
> * A new feature: "show latest timeseries", for showing the latest active timeseries, i.e., ordering all timeseries according to how long the series has not written new data.
> * The MTree is optimized
> * Memory cost experiment, 20M time series metadata in MTree takes about 7G memory.
>
>
> [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
>
>
> [in-progress]
> * Check point of MTree to accelerate the restart process when there are many timeseries. It takes about 2 minutes for 20M timeseries.
> * Memory control of storage engine, query engine, flushing memtable, index of each tsfile
> * Support time range Deletion
>
>
> [bug fix]
> [IOTDB-765] Fix failed to get upgrade.txt file when using HDFS storage
> [IOTDB-769] [To rel/0.10] fix precision lost when using PLAIN for FLOAT/DOUBLE
> [IOTDB-762] Fix set version IndexOutofBound
> [IOTDB-772] The start-server.sh uses tool long java command in WinOS
> Fix Restart deadlock bug
> [IOTDB-761]cherry-picked from v0.10: flink-tsfile-connecto and flink-example dependency conflict: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
> Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
>
>
>
>
> Have a great weekend.
>
>
> Thanks
> ——
>
>
> Dawei Liu
>
Re:[Weekly Report] IoTDB Weekly News (2020-06-12~2020-06-19)
Posted by Dawei Liu <at...@163.com>.
大家好，
很开心又一次与大家分享 IoTDB 社区周报。
[本周大事件]
* 我们计划每2-3周举行一次在线讨论，以便让更多的贡献者能够深入交流。
* IoTDB 社区的成熟度评估(draft)已经发布，欢迎各位审阅[1].
* 0.10.0-RC4 正在投票，欢迎检查并发表意见和建议!
* 新增加了 "show latest timeseries"功能, 该功能按照timeseries中最后一个点的时间戳降序输出，可以更方便地知道哪些时序还在写数据.
* 优化了 MTree
* 分享一个内存开销的实验结果，在MTree中保存2千万的时序元数据大约需要 7G 内存.
[1]https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
[in-progress]
* 新增 MTree 的检查点，加快当数据库中存在非常多的时序数据时的重启恢复速度。目前2千万数据大概需要花费2分钟。
* 新增对存储引擎、查询引擎、memTable 以及每个TsFile的索引内存控制。
* 新增基于时间范围删除数据的语句
[bug fix]
[IOTDB-765] 修复当使用HDFS存储时，获取 upgrade.txt 文件产生的错误
[IOTDB-769] [To rel/0.10] 修复当使用PLAIN编码时，FLOAT/DOUBLE丢失精度的问题
[IOTDB-762] 修复设置versionInfo时产生的 IndexOutofBound 错误
[IOTDB-772] 优化 WinOS 下 start-server.sh 脚本中的 java 命令过长
修复重启时死锁的bug
[IOTDB-761]cherry-picked from v0.10: 修复 flink-tsfile-connecto 和 flink-example工程中的依赖冲突: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
祝各位周末愉快
Thanks
-----
Dawei Liu
On 06/20/2020 01:15，Dawei Liu<at...@163.com> wrote：
Dear community,
I'd like to share the weekly report for you this week.
[Big Event]
* We plan to hold online-discussion every 2-3 weeks to let contributors communicate more deeply.
* Maturity Evaluation draft of IoTDB is published, please have a review [1]
* We are voting the 0.10.0-RC4 now, please check and vote!
* A new feature: "show latest timeseries", for showing the latest active timeseries, i.e., ordering all timeseries according to how long the series has not written new data.
* The MTree is optimized
* Memory cost experiment, 20M time series metadata in MTree takes about 7G memory.
[1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645763
[in-progress]
* Check point of MTree to accelerate the restart process when there are many timeseries. It takes about 2 minutes for 20M timeseries.
* Memory control of storage engine, query engine, flushing memtable, index of each tsfile
* Support time range Deletion
[bug fix]
[IOTDB-765] Fix failed to get upgrade.txt file when using HDFS storage
[IOTDB-769] [To rel/0.10] fix precision lost when using PLAIN for FLOAT/DOUBLE
[IOTDB-762] Fix set version IndexOutofBound
[IOTDB-772] The start-server.sh uses tool long java command in WinOS
Fix Restart deadlock bug
[IOTDB-761]cherry-picked from v0.10: flink-tsfile-connecto and flink-example dependency conflict: "Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed"
Fix spark tsfile bug in 0.9.4/0.10.1 and 0.11.0
Have a great weekend.
Thanks
——
Dawei Liu