You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by Lingzhe ZHANG <su...@foxmail.com> on 2020/10/21 11:08:52 UTC

[Discuss] Add Latest File Compaction Strategy

Hi all,

Current compaction strategy just merge old file.
When user write too fast, the compaction can just merge useless data. In this condition, the compaction can just cause bad effect.

So we have to develop a compaction strategy that can merge latest file as soon as possible, and use it as default startegy.

I will realize it[1] later. What do you think of it?

[1] https://issues.apache.org/jira/browse/IOTDB-952

Best,
-----------------
LingZhe ZHANG
School of Software, Tsinghua University

Re:[Discuss] Add Latest File Compaction Strategy

Posted by Houliang Qi <ne...@163.com>.
Hi  LingZhe,


Good idea,
A small question, how to define the latest files? 10 minutes, 1 hour, or the last day? I think this parameter is best left to the user to configure.


Thanks,
---------------------------------------
Houliang Qi


On 10/21/2020 19:13,Dawei Liu<at...@163.com> wrote:
Hi,


Big +1.


We have this usage scenario
where we want to read and take out data within
an hour in the quickest possible time,
beyond which the data becomes worthless.




Best,
Dawei
On 10/21/2020 19:08,Lingzhe ZHANG<su...@foxmail.com> wrote:
Hi all,

Current compaction strategy just merge old file.
When user write too fast, the compaction can just merge useless data. In this condition, the compaction can just cause bad effect.

So we have to develop a compaction strategy that can merge latest file as soon as possible, and use it as default startegy.

I will realize it[1] later. What do you think of it?

[1] https://issues.apache.org/jira/browse/IOTDB-952

Best,
-----------------
LingZhe ZHANG
School of Software, Tsinghua University

Re:[Discuss] Add Latest File Compaction Strategy

Posted by Dawei Liu <at...@163.com>.
Hi,


Big +1.


We have this usage scenario 
where we want to read and take out data within 
an hour in the quickest possible time, 
beyond which the data becomes worthless.




Best,
Dawei
On 10/21/2020 19:08,Lingzhe ZHANG<su...@foxmail.com> wrote:
Hi all,

Current compaction strategy just merge old file.
When user write too fast, the compaction can just merge useless data. In this condition, the compaction can just cause bad effect.

So we have to develop a compaction strategy that can merge latest file as soon as possible, and use it as default startegy.

I will realize it[1] later. What do you think of it?

[1] https://issues.apache.org/jira/browse/IOTDB-952

Best,
-----------------
LingZhe ZHANG
School of Software, Tsinghua University