[jira] [Commented] (IOTDB-5286) Fix target files should not be select to participate in other compaction task

["周沛辰 (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IOTDB-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17652455#comment-17652455 ] 

周沛辰 commented on IOTDB-5286:
----------------------------

合并生成目标文件 并进行了内存置换resource后 ，在删除磁盘源文件前会获取源文件resource的写锁，如果这个时候有其他查询长时间占着源文件的读锁，会导致合并任务卡住，且新置换到内存的文件被其他合并任务选中参与合并完成， 如果发生异常或系统崩溃 将会导致前一个合并任务抛异常。例如：顺序文件1-1-0-0，2-2-0-0 参与空间内合并生成了1-1-1-0，而源文件读锁被占用卡住，下一个空间内合并选中了1-1-1-0和3-3-0-0参与合并成功后会生成1-1-2-0，并删除1-1-1-0和3-3-0-0，这个时候第一个合并任务的目标文件其实被第二个合并任务给删了，后续如果第一个任务的锁释放了 删除了源文件，这时候系统崩溃，下次重启恢复或异常处理的时候会发现源文件被删了 目标文件也被删了，就会setAllowCompaction(false)。总结就是，第一个任务还没结束，但是它的目标文件就被后续的其他任务选中合并，若发生异常或系统中断 会导致恢复的时候报错 源文件和目标文件都被删了。

> Fix target files should not be select to participate in other compaction task
> -----------------------------------------------------------------------------
>
>                 Key: IOTDB-5286
>                 URL: https://issues.apache.org/jira/browse/IOTDB-5286
>             Project: Apache IoTDB
>          Issue Type: Bug
>    Affects Versions: master branch, 0.13.0, 1.0.0
>            Reporter: 周沛辰
>            Assignee: 周沛辰
>            Priority: Major
>             Fix For: master branch, 0.13.4, 1.0.1
>
>         Attachments: image-2022-12-28-17-59-46-918.png
>
>
> 2022-12-20 09:23:53,086 [pool-12-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.e.c.t.CompactionRecoverTask:300 - root.iot-0 [Compaction][ExceptionHandler] target file sequence/root.iot/0/0/1670572962795-1051-2-1.inner is not complete, and some source files is lost, do nothing. Set allowCompaction to false 
> 2022-12-20 09:23:53,087 [pool-12-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.e.c.t.CompactionRecoverTask:133 - root.iot-0 [Compaction][Recover] Failed to recover compaction, set allowCompaction to false
> !image-2022-12-28-17-59-46-918.png!
> *Description*
> When the system is restarted, a compaction error is reported: the source file is deleted and the target file is also deleted, which will set allowCompaction to false.
>  
> *Reason*
> Target files are selected to participate in other compaction tasks before the end of the current task, causing unexpected errors.
>  
> *Solution*
> Set target file to COMPACTING until the end of task, so that it can be selected to participate in other task after this task finishes.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)