You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "xiaozhihong (Jira)" <ji...@apache.org> on 2022/04/19 03:25:00 UTC
[jira] [Commented] (IOTDB-2943) [ wal ] ERROR o.a.i.d.w.b.WALBuffer$SyncBufferTask:382 - Fail to roll wal node-0's log writer, change system mode to read-only.
[ https://issues.apache.org/jira/browse/IOTDB-2943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17524027#comment-17524027 ]
xiaozhihong commented on IOTDB-2943:
------------------------------------
经与海铭一起定位到：compaction导致iotdb进程无法退出
> [ wal ] ERROR o.a.i.d.w.b.WALBuffer$SyncBufferTask:382 - Fail to roll wal node-0's log writer, change system mode to read-only.
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: IOTDB-2943
> URL: https://issues.apache.org/jira/browse/IOTDB-2943
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/WAL
> Affects Versions: 0.14.0-SNAPSHOT
> Reporter: 刘珍
> Assignee: Haiming Zhu
> Priority: Major
> Attachments: config.properties.bk, log-all-20220417.0.log.gz, log_all.log.gz, stop_db.sh
>
>
> master 2691ade495513b54f00810c6cdd97c08326293e5
> 问题描述：
> benchmark 写入完成，执行flush , stop-server.sh
> 的时候，WAL-Sync线程报错，数据库read-only，{color:#DE350B}*数据库进程不退出*{color}。
> 2022-04-18 00:23:13,355 [pool-6-IoTDB-Compaction-1] ERROR o.a.i.d.e.c.i.InnerSpaceCompactionTask:197 - root.test.g_9-0 [Compaction] Throwable is caught during execution of SizeTieredCompaction, [Compaction] compaction for target file file is /data/iotdb_data/data/sequence/root.test.g_9/0/0/1650210199832-211-1-0.inner, status: abort
> {color:#DE350B}*2022-04-18 00:28:02,653 [pool-25-IoTDB-WAL-Sync(node-0)-1] ERROR o.a.i.d.w.b.WALBuffer$SyncBufferTask:382 - Fail to roll wal node-0's log writer, change system mode to read-only.*{color}
> java.io.FileNotFoundException: /data/iotdb_data/wal/0/_4035.wal (No such file or directory)
> at java.io.FileOutputStream.open0(Native Method)
> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> at org.apache.iotdb.db.wal.io.LogWriter.<init>(LogWriter.java:51)
> at org.apache.iotdb.db.wal.io.WALWriter.<init>(WALWriter.java:58)
> at org.apache.iotdb.db.wal.buffer.AbstractWALBuffer.rollLogWriter(AbstractWALBuffer.java:74)
> at org.apache.iotdb.db.wal.buffer.WALBuffer$SyncBufferTask.run(WALBuffer.java:379)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 测试步骤
> 1. 机器环境
> 8C32G *2
> 私有云 iotdb 172.20.70.2
> benchmark 172.20.70.16
> 数据库配置
> MAX_HEAP_SIZE="16G"
> wal_buffer_size_in_byte=1048576
> wal_buffer_queue_capacity=1000
> benchmark配置文件见附件。
> benchmark执行完之后，数据库端执行stop_db.sh（见附件）
> 数据库的日志：
> 2022-04-18 00:23:13,353 [pool-6-IoTDB-Compaction-3] ERROR o.a.i.d.e.c.i.InnerSpaceCompactionTask:197 - root.test.g_6-0 [Compaction] Throwable is caught during execution of SizeTieredCompaction, null
> 2022-04-18 00:23:13,354 [pool-6-IoTDB-Compaction-9] ERROR o.a.i.d.e.c.i.InnerSpaceCompactionTask:197 - root.test.g_39-0 [Compaction] Throwable is caught during execution of SizeTieredCompaction, [Compaction] compaction for target file file is /data/iotdb_data/data/sequence/root.test.g_39/0/0/1650210247564-211-1-0.inner, status: abort
> 2022-04-18 00:23:13,355 [pool-6-IoTDB-Compaction-1] ERROR o.a.i.d.e.c.i.InnerSpaceCompactionTask:197 - root.test.g_9-0 [Compaction] Throwable is caught during execution of SizeTieredCompaction, [Compaction] compaction for target file file is /data/iotdb_data/data/sequence/root.test.g_9/0/0/1650210199832-211-1-0.inner, status: abort
> 2022-04-18 00:28:02,653 [pool-25-IoTDB-WAL-Sync(node-0)-1] ERROR o.a.i.d.w.b.WALBuffer$SyncBufferTask:382 - Fail to roll wal node-0's log writer, change system mode to read-only.
> java.io.FileNotFoundException: /data/iotdb_data/wal/0/_4035.wal (No such file or directory)
> at java.io.FileOutputStream.open0(Native Method)
> at java.io.FileOutputStream.open(FileOutputStream.java:270)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
> at org.apache.iotdb.db.wal.io.LogWriter.<init>(LogWriter.java:51)
> at org.apache.iotdb.db.wal.io.WALWriter.<init>(WALWriter.java:58)
> at org.apache.iotdb.db.wal.buffer.AbstractWALBuffer.rollLogWriter(AbstractWALBuffer.java:74)
> at org.apache.iotdb.db.wal.buffer.WALBuffer$SyncBufferTask.run(WALBuffer.java:379)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {color:#DE350B}数据库进程不退出。{color}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)