You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@iotdb.apache.org by "刘珍 (Jira)" <ji...@apache.org> on 2022/03/15 05:46:00 UTC
[jira] [Assigned] (IOTDB-2741) [IDTable] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
[ https://issues.apache.org/jira/browse/IOTDB-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
刘珍 reassigned IOTDB-2741:
-------------------------
Assignee: Kaifeng Xue
> [IDTable] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> -------------------------------------------------------------------------------------
>
> Key: IOTDB-2741
> URL: https://issues.apache.org/jira/browse/IOTDB-2741
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/Schema Manager
> Affects Versions: 0.13.0-SNAPSHOT
> Reporter: 刘珍
> Assignee: Kaifeng Xue
> Priority: Major
> Attachments: config.properties, iotdb-engine.properties, iotdb-env.sh, log_error.log
>
>
> 测试版本：
> https://github.com/apache/iotdb/pull/5214
> 问题描述:
> 开启id table，benchmark工具写入，写入完成立即kill -u 当前用户，备份iotdb的数据，
> 重启iotdb恢复，报错：
> 2022-03-15 13:31:56,249 [pool-9-IoTDB-Recovery-Thread-Pool-5] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,249 [pool-9-IoTDB-Recovery-Thread-Pool-5] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,251 [pool-9-IoTDB-Recovery-Thread-Pool-2] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,252 [pool-9-IoTDB-Recovery-Thread-Pool-2] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,253 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,253 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,254 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,254 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,255 [Thread-2] ERROR o.a.i.d.c.IoTDBDefaultThreadExceptionHandler:31 - Exception in thread Thread-2-25
> org.apache.iotdb.db.exception.runtime.StorageEngineFailureException: StorageEngine failed to recover.
> at org.apache.iotdb.db.engine.StorageEngine.lambda$recover$1(StorageEngine.java:245)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: can't initialize disk schema manager at ./sbin/../data/system/storage_groups/root.test.g_30/SeriesKeyMapping.meta
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at org.apache.iotdb.db.engine.StorageEngine.lambda$recover$1(StorageEngine.java:243)
> ... 1 common frames omitted
> Caused by: java.lang.IllegalArgumentException: can't initialize disk schema manager at ./sbin/../data/system/storage_groups/root.test.g_30/SeriesKeyMapping.meta
> at org.apache.iotdb.db.metadata.idtable.AppendOnlyDiskSchemaManager.<init>(AppendOnlyDiskSchemaManager.java:71)
> at org.apache.iotdb.db.metadata.idtable.IDTableHashmapImpl.<init>(IDTableHashmapImpl.java:82)
> at org.apache.iotdb.db.metadata.idtable.IDTableManager.lambda$getIDTable$0(IDTableManager.java:90)
> at java.util.HashMap.computeIfAbsent(HashMap.java:1127)
> at org.apache.iotdb.db.metadata.idtable.IDTableManager.getIDTable(IDTableManager.java:86)
> at org.apache.iotdb.db.engine.storagegroup.VirtualStorageGroupProcessor.<init>(VirtualStorageGroupProcessor.java:398)
> at org.apache.iotdb.db.engine.StorageEngine.buildNewStorageGroupProcessor(StorageEngine.java:552)
> at org.apache.iotdb.db.engine.storagegroup.virtualSg.StorageGroupManager.lambda$asyncRecover$0(StorageGroupManager.java:208)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> ... 1 common frames omitted
> 2022-03-15 13:31:56,256 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,256 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,257 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,258 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,259 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,259 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,260 [pool-9-IoTDB-Recovery-Thread-Pool-6] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,260 [pool-9-IoTDB-Recovery-Thread-Pool-6] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,261 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,262 [pool-9-IoTDB-Recovery-Thread-Pool-8] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,262 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,263 [pool-9-IoTDB-Recovery-Thread-Pool-7] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,264 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,264 [pool-9-IoTDB-Recovery-Thread-Pool-4] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,267 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,267 [pool-9-IoTDB-Recovery-Thread-Pool-3] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 2022-03-15 13:31:56,268 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:136 - File length isn't right
> 2022-03-15 13:31:56,268 [pool-9-IoTDB-Recovery-Thread-Pool-1] ERROR o.a.i.d.m.i.AppendOnlyDiskSchemaManager:70 - File corruption
> 测试过程：
> 1. 测试机器
> 私有云172.20.70.4
> 配置文件见附件
> device_id_transformation_method=SHA256
> enable_id_table=true
> enable_id_table_log_file=true
> 2. 启动iotdb
> 3. benchmark写入
> 配置文件见附件。
> benchmark执行完之后，立即执行kill -u cluster
> 4. 备份data (用于bug复现）
> 5. 清缓存
> suod -s<<EOF
> echo 3 >/proc/sys/vm/drop_caches
> EOF
> 6.启动iotdb
> 报错，error日志见附件。
--
This message was sent by Atlassian Jira
(v8.20.1#820001)