You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@iotdb.apache.org by "Xiangdong Huang (Jira)" <ji...@apache.org> on 2020/02/22 10:04:00 UTC
[jira] [Created] (IOTDB-510) A NullPointException in the
TsFileProcessor when flushing memtables
Xiangdong Huang created IOTDB-510:
-------------------------------------
Summary: A NullPointException in the TsFileProcessor when flushing memtables
Key: IOTDB-510
URL: https://issues.apache.org/jira/browse/IOTDB-510
Project: Apache IoTDB
Issue Type: Bug
Components: Core/Engine
Affects Versions: 0.10.0-SNAPSHOT
Reporter: Xiangdong Huang
Sometimes when we run TravicCI, the TsFileProcessorTest will hang up.
If we add log in TsFileProcessor, we can find logs like:
WARN [2020-02-22 02:45:54,092] [main] org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 - C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile has spent 60s for waiting flushing one memtable; 1 left (first: org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f). FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
WARN [2020-02-22 02:46:54,095] [main] org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 - C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile has spent 120s for waiting flushing one memtable; 1 left (first: org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f). FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
WARN [2020-02-22 02:47:54,108] [main] org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 - C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile has spent 180s for waiting flushing one memtable; 1 left (first: org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f). FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
WARN [2020-02-22 02:48:54,122] [main] org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 - C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile has spent 240s for waiting flushing one memtable; 1 left (first: org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f). FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
WARN [2020-02-22 02:49:54,132] [main] org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 - C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile has spent 300s for waiting flushing one memtable; 1 left (first: org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f). FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
WARN [2020-02-22 02:50:54,099] [main] org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:289 - C:\Users\travis\build\jixuan1989\incubator-iotdb\server\target\data\testUnsealedTsFileProcessor.tsfile has spent 360s for waiting flushing one memtable; 1 left (first: org.apache.iotdb.db.engine.flush.NotifyFlushMemTable@29c53c4f). FlushingManager info: TSProcessors in the queue: 0, TaskPool size 0 + 0,
Actually it is caused by a nullpointException but we omit the exception in the releaseFlushedMemTable() method...
The NullPointException happens when:
# A: there is an async flushing task;
# B: users call TsFileProcessor.syncClose;
The process is:
# B set `shouldClose = true` and has not put the final memtable (maybe a signalMemtable) into the flushingMemtable in asyncClose() (which is called by syncClose()).
# A checks the `shouldeClose == true && flushingMemtables.isEmpty() ` in flushOneMemtable(), then A call endFile() to close the fileWriter;
# B then put the final memtable into the flushingMemtable. When begin to flush the memtable, nullPointException is thrown (because the file Writer was set to null in endFile()).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)