You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Li Cheng (Jira)" <ji...@apache.org> on 2019/11/01 03:44:00 UTC
[jira] [Created] (HDDS-2396) OM rocksdb core dump during writing
Li Cheng created HDDS-2396:
------------------------------
Summary: OM rocksdb core dump during writing
Key: HDDS-2396
URL: https://issues.apache.org/jira/browse/HDDS-2396
Project: Hadoop Distributed Data Store
Issue Type: Bug
Components: Ozone Manager
Affects Versions: 0.4.1
Reporter: Li Cheng
Attachments: hs_err_pid9340.log
Env: 4 VMs in total: 3 Datanodes on 3 VMs, 1 OM & 1 SCM on a separate VM, say it's VM0.
I use goofys as a fuse and enable ozone S3 gateway to mount ozone to a path on VM0, while reading data from VM0 local disk and write to mount path. The dataset has various sizes of files from 0 byte to GB-level and it has a number of ~50,000 files.
There happens core dump in rocksdb while it's occasional.
Stack: [0x00007f5891a23000,0x00007f5891b24000], sp=0x00007f5891b21bb8, free space=1018k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libc.so.6+0x151d60] __memmove_ssse3_back+0x1ae0
C [librocksdbjni3192271038586903156.so+0x358fec] rocksdb::MemTableInserter::PutCFImpl(unsigned int, rocksdb::Slice const&, rocksdb::Slice const&, rocksdb:
:ValueType)+0x51c
C [librocksdbjni3192271038586903156.so+0x359d17] rocksdb::MemTableInserter::PutCF(unsigned int, rocksdb::Slice const&, rocksdb::Slice const&)+0x17
C [librocksdbjni3192271038586903156.so+0x3513bc] rocksdb::WriteBatch::Iterate(rocksdb::WriteBatch::Handler*) const+0x45c
C [librocksdbjni3192271038586903156.so+0x354df9] rocksdb::WriteBatchInternal::InsertInto(rocksdb::WriteThread::WriteGroup&, unsigned long, rocksdb::ColumnFamilyMemTables*, rocksdb::FlushScheduler*, bool, unsigned long, rocksdb::DB*, bool, bool, bool)+0x1f9
C [librocksdbjni3192271038586903156.so+0x29fd79] rocksdb::DBImpl::WriteImpl(rocksdb::WriteOptions const&, rocksdb::WriteBatch*, rocksdb::WriteCallback*, unsigned long*, unsigned long, bool, unsigned long*, unsigned long, rocksdb::PreReleaseCallback*)+0x24b9
C [librocksdbjni3192271038586903156.so+0x2a0431] rocksdb::DBImpl::Write(rocksdb::WriteOptions const&, rocksdb::WriteBatch*)+0x21
C [librocksdbjni3192271038586903156.so+0x1a064c] Java_org_rocksdb_RocksDB_write0+0xcc
J 7899 org.rocksdb.RocksDB.write0(JJJ)V (0 bytes) @ 0x00007f58f1872dbe [0x00007f58f1872d00+0xbe]
J 10093% C1 org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushTransactions()V (400 bytes) @ 0x00007f58f2308b0c [0x00007f58f2307a40+0x10cc]
j org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer$$Lambda$29.run()V+4
j java.lang.Thread.run()V+11
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org