You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "dean chen (JIRA)" <ji...@apache.org> on 2017/04/24 01:20:04 UTC
[jira] [Commented] (MESOS-7413) master crashed suddenly one day
[ https://issues.apache.org/jira/browse/MESOS-7413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15980616#comment-15980616 ]
dean chen commented on MESOS-7413:
----------------------------------
I find the reason. Because I open the vlog using enviornment variable 'export GLOG_v=3' in mesos-master-env.sh
It outputs many logs, then /root disk is out of space. I delete the log files and restart master, it works again.
> master crashed suddenly one day
> -------------------------------
>
> Key: MESOS-7413
> URL: https://issues.apache.org/jira/browse/MESOS-7413
> Project: Mesos
> Issue Type: Bug
> Components: master
> Affects Versions: 1.2.0
> Environment: CentOS 7.2
> Reporter: dean chen
>
> My meos cluster has running over one week, but it crashes today, when launching the master, got folowing error:
> I0423 22:58:28.012387 12429 process.cpp:3072] Resuming hierarchical-allocator(1)@10.118.28.141:5050 at 2017-04-23 14:58:28.012367872+00:00
> I0423 22:58:28.012377 12423 process.cpp:3062] Spawned process hierarchical-allocator(1)@10.118.28.141:5050
> I0423 22:58:28.012550 12423 main.cpp:385] Using 'HierarchicalDRF' allocator
> *** Aborted at 1492959508 (unix time) try "date -d @1492959508" if you are using GNU date ***
> PC: @ 0x7fd6b1c3aefb __memcpy_ssse3_back
> *** SIGBUS (@0x7fd6b9cb1080) received by PID 12423 (TID 0x7fd6b9ccb8c0) from PID 18446744072531677312; stack trace: ***
> @ 0x7fd6b1ec1370 (unknown)
> @ 0x7fd6b1c3aefb __memcpy_ssse3_back
> @ 0x7fd6b851587c leveldb::(anonymous namespace)::PosixMmapFile::Append()
> @ 0x7fd6b850a5dc leveldb::TableBuilder::WriteRawBlock()
> @ 0x7fd6b850a52c leveldb::TableBuilder::WriteBlock()
> @ 0x7fd6b850a2f0 leveldb::TableBuilder::Flush()
> @ 0x7fd6b850a1ff leveldb::TableBuilder::Add()
> @ 0x7fd6b8510c64 leveldb::BuildTable()
> @ 0x7fd6b84e9c19 leveldb::DBImpl::WriteLevel0Table()
> @ 0x7fd6b84e9917 leveldb::DBImpl::RecoverLogFile()
> @ 0x7fd6b84e916a leveldb::DBImpl::Recover()
> @ 0x7fd6b84ee2d4 leveldb::DB::Open()
> @ 0x7fd6b8169a0c mesos::internal::log::LevelDBStorage::restore()
> @ 0x7fd6b825430e mesos::internal::log::ReplicaProcess::restore()
> I0423 22:58:28.111234 12432 clock.cpp:152] Handling timers up to 2017-04-23 14:58:28.111174144+00:00
> I0423 22:58:28.111376 12432 clock.cpp:159] Have timeout(s) at 2017-04-23 14:58:28.110176000+00:00
> I0423 22:58:28.111744 12431 process.cpp:3072] Resuming __reaper__(1)@10.118.28.141:5050 at 2017-04-23 14:58:28.111730176+00:00
> I0423 22:58:28.111883 12431 clock.cpp:277] Created a timer for __reaper__(1)@10.118.28.141:5050 in 100ms in the future (2017-04-23 14:58:28.211844096+00:00)
> @ 0x7fd6b82504bb mesos::internal::log::ReplicaProcess::ReplicaProcess()
> @ 0x7fd6b82546d4 mesos::internal::log::Replica::Replica()
> @ 0x7fd6b816e3e3 mesos::internal::log::LogProcess::LogProcess()
> @ 0x7fd6b8173bb7 mesos::log::Log::Log()
> @ 0x7fd6b9d5a060 main
> @ 0x7fd6b1b12b35 __libc_start_main
> @ 0x7fd6b9d58539 (unknown)
> Bus error
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)