[jira] [Comment Edited] (MESOS-7566) Master crash due to failed check in DRFSorter::remove

["Zhitao Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037565#comment-16037565 ] 

Zhitao Li edited comment on MESOS-7566 at 6/5/17 8:50 PM:
----------------------------------------------------------

A similar but maybe not identical crash's stack trace reported by gdb:

https://gist.github.com/zhitaoli/180f7aa3c619dab44db19af92fd7d3a1

This is a slightly patched version of {{1.1.2}}.


was (Author: zhitao):
A similar but maybe not identical crash's stack trace reported by gdb:

https://gist.github.com/zhitaoli/180f7aa3c619dab44db19af92fd7d3a1

This is a slightly patched version of `1.1.2`.

> Master crash due to failed check in DRFSorter::remove
> -----------------------------------------------------
>
>                 Key: MESOS-7566
>                 URL: https://issues.apache.org/jira/browse/MESOS-7566
>             Project: Mesos
>          Issue Type: Bug
>    Affects Versions: 1.1.1, 1.1.2
>            Reporter: Zhitao Li
>            Priority: Critical
>
> A check in [sorter.cpp#L355 in 1.1.2 | https://github.com/apache/mesos/blob/1.1.2/src/master/allocator/sorter/drf/sorter.cpp#L355] is triggered occasionally in our cluster and crashes the master leader.
> I manually modified that check to print out the related variables, and the following is a master log.
> https://gist.github.com/zhitaoli/0662d9fe1f6d57de344951c05b536bad#file-gistfile1-txt
> From the log, it seems like the check was using an stale value revocable CPU  {{26}} while the new value was updated to 25, thus the check crashed.
> So far two verified occurrence of this bug are both observed near an {{UNRESERVE}} operation (see lines above in the log).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)