You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by "Benjamin Hindman (JIRA)" <ji...@apache.org> on 2014/06/05 21:27:10 UTC
[jira] [Updated] (MESOS-547) MasterTest.MasterLost is flaky on
Jenkins
[ https://issues.apache.org/jira/browse/MESOS-547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Benjamin Hindman updated MESOS-547:
-----------------------------------
Sprint: (was: Q2)
> MasterTest.MasterLost is flaky on Jenkins
> -----------------------------------------
>
> Key: MESOS-547
> URL: https://issues.apache.org/jira/browse/MESOS-547
> Project: Mesos
> Issue Type: Bug
> Reporter: Benjamin Mahler
> Labels: twitter
>
> This is because the MasterLost test expects the Scheduler.registered call to happen a single time. However, on the Apache Jenkins machine, it runs slow enough for the reliable registration in the scheduler driver to retry registration (1 second timeout).
> The fix here will affect a lot of tests, so I'd like to do the right thing, possible fixes:
> EXPECT_CALL(sched, registered(&driver, _, _))
> .Times(1); // Change this to .WillRepeatedly(Return())
> Also, pausing the clock prior to starting the driver will work. But may have further impact on the tests.
> [ RUN ] MasterTest.MasterLost
> I0712 03:48:50.415976 4397 master.cpp:230] Master started on 67.195.138.9:47248
> I0712 03:48:50.416594 4399 slave.cpp:113] Slave started on 11)@67.195.138.9:47248
> I0712 03:48:50.417034 4395 sched.cpp:177] New master at master@67.195.138.9:47248
> I0712 03:48:50.423070 4397 master.cpp:245] Master ID: 201307120348-160088899-47248-4046
> I0712 03:48:50.431115 4399 slave.cpp:205] Slave resources: cpus=2; mem=1024; ports=[31000-32000]; disk=1024
> I0712 03:48:50.508512 4398 hierarchical_allocator_process.hpp:305] Initializing hierarchical allocator process with master : master@67.195.138.9:47248
> W0712 03:48:50.508554 4394 master.cpp:83] No whitelist given. Advertising offers for all slaves
> I0712 03:48:50.508767 4397 master.cpp:583] Elected as master!
> I0712 03:48:50.791844 4399 slave.cpp:529] New master detected at master@67.195.138.9:47248
> I0712 03:48:51.564163 4399 slave.cpp:544] Postponing registration until recovery is complete
> I0712 03:48:51.564257 4396 status_update_manager.cpp:155] New master detected at master@67.195.138.9:47248
> I0712 03:48:51.564288 4397 master.cpp:633] Registering framework 201307120348-160088899-47248-4046-0000 at scheduler(11)@67.195.138.9:47248
> I0712 03:48:51.751411 4399 slave.cpp:390] Finished recovery
> I0712 03:48:51.823364 4392 sched.cpp:222] Framework registered with 201307120348-160088899-47248-4046-0000
> I0712 03:48:51.823395 4397 master.cpp:620] Framework 201307120348-160088899-47248-4046-0000 (scheduler(11)@67.195.138.9:47248) already registered, resending acknowledgement
> I0712 03:48:51.823452 4395 hierarchical_allocator_process.hpp:335] Added framework 201307120348-160088899-47248-4046-0000
> I0712 03:48:52.110093 4395 hierarchical_allocator_process.hpp:704] No resources available to allocate!
> I0712 03:48:51.946835 4392 sched.cpp:194] No master detected, waiting for another master
> I0712 03:48:52.110155 4397 master.cpp:962] Attempting to register slave on vesta.apache.org at slave(11)@67.195.138.9:47248
> I0712 03:48:52.440776 4397 master.cpp:1929] Adding slave 201307120348-160088899-47248-4046-0 at vesta.apache.org with cpus=2; mem=1024; ports=[31000-32000]; disk=1024
> I0712 03:48:52.249390 4392 sched.cpp:222] Framework registered with 201307120348-160088899-47248-4046-0000
> ../../src/tests/master_tests.cpp:906: Failure
> Mock function called more times than expected - returning directly.
> Function call: registered(0x7fffaf90ca00, @0x2b7bfc004140 201307120348-160088899-47248-4046-0000, @0x2b7bfc003170 40-byte object <70-8B 2C-F5 7B-2B 00-00 00-00 00-00 00-00 00-00 F0-54 00-FC 7B-2B 00-00 43-C3 8A-09 90-B8 00-00 00-00 00-00 07-00 00-00>)
> Expected: to be called once
> Actual: called twice - over-saturated and active
> I0712 03:48:52.241426 4395 hierarchical_allocator_process.hpp:666] Performed allocation for 0 slaves in 131.314878ms
> I0712 03:48:52.448761 4399 slave.cpp:589] Registered with master master@67.195.138.9:47248; given slave ID 201307120348-160088899-47248-4046-0
> I0712 03:48:52.448806 4397 master.cpp:436] Master terminating
> I0712 03:48:52.481109 4397 slave.cpp:485] Slave asked to shut down by master@67.195.138.9:47248
> I0712 03:48:52.489028 4397 slave.cpp:440] Slave terminating
> I0712 03:48:52.456776 4392 sched.cpp:427] Stopping framework '201307120348-160088899-47248-4046-0000'
> I0712 03:48:52.481195 4046 master.cpp:209] Shutting down master
> I0712 03:48:52.464653 4395 hierarchical_allocator_process.hpp:448] Added slave 201307120348-160088899-47248-4046-0 (vesta.apache.org) with cpus=2; mem=1024; ports=[31000-32000]; disk=1024 (and cpus=2; mem=1024; ports=[31000-32000]; disk=1024 available)
> I0712 03:48:52.560148 4395 hierarchical_allocator_process.hpp:726] Offering cpus=2; mem=1024; ports=[31000-32000]; disk=1024 on slave 201307120348-160088899-47248-4046-0 to framework 201307120348-160088899-47248-4046-0000
> I0712 03:48:52.568372 4395 hierarchical_allocator_process.hpp:686] Performed allocation for slave 201307120348-160088899-47248-4046-0 in 8.255916ms
> [ FAILED ] MasterTest.MasterLost (2161 ms)
> [----------] 11 tests from MasterTest (15502 ms total)
--
This message was sent by Atlassian JIRA
(v6.2#6252)