You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mesos.apache.org by "haosdent (JIRA)" <ji...@apache.org> on 2016/10/01 07:31:20 UTC
[jira] [Commented] (MESOS-6293)
HealthCheckTest.HealthyTaskViaHTTPWithoutType fails on some distros.
[ https://issues.apache.org/jira/browse/MESOS-6293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15538091#comment-15538091 ]
haosdent commented on MESOS-6293:
---------------------------------
Try in aws c4.4 xlarge with Ubuntu 14.04 and Ubuntu 16.04, curl version
{code}
# 16.04
curl --version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets
# 14.04
curl --version
curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP
{code}
Run with stress {{stress -c 16 -m 16 -i 16 -d 1}} after 3000 runs and could not reproduce the failure.
{code}
MESOS_VERBOSE=1 GLOG_v=1 GTEST_REPEAT=-1 GTEST_BREAK_ON_FAILURE=1 GTEST_FILTER="HealthCheckTest.HealthyTaskViaHTTPWithoutType" bin/mesos-tests.sh
{code}
Try in a virualbox slow vm (Ubuntu 14.04), could not reproduce as well.
> HealthCheckTest.HealthyTaskViaHTTPWithoutType fails on some distros.
> --------------------------------------------------------------------
>
> Key: MESOS-6293
> URL: https://issues.apache.org/jira/browse/MESOS-6293
> Project: Mesos
> Issue Type: Bug
> Reporter: Alexander Rukletsov
> Assignee: haosdent
> Labels: health-check, mesosphere
>
> I see consistent failures of this test in the internal CI in *some* distros, specifically CentOS 6, Ubuntu 14, 15, 16. The source of the health check failure is always the same: {{curl}} cannot connect to the target:
> {noformat}
> Received task health update, healthy: false
> W0929 17:22:05.270992 2730 health_checker.cpp:204] Health check failed 1 times consecutively: HTTP health check failed: curl returned exited with status 7: curl: (7) couldn't connect to host
> I0929 17:22:05.273634 26850 slave.cpp:3609] Handling status update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000 from executor(1)@172.30.2.20:58660
> I0929 17:22:05.274178 26844 status_update_manager.cpp:323] Received status update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000
> I0929 17:22:05.274226 26844 status_update_manager.cpp:377] Forwarding update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000 to the agent
> I0929 17:22:05.274314 26845 slave.cpp:4026] Forwarding the update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000 to master@172.30.2.20:38955
> I0929 17:22:05.274415 26845 slave.cpp:3920] Status update manager successfully handled status update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000
> I0929 17:22:05.274436 26845 slave.cpp:3936] Sending acknowledgement for status update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000 to executor(1)@172.30.2.20:58660
> I0929 17:22:05.274534 26849 master.cpp:5661] Status update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000 from agent 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-S0 at slave(77)@172.30.2.20:38955 (ip-172-30-2-20.mesosphere.io)
> ../../src/tests/health_check_tests.cpp:1398: Failure
> I0929 17:22:05.274567 26849 master.cpp:5723] Forwarding status update TASK_RUNNING (UUID: f5408ac9-f6ba-447f-b3d7-9dce44384ffe) for task aa0792d3-8d85-4c32-bd04-56a9b552ebda in health state unhealthy of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000
> Value of: statusHealth.get().healthy()
> Actual: false
> Expected: true
> I0929 17:22:05.274636 26849 master.cpp:7560] Updating the state of task aa0792d3-8d85-4c32-bd04-56a9b552ebda of framework 2e0e9ea1-0ae5-4f28-80bb-a9abc56c5a6f-0000 (latest state: TASK_RUNNING, status update state: TASK_RUNNING)
> I0929 17:22:05.274829 26844 sched.cpp:1025] Scheduler::statusUpdate took 43297ns
> Received SHUTDOWN event
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)