You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@qpid.apache.org by "Charles E. Rolke (Jira)" <ji...@apache.org> on 2021/02/12 14:44:00 UTC
[jira] [Commented] (DISPATCH-1660) Intermittent failure in
system_tests_policy_oversize_basic
[ https://issues.apache.org/jira/browse/DISPATCH-1660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17283734#comment-17283734 ]
Charles E. Rolke commented on DISPATCH-1660:
--------------------------------------------
An Ubuntu Xenial job fails intermittently.
A recently failed job:
[https://travis-ci.com/github/apache/qpid-dispatch/jobs/481281093]
Fails with:
{code:java}
Start 25: system_tests_policy_oversize_basic
25: Test command: /usr/bin/python "/home/travis/build/apache/qpid-dispatch/build/tests/run.py" "-m" "unittest" "-v" "system_tests_policy_oversize_basic"
25: Test timeout computed to be: 600
25: test_40_block_oversize_INTA_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
25: test_41_block_oversize_INTA_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
25: test_42_block_oversize_INTA_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
25: test_43_block_oversize_INTA_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
25: test_44_block_oversize_INTB_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
25: test_45_block_oversize_INTB_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
*** Error in `qdrouterd': free(): invalid pointer: 0x0000000001f63160 ***
======= Memory map: ========
01b07000-02457000 rw-p 00000000 00:00 0 [heap]
...
{code}
The pointer is within the heap as reported by the failed-test router dump and is probably an indication of corrupted heap.
The issue is not reproducible on a local docker image. It has run overnight several times with no problem, 3600 passes just running the oversize_basic test.
The docker image is set like this on a Fedora 32 host:
{code:java}
Run from console prompt in docker image
# inxi -F
System: Host: b244a41a8c76 Kernel: 5.10.11-100.fc32.x86_64 x86_64 (64 bit) Console: tty 0
Distro: Ubuntu 16.04 xenial
Machine: System: LENOVO (portable) product: 20EQS64N0B v: ThinkPad P50 serial: PC0WXYZP
Mobo: LENOVO model: 20EQS64N0B serial: L1HF86D9999 Bios: LENOVO v: N1EET74W (1.47 ) date: 11/21/2017
CPU: Quad core Intel Core i7-6820HQ (-HT-MCP-) cache: 8192 KB
clock speeds: max: 3600 MHz 1: 800 MHz 2: 800 MHz 3: 800 MHz 4: 800 MHz 5: 800 MHz 6: 800 MHz
7: 800 MHz 8: 800 MHz
Graphics: Card: NVIDIA GM107GLM [Quadro M1000M]
Display Server: N/A driver: N/A tty size: 205x38 Advanced Data: N/A for root out of X
Audio: Card-1 Intel 100 Series/C230 Series Family HD Audio Controller driver: snd_hda_intel
Card-2 NVIDIA Device 0fbc driver: snd_hda_intel
Network: Card-1: Intel Ethernet Connection (2) I219-LM driver: e1000e
IF: null-if-id state: N/A speed: N/A duplex: N/A mac: N/A
Card-2: Intel Wireless 8260 driver: iwlwifi
IF: null-if-id state: N/A mac: N/A
Drives: HDD Total Size: NA (-) ID-1: /dev/nvme0n1 model: N/A size: 512.1GB
Partition: ID-1: / size: 74G used: 43G (62%) fs: overlay dev: N/A
ID-2: swap-1 size: 34.36GB used: 0.00GB (0%) fs: swap dev: /dev/dm-1
RAID: No RAID devices: /proc/mdstat, md_mod kernel module present
Sensors: System Temperatures: cpu: 49.0C mobo: 48.0C
Fan Speeds (in rpm): cpu: 0 fan-1: 0
Info: Processes: 3 Uptime: 13 days Memory: 13382.6/31551.5MB Init: N/A Client: Shell (bash) inxi: 2.2.35
{code}
> Intermittent failure in system_tests_policy_oversize_basic
> ----------------------------------------------------------
>
> Key: DISPATCH-1660
> URL: https://issues.apache.org/jira/browse/DISPATCH-1660
> Project: Qpid Dispatch
> Issue Type: Bug
> Components: Tests
> Affects Versions: 1.12.0
> Reporter: Ganesh Murthy
> Assignee: Charles E. Rolke
> Priority: Major
>
> {noformat}
> Start 25: system_tests_policy_oversize_basic25: Test command: /usr/bin/python "/foo/qpid-dispatch/build/tests/run.py" "-m" "unittest" "-v" "system_tests_policy_oversize_basic"
> 25: Test timeout computed to be: 600
> 25: test_40_block_oversize_INTA_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: test_41_block_oversize_INTA_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: test_42_block_oversize_INTA_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: test_43_block_oversize_INTA_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: test_44_block_oversize_INTB_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: test_45_block_oversize_INTB_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: OversizeMessageTransferTest - e46
> 25: 2020-05-20 12:11:40.002177 on_start
> 25: 2020-05-20 12:11:40.002267 on_start: opening receiver connection to amqp://0.0.0.0:24197
> 25: 2020-05-20 12:11:40.002545 on_start: opening sender connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:11:40.002714 on_start: Creating receiver
> 25: 2020-05-20 12:11:40.002878 on_start: Creating sender
> 25: 2020-05-20 12:11:40.002997 on_start: done
> 25: 2020-05-20 12:12:40.003628 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:12:45.289228 test_46 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_46_block_oversize_INTB_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: OversizeMessageTransferTest - e47
> 25: 2020-05-20 12:12:45.310757 on_start
> 25: 2020-05-20 12:12:45.310873 on_start: opening receiver connection to amqp://0.0.0.0:24199
> 25: 2020-05-20 12:12:45.311197 on_start: opening sender connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:12:45.311385 on_start: Creating receiver
> 25: 2020-05-20 12:12:45.311570 on_start: Creating sender
> 25: 2020-05-20 12:12:45.311701 on_start: done
> 25: 2020-05-20 12:13:45.311633 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:13:48.055263 test_47 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_47_block_oversize_INTB_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: test_48_block_oversize_EA1_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: OversizeMessageTransferTest - e49
> 25: 2020-05-20 12:13:48.275921 on_start
> 25: 2020-05-20 12:13:48.275995 on_start: opening receiver connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:13:48.276285 on_start: opening sender connection to amqp://0.0.0.0:24197
> 25: 2020-05-20 12:13:48.276454 on_start: Creating receiver
> 25: 2020-05-20 12:13:48.276615 on_start: Creating sender
> 25: 2020-05-20 12:13:48.276735 on_start: done
> 25: 2020-05-20 12:14:48.277283 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:14:51.017235 test_49 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_49_block_oversize_EA1_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: test_4a_block_oversize_EA1_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: OversizeMessageTransferTest - e4b
> 25: 2020-05-20 12:14:51.235405 on_start
> 25: 2020-05-20 12:14:51.235481 on_start: opening receiver connection to amqp://0.0.0.0:24199
> 25: 2020-05-20 12:14:51.235753 on_start: opening sender connection to amqp://0.0.0.0:24197
> 25: 2020-05-20 12:14:51.235920 on_start: Creating receiver
> 25: 2020-05-20 12:14:51.236082 on_start: Creating sender
> 25: 2020-05-20 12:14:51.236201 on_start: done
> 25: 2020-05-20 12:15:51.236483 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:15:51.246181 test_4b test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_4b_block_oversize_EA1_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: test_4c_block_oversize_EB1_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... skipped 'EB1 sending to INT.A may be blocked by EB1 limit and also by INT.B limit. That condition is tested in compound test.'
> 25: test_4d_block_oversize_EB1_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... skipped 'EB1 sending to INT.B may be blocked by EB1 limit and also by INT.B limit. That condition is tested in compound test.'
> 25: test_4e_block_oversize_EB1_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... skipped 'EB1 sending to EA1 may be blocked by EB1 limit and also by INT.B limit. That condition is tested in compound test.'
> 25: test_4f_block_oversize_EB1_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: test_50_allow_undersize_INTA_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: OversizeMessageTransferTest - e51
> 25: 2020-05-20 12:15:53.637504 on_start
> 25: 2020-05-20 12:15:53.637586 on_start: opening receiver connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:15:53.637871 on_start: opening sender connection to amqp://0.0.0.0:24194
> 25: 2020-05-20 12:15:53.638042 on_start: Creating receiver
> 25: 2020-05-20 12:15:53.638208 on_start: Creating sender
> 25: 2020-05-20 12:15:53.638336 on_start: done
> 25: 2020-05-20 12:16:53.638800 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:16:56.375683 test_51 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_51_allow_undersize_INTA_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: test_52_allow_undersize_INTA_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... ok
> 25: OversizeMessageTransferTest - e53
> 25: 2020-05-20 12:16:56.981609 on_start
> 25: 2020-05-20 12:16:56.981683 on_start: opening receiver connection to amqp://0.0.0.0:24199
> 25: 2020-05-20 12:16:56.981950 on_start: opening sender connection to amqp://0.0.0.0:24194
> 25: 2020-05-20 12:16:56.982119 on_start: Creating receiver
> 25: 2020-05-20 12:16:56.982283 on_start: Creating sender
> 25: 2020-05-20 12:16:56.982401 on_start: done
> 25: 2020-05-20 12:17:56.983678 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:17:56.992594 test_53 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_53_allow_undersize_INTA_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: OversizeMessageTransferTest - e54
> 25: 2020-05-20 12:17:57.011138 on_start
> 25: 2020-05-20 12:17:57.011215 on_start: opening receiver connection to amqp://0.0.0.0:24194
> 25: 2020-05-20 12:17:57.011485 on_start: opening sender connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:17:57.011653 on_start: Creating receiver
> 25: 2020-05-20 12:17:57.011811 on_start: Creating sender
> 25: 2020-05-20 12:17:57.011929 on_start: done
> 25: 2020-05-20 12:18:57.012497 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:18:59.752664 test_54 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_54_allow_undersize_INTB_INTA (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: OversizeMessageTransferTest - e55
> 25: 2020-05-20 12:18:59.772203 on_start
> 25: 2020-05-20 12:18:59.772287 on_start: opening receiver connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:18:59.772555 on_start: opening sender connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:18:59.772723 on_start: Creating receiver
> 25: 2020-05-20 12:18:59.772885 on_start: Creating sender
> 25: 2020-05-20 12:18:59.773005 on_start: done
> 25: 2020-05-20 12:19:59.772590 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:20:02.522946 test_55 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_55_allow_undersize_INTB_INTB (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: OversizeMessageTransferTest - e56
> 25: 2020-05-20 12:20:02.542721 on_start
> 25: 2020-05-20 12:20:02.542801 on_start: opening receiver connection to amqp://0.0.0.0:24197
> 25: 2020-05-20 12:20:02.543089 on_start: opening sender connection to amqp://0.0.0.0:24195
> 25: 2020-05-20 12:20:02.543269 on_start: Creating receiver
> 25: 2020-05-20 12:20:02.543433 on_start: Creating sender
> 25: 2020-05-20 12:20:02.543553 on_start: done
> 25: 2020-05-20 12:21:02.543622 self.timeout Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: 2020-05-20 12:21:05.284396 test_56 test error: Timeout Expired: n_sent=0 n_rcvd=0 n_rejected=0 n_aborted=0
> 25: test_56_allow_undersize_INTB_EA1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ... FAIL
> 25: test_57_allow_undersize_INTB_EB1 (system_tests_policy_oversize_basic.MaxMessageSizeBlockOversize) ...
> 25/66 Test #25: system_tests_policy_oversize_basic ................***Timeout 600.01 sec
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@qpid.apache.org
For additional commands, e-mail: dev-help@qpid.apache.org