You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@doris.apache.org by 刘波 <27...@qq.com.INVALID> on 2022/03/08 04:28:34 UTC

回复: be异常退出

开发者,您好:
&nbsp; &nbsp; gdb查看的结果如下图,可以看到关键信息signal SIGSEGV,但不会进一步分析具体异常原因,烦请帮忙看下,谢谢!






------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "dev"                                                                                    <wangbo13131@gmail.com&gt;;
发送时间:&nbsp;2022年3月4日(星期五) 中午1:31
收件人:&nbsp;"dev"<dev@doris.apache.org&gt;;

主题:&nbsp;Re: be异常退出



目前的堆栈看起来不足以支撑做出判断;
如果线上有开core dump的话,可以用gdb palo_be core_dump文件看看堆栈
后者看下be.out是否进程怪盗时的堆栈

刘波 <270309321@qq.com.invalid&gt; 于2022年3月4日周五 12:25写道:

&gt; 尊敬的开发者,您好:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 我们的doris
&gt; be节点经常会挂掉其中几个,经分析资源情况正常,具体信息可参见附件,日志层面未发现是OOM,未能定位出异常,请求协助,具体信息如下:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; *时间: 2022-03-04 10:54:43*
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; doris报错*:detailMessage = tablet 45664199 has few replicas: 1,
&gt; alive backends: [10004]*
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 环境信息:华为云cetnos 8 64位,16核64G
&gt;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; 系统日志: less /var/log/message(*非OOM*)
&gt;
&gt; Mar&nbsp; 4 10:54:38 narwal-doris-be-0004 systemd[1]: Started Process Core Dump
&gt; (PID 1881819/UID 0).
&gt;
&gt; Mar&nbsp; 4 10:54:41 narwal-doris-be-0004 systemd-coredump[1881820]: Core file
&gt; was truncated to 2147483648 bytes.
&gt;
&gt; Mar&nbsp; 4 10:55:00 narwal-doris-be-0004 systemd-coredump[1881820]: Process
&gt; 2800630 (palo_be) of user 0 dumped core.#012#012Stack trace of thread
&gt; 1877601:#012#0&nbsp; 0x00000000039480a2 memcpy
&gt;
&gt;&nbsp; (/mnt/be/lib/palo_be)#012#012Stack trace of thread 2800630:#012#0
&gt; 0x00007fccb330efc8 n/a (n/a)
&gt;
&gt; Mar&nbsp; 4 10:55:01 narwal-doris-be-0004 systemd[1]:
&gt; systemd-coredump@1-1881819-0.service: Succeeded.
&gt;
&gt; Mar&nbsp; 4 10:55:03 narwal-doris-be-0004 systemd[1]: session-12.scope:
&gt; Succeeded.
&gt;
&gt;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; dump信息:*coredumpctl info 2800630*
&gt;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; PID: 2800630 (palo_be)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; UID: 0 (root)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; GID: 0 (root)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Signal: 11 (SEGV)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Timestamp: Fri 2022-03-04 10:54:38 CST (1h 10min ago)
&gt;&nbsp;&nbsp; Command Line: /mnt/be/lib/palo_be
&gt;&nbsp;&nbsp;&nbsp;&nbsp; Executable: /mnt/be/lib/palo_be
&gt;&nbsp; Control Group: /
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Slice: -.slice
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Boot ID: ca3ef395d7c547d2aecb1c251097066f
&gt;&nbsp;&nbsp;&nbsp;&nbsp; Machine ID: 501f93b5c19d4ca38db845c29176e3c5
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Hostname: narwal-doris-be-0004
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Storage:
&gt; /var/lib/systemd/coredump/core.palo_be.0.ca3ef395d7c547d2aecb1c251097066f.2800630.1646362478000000.lz4
&gt; (truncated)
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Message: Process 2800630 (palo_be) of user 0 dumped core.
&gt;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Stack trace of thread 1877601:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; #0&nbsp; 0x00000000039480a2 memcpy (/mnt/be/lib/palo_be)
&gt;
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Stack trace of thread 2800630:
&gt;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; #0&nbsp; 0x00007fccb330efc8 n/a (n/a)
&gt;
&gt;
&gt;
&gt;
&gt;
&gt; ---------------------------------------------------------------------
&gt; To unsubscribe, e-mail: dev-unsubscribe@doris.apache.org
&gt; For additional commands, e-mail: dev-help@doris.apache.org



-- 
王博&nbsp; Wang Bo