You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2017/08/24 21:39:00 UTC
[jira] [Resolved] (IMPALA-5578) Impala services crash after Linux
kernel update
[ https://issues.apache.org/jira/browse/IMPALA-5578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-5578.
-----------------------------------
Resolution: Information Provided
> Impala services crash after Linux kernel update
> -----------------------------------------------
>
> Key: IMPALA-5578
> URL: https://issues.apache.org/jira/browse/IMPALA-5578
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.7.0, Impala 2.7.1
> Environment: # cat /etc/centos-release
> CentOS release 6.9 (Final)
> # cat /etc/grub.conf
> default=0
> timeout=1
> title CentOS (2.6.32-696.3.2.el6.x86_64)
> root (hd0)
> kernel /boot/vmlinuz-2.6.32-696.3.2.el6.x86_64 root=LABEL=centos_root ro crashkernel=auto LANG=en_US.UTF-8 KEYTABLE=us
> initrd /boot/initramfs-2.6.32-696.3.2.el6.x86_64.img
> Reporter: Neil Johnson
> Attachments: cloudera-package-list.txt, hs_err_pid3044.log, impalad.ip-10-45-1-67.impala.log.ERROR.20170626-120509.3044, impalad.ip-10-45-1-67.impala.log.INFO.20170626-120509.3044, impala-logs-20170626-142537.tar.gz
>
>
> I had a working configuration (Impala Shell v2.7.0-cdh5.10.1) and updated the Linux kernel from "2.6.32-642.4.2.el6" to "2.6.32-696.3.2.el6" as below:
> {code}
> Installing:
> kernel x86_64 2.6.32-696.3.2.el6 updates 32 M
> kernel-devel x86_64 2.6.32-696.3.2.el6 updates 11 M
> Updating:
> java-1.7.0-openjdk x86_64 1:1.7.0.141-2.6.10.1.el6_9 updates 26 M
> java-1.7.0-openjdk-devel x86_64 1:1.7.0.141-2.6.10.1.el6_9 updates 9.5 M
> kernel-firmware noarch 2.6.32-696.3.2.el6 updates 29 M
> kernel-headers x86_64 2.6.32-696.3.2.el6 updates 4.5 M
> {code}
> After this impala services crash at startup, this does not happen on CDH5.9, but does happen with CDH 5.10.0 and 5.10.1.
> {code}
> # service impala-catalog status
> Impala Catalog Server is dead and pid file exists [FAILED]
> # service impala-server status
> Impala Server is dead and pid file exists [FAILED]
> # service impala-state-store status
> Impala State Store Server is running [ OK ]
> {code}
> In impalad.ip-10-45-1-67.impala.log.INFO.20170626-120509.3044 I see:
> {code}
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGBUS (0x7) at pc=0x00007f028e8e046f, pid=3044, tid=139649127274624
> #
> # JRE version: (7.0_71-b14) (build )
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (24.71-b01 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # j java.lang.Object.<clinit>()V+0
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /var/lib/impala/hs_err_pid3044.log
> {code}
> I've uploaded hs_err_pid3044.log for reference.
> If I switch to the previous kernel and restart by changing grub.conf "default" to "1" as below then impala services start successfully:
> {code}
> # cat /etc/grub.conf
> default=0
> timeout=1
> title CentOS (2.6.32-696.3.2.el6.x86_64)
> root (hd0)
> kernel /boot/vmlinuz-2.6.32-696.3.2.el6.x86_64 root=LABEL=centos_root ro crashkernel=auto LANG=en_US.UTF-8 KEYTABLE=us
> initrd /boot/initramfs-2.6.32-696.3.2.el6.x86_64.img
> title CentOS (2.6.32-642.4.2.el6.x86_64)
> root (hd0)
> kernel /boot/vmlinuz-2.6.32-642.4.2.el6.x86_64 root=LABEL=centos_root ro crashkernel=auto LANG=en_US.UTF-8 KEYTABLE=us
> initrd /boot/initramfs-2.6.32-642.4.2.el6.x86_64.img
> # shutdown -r now
> {code}
> Likewise if I repeat the above with "default=0" then the issue returns.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)