You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Bikramjeet Vig (JIRA)" <ji...@apache.org> on 2018/10/18 01:26:00 UTC
[jira] [Commented] (IMPALA-7696) llvm crash in custom cluster test
[ https://issues.apache.org/jira/browse/IMPALA-7696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654512#comment-16654512 ]
Bikramjeet Vig commented on IMPALA-7696:
----------------------------------------
Since there really is no info in the logs to figure out which test this was a part of, I tried to follow the stack-trace through codegen codebase. It seems like this is failing in some part of the codebase that does not uses input from impala (happens while creating and registering passes that needs to be done before it even touches the module created by impala). I might be missing something more subtle here, but that all I could figure out for now unless we find this again and figure out which test crashed
> llvm crash in custom cluster test
> ---------------------------------
>
> Key: IMPALA-7696
> URL: https://issues.apache.org/jira/browse/IMPALA-7696
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: Michael Brown
> Assignee: Bikramjeet Vig
> Priority: Critical
> Labels: crash
>
> [~bikramjeet.vig] please take a look at this crash that happened on GVO.
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3346/
> {noformat}
> Crash reason: SIGSEGV
> Crash address: 0x0
> Process uptime: not available
> Thread 202 (crashed)
> 0 libc-2.23.so + 0x16efe5
> rax = 0x0000000000000000 rdx = 0x0000000000000010
> rcx = 0x00007fab7869cfe5 rbx = 0x0000000000000000
> rsi = 0x00000000046edc91 rdi = 0x0000000000000010
> rbp = 0x000000000665d290 rsp = 0x00007faae72d5468
> r8 = 0x0000000000000081 r9 = 0x000000000a92f5c0
> r10 = 0x0000000000000001 r11 = 0x00007fab786c2f90
> r12 = 0x0000000000000010 r13 = 0x0000000000000086
> r14 = 0x00000000046edc81 r15 = 0x0000000007731b00
> rip = 0x00007fab7869cfe5
> Found by: given as instruction pointer in context
> 1 impalad!llvm::cl::generic_parser_base::findOption(llvm::StringRef) + 0x54
> rsp = 0x00007faae72d5470 rip = 0x0000000004392784
> Found by: stack scanning
> 2 impalad!llvm::PassNameParser::passRegistered(llvm::PassInfo const*) + 0x67
> rbx = 0x000000000665d288 rbp = 0x000000000af15250
> rsp = 0x00007faae72d54a0 r12 = 0x00000000046edc81
> r13 = 0x000000000432d430 r14 = 0x000000000665d290
> rip = 0x000000000432e817
> Found by: call frame info
> 3 impalad!llvm::PassRegistry::registerPass(llvm::PassInfo const&, bool) + 0x342
> rbx = 0x000000000af15250 rbp = 0x000000000777d720
> rsp = 0x00007faae72d54f0 r12 = 0x000000000777d720
> r13 = 0x000000000432d430 r14 = 0x0000000000000010
> r15 = 0x0000000007731b00 rip = 0x00000000043347f2
> Found by: call frame info
> 4 impalad!initializeMachineCombinerPassOnce(llvm::PassRegistry&) + 0x92
> rbx = 0x000000000af15250 rbp = 0x0000000007731b00
> rsp = 0x00007faae72d5560 r12 = 0x00007faae72d5590
> r13 = 0x0000000000000000 r14 = 0x0000000000000000
> r15 = 0x0000000000000000 rip = 0x00000000039f9a62
> Found by: call frame info
> 5 libpthread-2.23.so + 0xea99
> rbx = 0x000000000664773c rbp = 0x00007fab790ebac0
> rsp = 0x00007faae72d5580 r12 = 0x00007faae72d5590
> r13 = 0x0000000000000000 r14 = 0x0000000000000000
> r15 = 0x0000000000000000 rip = 0x00007fab78906a99
> Found by: call frame info
> 6 impalad!llvm::APInt::operator[](unsigned int) const [APInt.h : 1098 + 0x1e]
> rbp = 0x00007fab790ebac0 rsp = 0x00007faae72d5590
> rip = 0x0000000002157216
> Found by: stack scanning
> 7 libpthread-2.23.so + 0xeae0
> rbp = 0x00007fab790ebac0 rsp = 0x00007faae72d5598
> rip = 0x00007fab78906ae0
> Found by: stack scanning
> 8 impalad!llvm::initializeMachineCombinerPass(llvm::PassRegistry&) + 0x5e
> rbp = 0x00007fab790ebac0 rsp = 0x00007faae72d55d0
> rip = 0x00000000039fa06e
> Found by: stack scanning
> 9 impalad!llvm::initializeCodeGen(llvm::PassRegistry&) + 0x109
> rbp = 0x00007fab790ebac0 rsp = 0x00007faae72d55f0
> rip = 0x0000000003ad33f9
> Found by: call frame info
> 10 impalad!llvm::TargetPassConfig::TargetPassConfig(llvm::LLVMTargetMachine&, llvm::legacy::PassManagerBase&) + 0xc2
> rbx = 0x000000000af21720 rbp = 0x00007fab790ebac0
> rsp = 0x00007faae72d5600 rip = 0x0000000003aa3432
> Found by: call frame info
> 11 impalad!llvm::X86TargetMachine::createPassConfig(llvm::legacy::PassManagerBase&) + 0x25
> rbx = 0x000000000af21720 rbp = 0x000000000aef5380
> rsp = 0x00007faae72d5620 rip = 0x0000000003546395
> Found by: call frame info
> 12 impalad!addPassesToGenerateCode(llvm::LLVMTargetMachine*, llvm::legacy::PassManagerBase&, bool, void const*, void const*, void const*, void const*) + 0x28
> rbx = 0x000000000bc8c268 rbp = 0x00007faae72d57a0
> rsp = 0x00007faae72d5640 r12 = 0x000000000aef5380
> rip = 0x00000000039e0438
> Found by: call frame info
> 13 impalad!llvm::LLVMTargetMachine::addPassesToEmitMC(llvm::legacy::PassManagerBase&, llvm::MCContext*&, llvm::raw_pwrite_stream&, bool) + 0x37
> rbx = 0x000000000bc8c268 rbp = 0x00007faae72d57a0
> rsp = 0x00007faae72d5690 r12 = 0x00007faae72d57a0
> r13 = 0x00007faae72d57d0 r14 = 0x000000000bc8c320
> r15 = 0x000000000aef5380 rip = 0x00000000039e1d07
> Found by: call frame info
> 14 impalad!llvm::MCJIT::emitObject(llvm::Module*) + 0xde
> rbx = 0x00007faae72d5800 rbp = 0x000000000bc8c000
> rsp = 0x00007faae72d5780 r12 = 0x00007faae72d57a0
> r13 = 0x00007faae72d57d0 r14 = 0x000000000bc8c320
> r15 = 0x00007faae72d5818 rip = 0x0000000003f4c2de
> Found by: call frame info
> 15 impalad!llvm::MCJIT::generateCodeForModule(llvm::Module*) + 0x511
> rbx = 0x000000000a722800 rbp = 0x000000000bc8c308
> rsp = 0x00007faae72d6860 r12 = 0x000000000bc8c2e0
> r13 = 0x000000000bc8c250 r14 = 0x000000000bc8c320
> r15 = 0x000000000bc8c000 rip = 0x0000000003f4c9e1
> Found by: call frame info
> 16 impalad!llvm::MCJIT::finalizeObject() + 0x1c1
> rbx = 0x00007faae72d69a0 rbp = 0x00007faae72d69c0
> {noformat}
> I can't figure out which test may have caused it, other than it's a custom cluster test. The minidump path is the clue there.
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/3346/artifact/Impala/logs_static/logs/custom_cluster_tests/minidumps/impalad/
> All the tests passed. The timestamp of the minidump was not preserved across the archive, so I can't backtrack the timestamp of the file to the timestamps of the tests running at the time. I've combed through custom cluster log sections and can't find anything, either.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org