[jira] [Created] (TRAFODION-675) LP Bug: 1377260 - Update statistics sees error 8838 and cores dtm_arkcmp at NAHeap::allocateHeapMemory()

["Alice Chen (JIRA)" <ji...@apache.org>]

Alice Chen created TRAFODION-675:
------------------------------------

             Summary: LP Bug: 1377260 - Update statistics sees error 8838 and cores dtm_arkcmp at NAHeap::allocateHeapMemory()
                 Key: TRAFODION-675
                 URL: https://issues.apache.org/jira/browse/TRAFODION-675
             Project: Apache Trafodion
          Issue Type: Bug
          Components: sql-cmp
            Reporter: Weishiun Tsai
            Assignee: Barry Fritchman
            Priority: Blocker


Update statistics would cause tdm_arkcmp to crash and dump cores at NAHeap::allocateHeapMemory(), presumably because of memory violation or exhaustion of the heap space.  This is seen during QA test runs and this problem leaves several tdm_arkcmp core files on the system.  A typical stack of the core is shown below.  This is seen on the hbase 0.98 v1002 build installed on a 4-node HortonWorks cluster Amethyst1.  The symptom of this problem is different from another bug report https://bugs.launchpad.net/trafodion/+bug/1376455 ‘update statistics fails ERROR[9200], ERROR[8841]’.  In this case, there is no error 9200 or 8841.  Since tdm_arkcmp on the target side has crashed, the client is seeing error 8838:

SQL>UPDATE STATISTICS FOR TABLE PATAB1 on every column;
*** ERROR[8838] Unable to receive reply from Compiler, possibly caused by internal errors when compiling SQL statements, processing DDL statements, or executing the builtin stored procedures. [2014-10-03 04:18:23]

Here is an example stack.

(gdb) bt
#0  0x00007fffedbca8a5 in raise () from /lib64/libc.so.6
#1  0x00007fffedbcc085 in abort () from /lib64/libc.so.6
#2  0x00007fffedbc3a1e in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007fffedbc3ae0 in __assert_fail () from /lib64/libc.so.6
#4  0x00007ffff6d85b09 in NAHeap::allocateHeapMemory (this=0x7fffdaf3f718,
    userSize=30, failureIsFatal=1) at ../common/NAMemory.cpp:3189
#5  0x00007ffff6d82721 in NAMemory::allocateMemory (this=0x7fffdaf3f718,
    size=30, failureIsFatal=1) at ../common/NAMemory.cpp:1353
#6  0x00007ffff6220775 in operator new[] (t=30, h=0x7fffdaf3f718)
    at ../export/NABasicObject.cpp:372
#7  0x00007ffff5afa281 in HSDataBuffer::operator= (this=0x1775b58,
    dataVal=0x1786da8 L"\124\111\115\105\040\047\060\061\072\060\061\072\060\061
", <incomplete sequence \047>) at ../ustat/hs_cli.h:258
#8  0x00007ffff5b7d8eb in FormatRow (srcDesc=0x7fffda0973b8,
    src=0x7fffd9585d08 "\020", target=...) at ../ustat/hs_util.cpp:663
#9  0x00007ffff5b28dcc in setBufferValue (value=..., group=0x7fffda0970d0,
    boundary=...) at ../ustat/hs_globals.cpp:13360
#10 0x00007ffff5b32bc2 in HSHistogram::addIntervalData<myVarChar> (
    this=0x7fffda09cb60, value=..., group=0x7fffda0970d0, numRows=1,
    bigGap=0, gapMagnitude=0, final=0) at ../ustat/hs_globals.h:2229
#11 0x00007ffff5b05ba8 in HSHistogram::processIntervalValues (
    this=0x7fffda09cb60, boundaryRowSet=0x7fffd957c0c0, group=0x7fffda0970d0,
    rowsInSet=@0x7fffffff9378, currentGapAvg=0)
    at ../ustat/hs_globals.cpp:2054
#12 0x00007ffff5aef6c5 in HSCursor::fetchBoundaries (this=0x7fffda0a0c88,
    group=0x7fffda0970d0, rowCount=@0x7fffffff9de8, intCount=50, sampleUsed=0)
    at ../ustat/hs_cli.cpp:3735
#13 0x00007ffff5b11bd2 in HSGlobalsClass::CollectStatistics (
    this=0x7fffffff9d70) at ../ustat/hs_globals.cpp:5587
#14 0x00007ffff5b7a7ea in UpdateStats (
    input=0x7fffdaf8b120 "UPDATE STATISTICS FOR TABLE PATAB1 on every column;",
requestedByCompiler=0) at ../ustat/hs_update.cpp:407
#15 0x00007ffff38cf2bc in CmpStatement::process (this=0x7fffdaf3f590,
    statement=...) at ../arkcmp/CmpStatement.cpp:824
#16 0x00007ffff38bbb45 in ExCmpMessage::actOnReceive (this=0x7fffffffbd40)
    at ../arkcmp/CmpConnection.cpp:405
#17 0x00007ffff6d5fc41 in IpcMessageStream::internalActOnReceive (
    this=0x7fffffffbd40, buffer=0xbacbc0, connection=0xbb4910)
    at ../common/Ipc.cpp:3553
#18 0x00007ffff6d5db67 in IpcMessageBuffer::callReceiveCallback (
    this=0xbacbc0, conn=0xbb4910) at ../common/Ipc.cpp:2427
#19 0x00007ffff6d71c50 in GuaConnectionToClient::acceptBuffer (this=0xbb4910,
    buffer=0xbacbc0, receivedDataLength=448) at ../common/IpcGuardian.cpp:2467
#20 0x00007ffff6d73667 in GuaReceiveControlConnection::wait (this=0xba4140,
    timeout=-1, eventConsumed=0x0, ipcAwaitiox=0x0)
    at ../common/IpcGuardian.cpp:3164
#21 0x00007ffff6d7106e in GuaConnectionToClient::wait (this=0xbb4910,
    timeout=-1, eventConsumed=0x0, ipcAwaitiox=0x0)
    at ../common/IpcGuardian.cpp:2136
#22 0x00007ffff6d5bff9 in IpcSetOfConnections::waitOnSet (
    this=0x7fffffffbee0, timeout=-1, calledByESP=0, timedout=0x0)
    at ../common/Ipc.cpp:1709
#23 0x00007ffff6d5f287 in IpcMessageStream::waitOnMsgStream (
    this=0x7fffffffbd40, timeout=-1) at ../common/Ipc.cpp:3272
#24 0x00007ffff6d5f1f5 in IpcMessageStream::receive (this=0x7fffffffbd40,
    waited=1) at ../common/Ipc.cpp:3254
#25 0x0000000000407bdc in main (argc=2, argv=0x7fffffffc4d8)
    at ../bin/arkcmp.cpp:303
(gdb)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)