You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafodion.apache.org by "Alice Chen (JIRA)" <ji...@apache.org> on 2015/07/22 20:20:12 UTC
[jira] [Created] (TRAFODION-1148) LP Bug: 1442258 - Startup failed,
exec of mxsscp created core file
Alice Chen created TRAFODION-1148:
-------------------------------------
Summary: LP Bug: 1442258 - Startup failed, exec of mxsscp created core file
Key: TRAFODION-1148
URL: https://issues.apache.org/jira/browse/TRAFODION-1148
Project: Apache Trafodion
Issue Type: Bug
Reporter: Christopher Sheedy
Assignee: Vladimir Berman
Priority: Critical
Trafodion startup hung when the mxsscp file made a core file. This happened repeatedly for at least 2 days, to different types of jobs, all running on the single Jenkins node cm53-172.16.0.72.
After a debugging session that included alchen, marvin-anderson, gonzalo.correa, justin-du-2 & chris-sheedy, we believe the problem was due to the kernel’s maximum pid size being too big. On that Jenkins node, and no other nodes, the value of kernel.pid_max was 4194303. On all other nodes, it is 32768 which is a value known to work for Trafodion.
The immediate problem was solved by changing this node with:
/sbin/sysctl -w kernel.pid_max=32768
The value can be seen with either of these commands:
$ /sbin/sysctl -n kernel.pid_max
32768
$ cat /proc/sys/kernel/pid_max
32768
Alchen has already made change 1476 in trafodion/infra, “Make sure kernel.pid_max is set to some reasonable value”.
Symptoms reported by arvind-narain :
In http://logs.trafodion.org/65/1465/1/gate/jdbc_test-cm5.3/652d98c/Install_Start.log :
Checking if processes are up.
Checking attempt: 1; user specified max: 1. Execution time in seconds: 0.
The SQ environment is not up all, or partially up and not operational. Check the logs.
Process Configured Actual Down
------- ---------- ------ ----
DTM 2 2
RMS 4 3 \$ZSC000 \$ZSM000 \$ZSM001
MXOSRVR 6 0 6
....................................................................................................Wed Apr 8 22:44:43 UTC 2015
Checking if processes are up.
Checking attempt: 1; user specified max: 1. Execution time in seconds: 0.
The SQ environment is not up all, or partially up and not operational. Check the logs.
Process Configured Actual Down
------- ---------- ------ ----
DTM 2 2
RMS 4 3 \$ZSC000 \$ZSM000 \$ZSM001
MXOSRVR 6 0 6
.........................................................................
http://logs.trafodion.org/65/1465/1/gate/jdbc_test-cm5.3/652d98c/console.html
***INFO: starting Trafodion instance
Build timed out (after 90 minutes). Marking the build as failed.
Build was aborted
http://logs.trafodion.org/65/1465/1/gate/jdbc_test-cm5.3/652d98c/traf_run/logs/sqmon.log
[$Z00kFVV] Shell/shell Version 1.0.1 Release 1.1.0 (Build debug [1.0.0-354-gfd1e290master_Bld439], date 20150408_212656)
set {process $ZSC001 } PERSIST_RETRIES=10,60
[$Z00kFVV] %set {process $ZSC001 } PERSIST_RETRIES=10,60
set {process $ZSC001 } PERSIST_ZONES=1,1
[$Z00kFVV] %set {process $ZSC001 } PERSIST_ZONES=1,1
exec {nowait, nid 1, name $ZSC001, out stdout_ZSC001 } mxsscp
[$Z00kFVV] %exec {nowait, nid 1, name $ZSC001, out stdout_ZSC001 } mxsscp
http://logs.trafodion.org/65/1465/1/gate/jdbc_test-cm5.3/652d98c/traf_run/logs/mon.20150408.22.10.41.slave-cm53.2204803.log
2015-04-08 22:11:19,069, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067: Executor assertion failure in file
2015-04-08 22:11:19,069, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067: ../cli/Globals.cp
2015-04-08 22:11:19,069, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067: on line
2015-04-08 22:11:19,070, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067: 17
2015-04-08 22:11:19,070, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067:
2015-04-08 22:11:19,070, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067: Message:
2015-04-08 22:11:19,070, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067: Could not initialize CLI globals.ComRtGetProgramInfo returned an error :201
2015-04-08 22:11:19,070, INFO, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204822, Message ID: 101130801, STDERR redirected from slave-cm53.$Z00kFVH.0.2206067:
2015-04-08 22:11:19,179, ERROR, MON, Node Number: 0,, PIN: 2204803 , Process Name: $MONITOR,,, TID: 2204839, Message ID: 101031202, [CProcessContainer::Exit_Process(process)], Exiting process $ZSC000 (0, 2206067) did not complete startup
(End of symptoms from arvind-narain)
Another symptom was a repeated pid in this file. This should not happen, each process should have a unique pid:
$ grep 2206067 monitor.map*
monitor.map.0.slave-cm53:BEGIN Wed Apr 8 22:11:18 2015 $ZSC000 (0, 2206067:22) P(0, 2206065:21) /home/jenkins/workspace/jdbc_test-cm5.3/traf_run/sql/scripts/mxsscp
monitor.map.0.slave-cm53:BEGIN Wed Apr 8 22:11:19 2015 $Z00kFVH (0, 2206067:23) P(-1, -1:-1) mxsscp
monitor.map.0.slave-cm53:END Wed Apr 8 22:11:19 2015 $ZSC000 (0, 2206067:22) P(0, 2206065:21) /home/jenkins/workspace/jdbc_test-cm5.3/traf_run/sql/scripts/mxsscp
Examining the core file:
$ cd /home/jenkins/workspace/pyodbc_test-cm5.3/traf_run
$ . ./sqenv.sh
$ core_bt /home/jenkins/workspace/pyodbc_test-cm5.3/traf_run/sql/scripts/core.slave-cm53.2091789.mxsscp
core file : -rw-rw-r-- 1 trafodion trafodion 78217216 Apr 8 19:24 ./sql/scripts/core.slave-cm53.2091789.mxsscp
executable : -rwxr-x--- 1 trafodion trafodion 3449752 Apr 8 19:01 /home/jenkins/workspace/pyodbc_test-cm5.3/traf_run/export/bin64d/mxsscp
gdb command:
gdb /home/jenkins/workspace/pyodbc_test-cm5.3/traf_run/export/bin64d/mxsscp ./sql/scripts/core.slave-cm53.2091789.mxsscp --batch -n -x /tmp/tmp.2gEkXOZQ2D 2>&1
(skipping many Missing separate debuginfo for ...)
[New Thread 2091789]
[New Thread 2091793]
[New Thread 2091792]
[New Thread 2091794]
[Thread debugging using libthread_db enabled]
Core was generated by `mxsscp SQMON1.1 00000 00000 2091789 $ZSC000 172.16.0.72:36948 00004 00000 00022'.
Program terminated with signal 6, Aborted.
#0 0x00000031eb832625 in raise () from /lib64/libc.so.6
#0 0x00000031eb832625 in raise () from /lib64/libc.so.6
#1 0x00000031eb833e05 in abort () from /lib64/libc.so.6
#2 0x00007ffff3f06f4c in assert_botch_abend (f=0x47835c "../cli/Globals.cpp", l=171, m=0x7fffffffbfa0 "Could not initialize CLI globals.ComRtGetProgramInfo returned an error :
201.", c=0x0) at ../export/NAAbort.cpp:282
#3 0x000000000042c3e9 in CliGlobals::init (this=0xc20c00, espProcess=0, statsGlobals=0x0) at ../cli/Globals.cpp:171
#4 0x000000000042c0f9 in CliGlobals::CliGlobals (this=0xc20c00, espProcess=0) at ../cli/Globals.cpp:115
#5 0x000000000042cce7 in CliGlobals::createCliGlobals (espProcess=0) at ../cli/Globals.cpp:549
#6 0x000000000042737e in runServer (argc=11, argv=0x7fffffffc438) at ../bin/ex_sscp_main.cpp:160
#7 0x0000000000427312 in main (argc=11, argv=0x7fffffffc438) at ../bin/ex_sscp_main.cpp:138
To debug this yourself:
ssh slave-cm53
cd /home/jenkins/workspace/pyodbc_test-cm5.3/traf_run; gdb /home/jenkins/workspace/pyodbc_test-cm5.3/traf_run/export/bin64d/mxsscp ./sql/scripts/core.slave-cm53.2091789.mxsscp
Running gdb manually, we saw:
(gdb) bt
#0 0x00000031eb832625 in raise () from /lib64/libc.so.6
#1 0x00000031eb833e05 in abort () from /lib64/libc.so.6
#2 0x00007ffff3f06f4c in assert_botch_abend (f=0x47835c "../cli/Globals.cpp", l=171,
m=0x7fffffffbfd0 "Could not initialize CLI globals.ComRtGetProgramInfo returned an error :201.", c=0x0) at ../export/NAAbort.cpp:282
#3 0x000000000042c3e9 in CliGlobals::init (this=0xc20cb0, espProcess=0, statsGlobals=0x0) at ../cli/Globals.cpp:171
#4 0x000000000042c0f9 in CliGlobals::CliGlobals (this=0xc20cb0, espProcess=0) at ../cli/Globals.cpp:115
#5 0x000000000042cce7 in CliGlobals::createCliGlobals (espProcess=0) at ../cli/Globals.cpp:549
#6 0x000000000042737e in runServer (argc=11, argv=0x7fffffffc468) at ../bin/ex_sscp_main.cpp:160
#7 0x0000000000427312 in main (argc=11, argv=0x7fffffffc468) at ../bin/ex_sscp_main.cpp:138
(gdb) f 3
#3 0x000000000042c3e9 in CliGlobals::init (this=0xc20cb0, espProcess=0, statsGlobals=0x0) at ../cli/Globals.cpp:171
171 ex_assert(0,errStr);//LCOV_EXCL_LINE
(gdb) p *this
$1 = {<NAAssertGlobals> = {globalsAreInitialized_ = 0, longJmpTgtPtr_ = 0x31ebb90598, longJmpTgt_ = {{__jmpbuf = {12717216, 12717216, 3544678368981039152,
3186637791869612346, 5989836223396333856, 3193126160962571345, 8452729170469801504, 3184080371950379629}, __mask_was_saved = 1431323424, __saved_mask = {__val = {
3616988885525549088, 5773663337932928560, 5629626473672568690, 3484136982561779041, 3182967725840821040, 8098443493244813600, 8315161617321959456,
8458160461809524851, 2865758500053870, 4294967295, 7307705458036271151, 8031940633258716014, 3415245162579979122, 8315180240047596650, 3401112950598217076,
7959393670512341620}}}}, numCliCalls_ = 0, logEmsEvents_ = 1, qfoProcessing_ = 1920216688}, currRootTcb_ = 0x0, sharedArkcmp_ = 0x0,
arkcmpInitFailed_ = arkcmpIS_OK_, sharedCtrl_ = 0x0, contextList_ = 0x0, defaultContext_ = 0x0, executorMemory_ = {<NAMemory> = {<NABasicObject> = {
_vptr.NABasicObject = 0x7ffff592b1f0, h_ = 0x0}, name_ = "Global Executor Memo", type_ = NAMemory::EXECUTOR_MEMORY, initialSize_ = 524288,
maximumSize_ = 18446744073709551615, incrementSize_ = 524288, parent_ = 0x0, firstBlk_ = 0x7fffe97f3000, allocSize_ = 1072, upperLimit_ = 0, highWaterMark_ = 1072,
intervalWaterMark_ = 1072, allocCnt_ = 2, totalSize_ = 524288, blockCnt_ = 1, thBlockCnt_ = 10, segGlobals_ = 0xc21338, memoryList_ = 0x0, lastListEntry_ = 0x0,
nextEntry_ = 0x0, debugLevel_ = 0, heapJumpBuf_ = 0x0, exhaustedMem_ = 0, errorsMask_ = 0, heapID_ = {heapNum = 10}, crowdedTotalSize_ = 0, allocationDelta_ = 524288,
procStatusFile_ = 0x0, executorVmReserveSize_ = 0, mmapErrno_ = 0, munmapErrno_ = 0, lastVmSize_ = 0, maxVmSize_ = 0, derivedClass_ = NAMemory::NAHEAP_CLASS,
nullNAStringRep_ = {560, 0, 176, 0, 135, 12717512, 0, 0, 0, 0, 0, 758382592, 840972336}, sharedMemory_ = 0, fillerForVtblPtr_ = 741945658}, smallmap_ = 0,
treemap_ = 0, dvsize_ = 0, topsize_ = 523152, least_addr_ = 0x7fffe97f3000 "", dv_ = 0x0, top_ = 0x7fffe97f3450, smallbins_ = {0x50202c3736303630, 0x4e20737365636f72,
0xc20f40, 0xc20f40, 0xc20f50, 0xc20f50, 0xc20f60, 0xc20f60, 0xc20f70, 0xc20f70, 0xc20f80, 0xc20f80, 0xc20f90, 0xc20f90, 0xc20fa0, 0xc20fa0, 0xc20fb0, 0xc20fb0,
0xc20fc0, 0xc20fc0, 0xc20fd0, 0xc20fd0, 0xc20fe0, 0xc20fe0, 0xc20ff0, 0xc20ff0, 0xc21000, 0xc21000, 0xc21010, 0xc21010, 0xc21020, 0xc21020, 0xc21030, 0xc21030,
0xc21040, 0xc21040, 0xc21050, 0xc21050, 0xc21060, 0xc21060, 0xc21070, 0xc21070, 0xc21080, 0xc21080, 0xc21090, 0xc21090, 0xc210a0, 0xc210a0, 0xc210b0, 0xc210b0,
0xc210c0, 0xc210c0, 0xc210d0, 0xc210d0, 0xc210e0, 0xc210e0, 0xc210f0, 0xc210f0, 0xc21100, 0xc21100, 0xc21110, 0xc21110, 0xc21120, 0xc21120, 0xc21130, 0xc21130},
treebins_ = {0x0 <repeats 32 times>}, dynamicAllocs = {count_ = 0, sum_ = 0, sum2_ = 0, histBuckets_ = {0 <repeats 18 times>}}, errCallback_ = 0, threadSafe_ = true,
mutex_ = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 1, __spins = 0, __list = {__prev = 0x0, __next = 0x0}},
__size = '\000' <repeats 16 times>, "\001", '\000' <repeats 22 times>, __align = 0}, la_ = 0x0}, segGlobals_ = {firstSegId_ = 0, firstSegStart_ = 0x0,
firstSegOffset_ = 0, firstSegLen_ = 0, firstSegMaxLen_ = 0, addedSegCount_ = 0, maxSecSegCount_ = 28, addedSegId_ = {0 <repeats 28 times>}, startAddresses_ = {
0x0 <repeats 28 times>}, lengths_ = {0 <repeats 28 times>}, lowWaterMark_ = 0x0, highWaterMark_ = 0x0}, ipcHeap_ = 0x0, memMonitor_ = 0x0, noWaitSQLHeap_ = 0x0,
quasiFileManager_ = 0x0, arlibHeap_ = 0x0, defaultVolSeed_ = 0, listOfVolNames_ = 0x0, listOfAuditedVols_ = 0x0, listOfVolNamesCacheTime_ = -1, sysVolNameInitialized_ = 0,
sysVolName_ = '\000' <repeats 17 times>, measProcEnabled_ = 0, measStmtEnabled_ = 0, measSubsysRunning_ = 0, measProcCntrs_ = 0x0, envvars_ = 0x0, envvarsContext_ = 0,
nextUniqueContextHandle = 0, eventConsumed_ = 0, processIsStopping_ = 0, totalCliCalls_ = 0, globalSbbCount_ = 0, savedSqlTerminateAction_ = 0, breakEnabled_ = 0,
SPBreakReceived_ = 0, isESPProcess_ = 0, programDir_ = 0x7fffe97f3050 "", processType_ = 2, logReclaimEventDone_ = 0, savedCompilerVersion_ = 2600,
myNodeName_ = "NSK\000\000\000\000", myCpu_ = 0, myVerifier_ = 23, myPin_ = 2206067, myNodeNumber_ = 0, myNumSegs_ = 0, myNumCpus_ = 0, myPriority_ = 0,
priorityChanged_ = 0, myStartTime_ = 0, lastUniqueNumber_ = 0, sessionUniqueNumber_ = 0, emsEventExperienceLevel_ = 0, nodeName_ = "\000\000\000\000\000\000\000\000",
qualifyingVolsPerNode_ = {<NACollection<char*>> = {<NABasicObject> = {_vptr.NABasicObject = 0x479390, h_ = 0x0}, maxLength_ = 0, usedLength_ = 0, entries_ = 0, arr_ = 0x0,
usages_ = 0x0, heap_ = 0x0}, first_ = 111111111, last_ = 111111111, userIndexCache_ = 111111111, arrayIndexCache_ = 111111111},
cpuNumbers_ = {<NACollection<int>> = {<NABasicObject> = {_vptr.NABasicObject = 0x479310, h_ = 0x0}, maxLength_ = 0, usedLength_ = 0, entries_ = 0, arr_ = 0x0,
usages_ = 0x0, heap_ = 0x0}, first_ = 111111111, last_ = 111111111, userIndexCache_ = 111111111, arrayIndexCache_ = 111111111},
capacities_ = {<NACollection<int>> = {<NABasicObject> = {_vptr.NABasicObject = 0x479310, h_ = 0x0}, maxLength_ = 0, usedLength_ = 0, entries_ = 0, arr_ = 0x0,
usages_ = 0x0, heap_ = 0x0}, first_ = 111111111, last_ = 111111111, userIndexCache_ = 111111111, arrayIndexCache_ = 111111111},
freespaces_ = {<NACollection<int>> = {<NABasicObject> = {_vptr.NABasicObject = 0x479310, h_ = 0x0}, maxLength_ = 0, usedLength_ = 0, entries_ = 0, arr_ = 0x0,
usages_ = 0x0, heap_ = 0x0}, first_ = 111111111, last_ = 111111111, userIndexCache_ = 111111111, arrayIndexCache_ = 111111111},
largestFragments_ = {<NACollection<int>> = {<NABasicObject> = {_vptr.NABasicObject = 0x479310, h_ = 0x0}, maxLength_ = 0, usedLength_ = 0, entries_ = 0, arr_ = 0x0,
usages_ = 0x0, heap_ = 0x0}, first_ = 111111111, last_ = 111111111, userIndexCache_ = 111111111, arrayIndexCache_ = 111111111}, statsGlobals_ = 0x0, statsHeap_ = 0x0,
semId_ = 0, inConstructor_ = 1, savedPriority_ = 148, shmId_ = 0, isUncProcess_ = 0, myProcessNameString_ = "$Z00kFVH:23", '\000' <repeats 28 times>,
parentProcessNameString_ = '\000' <repeats 39 times>, myProgName_ = '\000' <repeats 63 times>, tidList_ = 0x0, cliSemaphore_ = 0x0, processStats_ = 0x0}
(gdb) p myVerifier_
$2 = 23
(gdb) a: thread apply all bt
Ambiguous command "a: thread apply all bt": .
(gdb) thread apply all bt
Thread 4 (Thread 0x7fffecb97700 (LWP 2206069)):
#0 0x00000031ebc0b5bc in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007ffff2fef6a6 in SB_Thread::CV::wait (this=0xc1f408) at /home/jenkins/workspace/jdbc_test-cm5.3/trafodion/core/sqf/export/include/seabed/int/thread.inl:552
#2 0x00007ffff2fef782 in SB_Thread::CV::wait (this=0xc1f408, pv_lock=false)
at /home/jenkins/workspace/jdbc_test-cm5.3/trafodion/core/sqf/export/include/seabed/int/thread.inl:591
#3 0x00007ffff3c78c77 in SB_Sig_Queue::remove (this=0xc1f3b0) at queue.inl:473
#4 0x00007ffff3c869a7 in SB_Trans::Sock_Stream_Helper_Thread::run (this=0xc1ee60) at sockstream.cpp:2179
#5 0x00007ffff3c8626a in sock_helper_thread_fun (pp_arg=0xc1ee60) at sockstream.cpp:2029
#6 0x00007ffff2fecbef in SB_Thread::Thread::disp (this=0xc1ee60, pp_arg=0xc1ee60) at thread.cpp:211
#7 0x00007ffff2fed047 in thread_fun (pp_arg=0xc1ee60) at thread.cpp:307
#8 0x00007ffff2ff02e0 in sb_thread_sthr_disp (pp_arg=0xc1f570) at threadl.cpp:253
#9 0x00000031ebc079d1 in start_thread () from /lib64/libpthread.so.0
#10 0x00000031eb8e88fd in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7fffed598700 (LWP 2206068)):
#0 0x00000031eb8e94bd in accept () from /lib64/libc.so.6
#1 0x00007ffff3c7f9c5 in SB_Trans::Sock_Listener::accept (this=0xc17c30) at sock.cpp:618
#2 0x00007ffff3c86516 in SB_Trans::Sock_Stream_Accept_Thread::run (this=0xc1d010) at sockstream.cpp:2085
#3 0x00007ffff3c86243 in sock_stream_accept_thread_fun (pp_arg=0xc1d010) at sockstream.cpp:2019
#4 0x00007ffff2fecbef in SB_Thread::Thread::disp (this=0xc1d010, pp_arg=0xc1d010) at thread.cpp:211
#5 0x00007ffff2fed047 in thread_fun (pp_arg=0xc1d010) at thread.cpp:307
#6 0x00007ffff2ff02e0 in sb_thread_sthr_disp (pp_arg=0xc1eb60) at threadl.cpp:253
#7 0x00000031ebc079d1 in start_thread () from /lib64/libpthread.so.0
#8 0x00000031eb8e88fd in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7fffea273700 (LWP 2206070)):
#0 0x00000031eb83371d in sigtimedwait () from /lib64/libc.so.6
#1 0x00007ffff3c445f4 in local_monitor_reader (pp_arg=0x21a483) at ../../../monitor/linux/clio.cxx:130
#2 0x00000031ebc079d1 in start_thread () from /lib64/libpthread.so.0
#3 0x00000031eb8e88fd in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7fffed8073e0 (LWP 2206067)):
#0 0x00000031eb832625 in raise () from /lib64/libc.so.6
#1 0x00000031eb833e05 in abort () from /lib64/libc.so.6
#2 0x00007ffff3f06f4c in assert_botch_abend (f=0x47835c "../cli/Globals.cpp", l=171,
m=0x7fffffffbfd0 "Could not initialize CLI globals.ComRtGetProgramInfo returned an error :201.", c=0x0) at ../export/NAAbort.cpp:282
#3 0x000000000042c3e9 in CliGlobals::init (this=0xc20cb0, espProcess=0, statsGlobals=0x0) at ../cli/Globals.cpp:171
#4 0x000000000042c0f9 in CliGlobals::CliGlobals (this=0xc20cb0, espProcess=0) at ../cli/Globals.cpp:115
#5 0x000000000042cce7 in CliGlobals::createCliGlobals (espProcess=0) at ../cli/Globals.cpp:549
#6 0x000000000042737e in runServer (argc=11, argv=0x7fffffffc468) at ../bin/ex_sscp_main.cpp:160
#7 0x0000000000427312 in main (argc=11, argv=0x7fffffffc468) at ../bin/ex_sscp_main.cpp:138
(gdb)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)