You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Chidambaranathan (JIRA)" <ji...@apache.org> on 2016/05/27 12:26:12 UTC

[jira] [Created] (HBASE-15901) Region server process is often getting crashed

Chidambaranathan created HBASE-15901:
----------------------------------------

             Summary: Region server process is often getting crashed
                 Key: HBASE-15901
                 URL: https://issues.apache.org/jira/browse/HBASE-15901
             Project: HBase
          Issue Type: Bug
          Components: regionserver
    Affects Versions: 0.94.6
         Environment: System Setup:
============

1 Hmaster + 1 HsecMaster + 4 Hslaves

Software Versions:
=================

RHEL_VERSION	6.4
HBASE_VERSION	0.94.6-cdh4.3.1
HADOOP_VERSION	2.0.0-cdh4.3.1
ZOOKEEPER_VERSION	3.4.5-cdh4.3.1
JAVA_VERSION	1.6u45

System Configuration:
====================

CPU: 2 Physical CPU x 6 Cores
Memory: 64 GB
            Reporter: Chidambaranathan
            Priority: Blocker


Issue Description:
=================

Region server process is often getting crashed.When analyzed further, we came to know that 'Region server process JVM' was consuming more physical memory due to which Linux 'OOM killer' killed the process. This randomly occurs in different HSlave machines in the cluster.

The Heap size for the Region server is 38 GB(aware this is on higher side :))

I see lot of 64MB pages allocated for "anon" from PMAP.

I would like to understand the following,

1) What would be the reason for these many 64MB physical memory pages acquired by region server process?
2) How to control this behaviour, in order to reduce the memory foot print outside heap?


Log Details:
===========

Here is the memory usage in HBase cluster reported by "free, top, JMAP & PMAP"

HSLave1
-------

Free:

total       used       free     shared    buffers     cached
Mem:         64386      60966       3420          0        291       5870
-/+ buffers/cache:      54804       9581
Swap:        28963        768      28195


Top:

top - 19:12:24 up 329 days,  2:11,  2 users,  load average: 0.66, 0.53, 0.58
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
Cpu(s):  4.6%us,  3.6%sy,  0.0%ni, 90.6%id,  0.0%wa,  0.0%hi,  1.2%si,  0.0%st
Mem:  65932072k total, 63597656k used,  2334416k free,   300284k buffers
Swap: 29659128k total,   787304k used, 28871824k free,  7174108k cached

 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                   
62381 hbase     20   0 50.5g  49g 7352 S 35.6 79.4  19273:50 java  


JMAP:

Attaching to process ID 62381, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.45-b01

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 40802189312 (38912.0MB)
   NewSize          = 21757952 (20.75MB)
   MaxNewSize       = 392560640 (374.375MB)
   OldSize          = 65404928 (62.375MB)
   NewRatio         = 7
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 85983232 (82.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 353304576 (336.9375MB)
   used     = 328423048 (313.20862579345703MB)
   free     = 24881528 (23.72887420654297MB)
   92.95748493220762% used
Eden Space:
   capacity = 314048512 (299.5MB)
   used     = 314048512 (299.5MB)
   free     = 0 (0.0MB)
   100.0% used
From Space:
   capacity = 39256064 (37.4375MB)
   used     = 14374536 (13.708625793457031MB)
   free     = 24881528 (23.72887420654297MB)
   36.617364389868534% used
To Space:
   capacity = 39256064 (37.4375MB)
   used     = 1519584 (1.449188232421875MB)
   free     = 37736480 (35.988311767578125MB)
   3.8709535423622703% used
concurrent mark-sweep generation:
   capacity = 40409628672 (38537.625MB)
   used     = 6564587064 (6260.478080749512MB)
   free     = 33845041608 (32277.14691925049MB)
   16.24510612874953% used
Perm Generation:
   capacity = 64958464 (61.94921875MB)
   used     = 38873344 (37.072509765625MB)
   free     = 26085120 (24.876708984375MB)
   59.84338545936062% used


PMAP:

In PMAP OUTPUT we could see a lot of anonymos [anon] memory each of 64 MB , allocated to the region pid,

62381:   /usr/java/latest/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M -Xloggc:/opt/hbase/logs/hbase-hbas
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000040000000      36       4       0 r-x--  java
0000000040108000       8       8       8 rwx--  java
000000004010a000   75316   75188   75188 rwx--    [ anon ]
00007f8868000000   65476   65476   65476 rwx--    [ anon ]
00007f886bff1000      60       0       0 -----    [ anon ]
00007f886c000000   65536   22192   22192 rwx--    [ anon ]
00007f8870000000   65476   65476   65476 rwx--    [ anon ]
00007f8873ff1000      60       0       0 -----    [ anon ]
00007f8878000000   65484   61048   61048 rwx--    [ anon ]
00007f887bff3000      52       0       0 -----    [ anon ]
00007f887c000000   65532   65532   65532 rwx--    [ anon ]
00007f887ffff000       4       0       0 -----    [ anon ]
00007f8880000000   65512   65512   65512 rwx--    [ anon ]
00007f8883ffa000      24       0       0 -----    [ anon ]
00007f8888000000   65488   65488   65488 rwx--    [ anon ]
00007f888bff4000      48       0       0 -----    [ anon ]
00007f888c000000   65512   65512   65512 rwx--    [ anon ]
00007f888fffa000      24       0       0 -----    [ anon ]
.
.
.
00007f8b7c52a000    1016     100     100 rwx--    [ anon ]
00007f8b7f658000      12       0       0 -----    [ anon ]
00007f8b7f65b000    1016     116     116 rwx--    [ anon ]
00007f8b7fa5c000      12       0       0 -----    [ anon ]
00007f8b7fa5f000    1016      96      96 rwx--    [ anon ]
00007f8b7fb5d000      12       0       0 -----    [ anon ]
00007f8b7fb60000    1016      88      88 rwx--    [ anon ]
00007f8b7fd5f000      12       0       0 -----    [ anon ]
00007f8b7fd62000    1016      96      96 rwx--    [ anon ]
00007f8b8086a000      12       0       0 -----    [ anon ]
00007f8b8086d000    1016      96      96 rwx--    [ anon ]
00007f8b80d6f000      12       0       0 -----    [ anon ]
.
.
.

00007f95539fb000    1024      12      12 rwx--    [ anon ]
00007f9553afb000       4       0       0 -----    [ anon ]
00007f9553afc000    1024      12      12 rwx--    [ anon ]
00007f9553bfc000       4       0       0 -----    [ anon ]
00007f9553bfd000    1024      12      12 rwx--    [ anon ]
00007f9553cfd000       4       0       0 -----    [ anon ]
00007f9553cfe000    1024      12      12 rwx--    [ anon ]
00007f9553dfe000       4       0       0 -----    [ anon ]
00007f9553dff000    1024      12      12 rwx--    [ anon ]
00007f9553eff000       4       0       0 -----    [ anon ]
.
.
.
.
----------------  ------  ------  ------
total kB        52905432 52327812 52320456


HSLave2
-------

Free:

             total       used       free     shared    buffers     cached
Mem:         64386      64059        327          0        448      11860
-/+ buffers/cache:      51750      12635
Swap:        28963        558      28405


Top:

   top - 19:22:43 up 329 days,  1:54,  2 users,  load average: 2.86, 2.35, 2.35
Tasks: 781 total,   1 running, 780 sleeping,   0 stopped,   0 zombie
Cpu(s): 11.3%us,  0.8%sy,  0.0%ni, 87.8%id,  0.0%wa,  0.0%hi,  0.1%si,  0.0%st
Mem:  65932072k total, 64978056k used,   954016k free,   462020k buffers
Swap: 29659128k total,   571868k used, 29087260k free, 11518808k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                   
20890 hbase     20   0 61.9g  45g  15m S 273.7 72.8  49920:48 java 


JMAP:

Attaching to process ID 20890, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.45-b01

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 40802189312 (38912.0MB)
   NewSize          = 21757952 (20.75MB)
   MaxNewSize       = 392560640 (374.375MB)
   OldSize          = 65404928 (62.375MB)
   NewRatio         = 7
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 85983232 (82.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 353304576 (336.9375MB)
   used     = 143119808 (136.48968505859375MB)
   free     = 210184768 (200.44781494140625MB)
   40.50890300384901% used
Eden Space:
   capacity = 314048512 (299.5MB)
   used     = 140928488 (134.3998794555664MB)
   free     = 173120024 (165.1001205444336MB)
   44.874751070305976% used
From Space:
   capacity = 39256064 (37.4375MB)
   used     = 2191320 (2.0898056030273438MB)
   free     = 37064744 (35.347694396972656MB)
   5.582118472193239% used
To Space:
   capacity = 39256064 (37.4375MB)
   used     = 0 (0.0MB)
   free     = 39256064 (37.4375MB)
   0.0% used
concurrent mark-sweep generation:
   capacity = 25221365760 (24052.96875MB)
   used     = 16879822464 (16097.853149414062MB)
   free     = 8341543296 (7955.1156005859375MB)
   66.926678850876% used
Perm Generation:
   capacity = 65531904 (62.49609375MB)
   used     = 39318808 (37.497337341308594MB)
   free     = 26213096 (24.998756408691406MB)
   59.99948971420089% used


PMAP:

In PMAP OUTPUT we could see a lot of anonymos [anon] memory each of 64 MB , allocated to the region pid,

20890:   /usr/java/latest/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M -Xloggc:/opt/hbase/logs/hbase-hbas
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000040000000      36      36       0 r-x--  java
0000000040108000       8       8       8 rwx--  java
000000004010a000 23320596 23320596 23320596 rwx--    [ anon ]
00007fece45c9000  196608   68280   68280 rwx--    [ anon ]
00007fecf0ed2000      12       0       0 -----    [ anon ]
00007fecf0ed5000    1016      88      88 rwx--    [ anon ]
00007fecf0fd3000      12       0       0 -----    [ anon ]
00007fecf0fd6000    1016      24      24 rwx--    [ anon ]
00007fecf1ee2000      12       0       0 -----    [ anon ]
00007fecf1ee5000    1016      24      24 rwx--    [ anon ]
00007fecf20e4000      12       0       0 -----    [ anon ]
00007fecf20e7000    1016      24      24 rwx--    [ anon ]
00007fecf23e7000      12       0       0 -----    [ anon ]
00007fecf23ea000    1016      24      24 rwx--    [ anon ]
00007fecf27eb000      12       0       0 -----    [ anon ]
00007fecf27ee000    1016      92      92 rwx--    [ anon ]
00007fecf28ec000      12       0       0 -----    [ anon ]
.
.
.

00007ff6c8bfd000    1024      12      12 rwx--    [ anon ]
00007ff6c8cfd000       4       0       0 -----    [ anon ]
00007ff6c8cfe000    1024      12      12 rwx--    [ anon ]
00007ff6c8dfe000       4       0       0 -----    [ anon ]
00007ff6c8dff000    1024      12      12 rwx--    [ anon ]
00007ff6c8eff000       4       0       0 -----    [ anon ]
00007ff6c8f00000   10176    9148    9148 rwx--    [ anon ]
00007ff6c98f0000  105516   55732   55732 rwx--    [ anon ]
00007ff6cfffb000      20       0       0 -----    [ anon ]
00007ff6d009d000       4       0       0 -----    [ anon ]
00007ff6d009e000    1024      12      12 rwx--    [ anon ]
00007ff6d019e000       4       0       0 -----    [ anon ]
00007ff6d019f000    1024      12      12 rwx--    [ anon ]
00007ff6d029f000       4       0       0 -----    [ anon ]
00007ff6d02a0000    1024      12      12 rwx--    [ anon ]
00007ff6d03a0000       4       0       0 -----    [ anon ]
.
.
.
00007fff1c45a000      96      56      56 rwx--    [ stack ]
00007fff1c561000       4       4       0 r-x--    [ anon ]
ffffffffff600000       4       0       0 r-x--    [ anon ]
----------------  ------  ------  ------
total kB        64942000 47998468 47983064


HSLave3
-------

Free:

             total       used       free     shared    buffers     cached
Mem:         64386      63757        629          0        315       2004
-/+ buffers/cache:      61437       2949
Swap:        28963        154      28809


Top:

top - 19:29:12 up 329 days,  1:39,  2 users,  load average: 0.70, 0.90, 1.22
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.4%us,  1.5%sy,  0.0%ni, 90.8%id,  0.1%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  65932072k total, 65283080k used,   648992k free,   322804k buffers
Swap: 29659128k total,   158364k used, 29500764k free,  1975488k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                   
41644 hbase     20   0 62.6g  52g 8620 S 15.5 83.6  48068:32 java                   


JMAP:

Attaching to process ID 41644, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.45-b01

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 40802189312 (38912.0MB)
   NewSize          = 21757952 (20.75MB)
   MaxNewSize       = 392560640 (374.375MB)
   OldSize          = 65404928 (62.375MB)
   NewRatio         = 7
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 85983232 (82.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 353304576 (336.9375MB)
   used     = 96493664 (92.02352905273438MB)
   free     = 256810912 (244.91397094726562MB)
   27.311750414463923% used
Eden Space:
   capacity = 314048512 (299.5MB)
   used     = 91050000 (86.83204650878906MB)
   free     = 222998512 (212.66795349121094MB)
   28.992336063034745% used
From Space:
   capacity = 39256064 (37.4375MB)
   used     = 5443664 (5.1914825439453125MB)
   free     = 33812400 (32.24601745605469MB)
   13.86706522589733% used
To Space:
   capacity = 39256064 (37.4375MB)
   used     = 0 (0.0MB)
   free     = 39256064 (37.4375MB)
   0.0% used
concurrent mark-sweep generation:
   capacity = 30304894976 (28901.0MB)
   used     = 17871116576 (17043.224884033203MB)
   free     = 12433778400 (11857.775115966797MB)
   58.971055963576354% used
Perm Generation:
   capacity = 67346432 (64.2265625MB)
   used     = 40325792 (38.457672119140625MB)
   free     = 27020640 (25.768890380859375MB)
   59.87814172545919% used


PMAP:

In PMAP OUTPUT we could see a lot of anonymos [anon] memory each of 64 MB , allocated to the region pid,

41644:   /usr/java/latest/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M -Xloggc:/opt/hbase/logs/hbase-hbas
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000040000000      36       8       0 r-x--  java
0000000040108000       8       8       8 rwx--  java
000000004010a000   43532   43404   43404 rwx--    [ anon ]
00007f4f10000000   65532   65492   65492 rwx--    [ anon ]
00007f4f13fff000       4       0       0 -----    [ anon ]
00007f4f34000000   65496   65496   65496 rwx--    [ anon ]
00007f4f37ff6000      40       0       0 -----    [ anon ]
00007f4f3c000000   65536   65536   65536 rwx--    [ anon ]
00007f4f74000000   65480   65480   65480 rwx--    [ anon ]
00007f4f77ff2000      56       0       0 -----    [ anon ]
00007f5048000000   65524   17568   17568 rwx--    [ anon ]
00007f504bffd000      12       0       0 -----    [ anon ]
00007f5050000000   65520   65520   65520 rwx--    [ anon ]
00007f5053ffc000      16       0       0 -----    [ anon ]
00007f5054000000   65536   65536   65536 rwx--    [ anon ]
00007f5058000000   65496   65496   65496 rwx--    [ anon ]
00007f505bff6000      40       0       0 -----    [ anon ]
00007f505c000000   65532   65484   65484 rwx--    [ anon ]
00007f505ffff000       4       0       0 -----    [ anon ]
00007f5060000000   65508   65508   65508 rwx--    [ anon ]
00007f5063ff9000      28       0       0 -----    [ anon ]
.
.
.
00007f56add2a000      12       0       0 -----    [ anon ]
00007f56add2d000    1016     108     108 rwx--    [ anon ]
00007f56ade2b000      12       0       0 -----    [ anon ]
00007f56ade2e000    1016     104     104 rwx--    [ anon ]
00007f56adf2c000      12       0       0 -----    [ anon ]
00007f56adf2f000    1016      96      96 rwx--    [ anon ]
00007f56ae02d000      12       0       0 -----    [ anon ]
00007f56ae030000    1016     112     112 rwx--    [ anon ]
00007f56ae12e000      12       0       0 -----    [ anon ]
00007f56ae131000    1016     104     104 rwx--    [ anon ]
00007f56ae22f000      12       0       0 -----    [ anon ]
00007f56ae232000    1016      92      92 rwx--    [ anon ]
.
.
.
.
00007f56b24ca000      12       0       0 -----    [ anon ]
00007f56b24cd000    1016       8       8 rwx--    [ anon ]
00007f56b25cb000      12       0       0 -----    [ anon ]
00007f56b25ce000    1016      56      56 rwx--    [ anon ]
00007f56b26cc000      12       0       0 -----    [ anon ]
00007f56b26cf000    1016      48      48 rwx--    [ anon ]
00007f56b27cd000      12       0       0 -----    [ anon ]
00007f56b27d0000    1016       8       8 rwx--    [ anon ]
00007f56b28ce000      12       0       0 -----    [ anon ]
00007f56b28d1000    1016       8       8 rwx--    [ anon ]
00007f56b29cf000   96836       8       0 r-x--  locale-archive
00007f56b8860000      12       0       0 -----    [ anon ]
.
.
.
----------------  ------  ------  ------
total kB        65653168 55086808 55078192


HSLave4
-------

Free:

             total       used       free     shared    buffers     cached
Mem:         64386      64163        223          0         71         24
-/+ buffers/cache:      64067        319
Swap:        28963       3807      25156


Top:

top - 19:34:41 up 329 days,  1:25,  1 user,  load average: 4.09, 4.45, 4.25
Tasks:   1 total,   0 running,   1 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.0%us,  1.6%sy,  0.0%ni, 91.1%id,  0.1%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  65932072k total, 65707276k used,   224796k free,     8180k buffers
Swap: 29659128k total,  3923944k used, 25735184k free,   158556k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                   
59864 hbase     20   0 60.5g  56g 2480 S 134.9 89.8  49529:39 java


JMAP:

Attaching to process ID 59864, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 20.45-b01

using parallel threads in the new generation.
using thread-local object allocation.
Concurrent Mark-Sweep GC

Heap Configuration:
   MinHeapFreeRatio = 40
   MaxHeapFreeRatio = 70
   MaxHeapSize      = 40802189312 (38912.0MB)
   NewSize          = 21757952 (20.75MB)
   MaxNewSize       = 392560640 (374.375MB)
   OldSize          = 65404928 (62.375MB)
   NewRatio         = 7
   SurvivorRatio    = 8
   PermSize         = 21757952 (20.75MB)
   MaxPermSize      = 85983232 (82.0MB)

Heap Usage:
New Generation (Eden + 1 Survivor Space):
   capacity = 353304576 (336.9375MB)
   used     = 92135560 (87.86731719970703MB)
   free     = 261169016 (249.07018280029297MB)
   26.078224359030095% used
Eden Space:
   capacity = 314048512 (299.5MB)
   used     = 79957848 (76.25374603271484MB)
   free     = 234090664 (223.24625396728516MB)
   25.460349259671066% used
From Space:
   capacity = 39256064 (37.4375MB)
   used     = 12177712 (11.613571166992188MB)
   free     = 27078352 (25.823928833007812MB)
   31.021225153902336% used
To Space:
   capacity = 39256064 (37.4375MB)
   used     = 0 (0.0MB)
   free     = 39256064 (37.4375MB)
   0.0% used
concurrent mark-sweep generation:
   capacity = 40409628672 (38537.625MB)
   used     = 10409575160 (9927.34447479248MB)
   free     = 30000053512 (28610.28052520752MB)
   25.760135646118517% used
Perm Generation:
   capacity = 68177920 (65.01953125MB)
   used     = 40687664 (38.80278015136719MB)
   free     = 27490256 (26.216751098632812MB)
   59.67865256082908% used


PMAP:

In PMAP OUTPUT we could see a lot of anonymos [anon] memory each of 64 MB , allocated to the region pid,

59864:   /usr/java/latest/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Xmx8000m -XX:+UseConcMarkSweepGC -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false -XX:MaxTenuringThreshold=8 -XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+ScavengeBeforeFullGC -XX:+CMSScavengeBeforeRemark -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M -Xloggc:/opt/hbase/logs/hbase-hbas
Address           Kbytes     RSS   Dirty Mode   Mapping
0000000040000000      36       0       0 r-x--  java
0000000040108000       8       4       0 rwx--  java
000000004010a000 4872320 4831728 4831680 rwx--    [ anon ]
00007fd8ac000000   65512   20664   20664 rwx--    [ anon ]
00007fd8afffa000      24       0       0 -----    [ anon ]
00007fd8b4000000   65532   65532   65532 rwx--    [ anon ]
00007fd8b7fff000       4       0       0 -----    [ anon ]
00007fd8b8000000   65536   65536   65536 rwx--    [ anon ]
00007fd8bc000000   65532   65532   65532 rwx--    [ anon ]
00007fd8bffff000       4       0       0 -----    [ anon ]
00007fd8c0000000   65488   65488   65488 rwx--    [ anon ]
00007fd8c3ff4000      48       0       0 -----    [ anon ]
00007fd8c4000000   65536   65536   65536 rwx--    [ anon ]
00007fd8cc000000   65532   65532   65532 rwx--    [ anon ]
00007fd8cffff000       4       0       0 -----    [ anon ]
00007fd8d4000000   65536   65512   65512 rwx--    [ anon ]
.
.
.
.
00007fdde145e000      12       0       0 -----    [ anon ]
00007fdde1461000    1016      88      88 rwx--    [ anon ]
00007fdde155f000      12       0       0 -----    [ anon ]
00007fdde1562000    1016      92      92 rwx--    [ anon ]
00007fdde1660000      12       0       0 -----    [ anon ]
00007fdde1663000    1016      92      92 rwx--    [ anon ]
00007fdde1761000      12       0       0 -----    [ anon ]
00007fdde1764000    1016      24      24 rwx--    [ anon ]
00007fdde1862000      12       0       0 -----    [ anon ]
00007fdde1865000    1016      24      24 rwx--    [ anon ]
00007fdde1963000      12       0       0 -----    [ anon ]
00007fdde1966000    1016      96      96 rwx--    [ anon ]
00007fdde1a64000      12       0       0 -----    [ anon ]
00007fdde1a67000    1016      92      92 rwx--    [ anon ]
00007fdde1b65000      12       0       0 -----    [ anon ]
00007fdde1b68000    1016      24      24 rwx--    [ anon ]
00007fdde1c66000      12       0       0 -----    [ anon ]
.
.
.
00007fe7b0000000   65528   65416   65416 rwx--    [ anon ]
00007fe7b3ffe000       8       0       0 -----    [ anon ]
00007fe7b408c000   27536   27116   26876 rwx--    [ anon ]
00007fe7b5b70000  102952   41620   39716 rwx--    [ anon ]
00007fe7bbffa000      24       0       0 -----    [ anon ]
00007fe7bc07a000   12336   12336   12336 rwx--    [ anon ]
00007fe7bcc86000       4       0       0 -----    [ anon ]
00007fe7bcc87000    1024      12      12 rwx--    [ anon ]
00007fe7bcd87000       4       0       0 -----    [ anon ]
00007fe7bcd88000    1024      12      12 rwx--    [ anon ]
00007fe7bce88000       4       0       0 -----    [ anon ]
00007fe7bce89000    1024      12      12 rwx--    [ anon ]
00007fe7bcf89000       4       0       0 -----    [ anon ]
.
.
.

00007fe7bfeee000     212       0       0 r-xs-  passwd
00007fe7bff23000       4       4       4 rwx--    [ anon ]
00007fe7bff24000       4       0       0 -----    [ anon ]
00007fe7bff25000      12       8       8 rwx--    [ anon ]
00007fe7bff28000      28       0       0 r-x--  libjli.so
00007fe7bff2f000    1028       0       0 -----  libjli.so
00007fe7c0030000       8       4       0 rwx--  libjli.so
00007fe7c0032000       8       8       0 rwx--    [ anon ]
00007fe7c0034000       4       4       4 r-x--  ld-2.12.so
00007fe7c0035000       4       4       4 rwx--  ld-2.12.so
00007fe7c0036000       4       4       0 rwx--    [ anon ]
00007fff81efc000      96      20      12 rwx--    [ stack ]
00007fff81fff000       4       4       0 r-x--    [ anon ]
ffffffffff600000       4       0       0 r-x--    [ anon ]
----------------  ------  ------  ------
total kB        63470632 59178756 59174032


----------------------------------




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)