You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Duo Zhang (Jira)" <ji...@apache.org> on 2022/11/01 06:37:00 UTC
[jira] [Commented] (HBASE-27452) RegionServer JVM crash when scan
[ https://issues.apache.org/jira/browse/HBASE-27452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17626975#comment-17626975 ]
Duo Zhang commented on HBASE-27452:
-----------------------------------
Let's discuss here.
First, please see the comment from [~Xiaolin Ha]
{quote}
Hi, gaofeng , you can see whether HBASE-26155 and HBASE-26281 can resolve your problem.
{quote}
I think this could be done by upgrading hbase version.
And the crash here seems to be in the GC thread, could you please try to upgrade to newer jdk?
Thanks.
> RegionServer JVM crash when scan
> --------------------------------
>
> Key: HBASE-27452
> URL: https://issues.apache.org/jira/browse/HBASE-27452
> Project: HBase
> Issue Type: Bug
> Components: scan
> Affects Versions: 2.3.5
> Reporter: gaofeng
> Priority: Critical
> Attachments: hs_err_pid117788.log
>
>
> In our cluster, we found region servers may be crashed in several cases
> hs_err_pid117788.log
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> # SIGSEGV (0xb) at pc=0x00007fdc733a9a4e, pid=117788, tid=0x00007fdc0a34b700
> #
> # JRE version: Java(TM) SE Runtime Environment (8.0_171-b11) (build 1.8.0_171-b11)
> # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.171-b11 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # j org.apache.hadoop.hbase.ipc.NettyServerRpcConnection.isConnectionOpen()Z+4
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
> #
> # If you would like to submit a bug report, please visit:
> # http://bugreport.java.com/bugreport/crash.jsp
> #
>
> --------------- T H R E A D ---------------
>
> Current thread (0x00007fd48004f000): JavaThread "RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020" daemon [_thread_in_Java, id=118639, stack(0x00007fdc0a24b000,0x00007fdc0a34c000)]
>
> regionserver日志
> 2022-10-28 03:33:03,611 WARN [RpcServer.default.FPBQ.Fifo.handler=25,queue=0,port=16020] ipc.RpcServer: (responseTooSlow): {"call":"Scan(org.apache.hadoop.h
> base.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1666899153611","responsesize":"19","method":"Scan","param":"scanner_id: 119667009731
> 85298703 number_of_rows: 1000 close_scanner: false next_call_seq: 0 client_handles_partials: true client_handles_heartbeats: t <TRUNCATED>","processingtimems
> ":30000,"client":"10.64.143.22:48550","queuetimems":0,"class":"HRegionServer","scandetails":"table: action_user_track_v2 region: action_user_track_v2,\\xE8\\
> x00\\x00\\x00-\\xAA\\xF9\\xFB\\x00\\x00\\x01\{\\xCF\\xA6\\xDE\\x09,1645585735023.efb249fd8f4c26d4911d50f14fc30ce8."}
>
> 2022-10-28 03:33:03,611 WARN [RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020] hfile.BlockCacheUtil: Caching an already cached block: 58a6823d3958
> 49099bb226e18acb79b2_SeqId_6__1032126997. This is harmless and can happen in rare cases (see HBASE-8547)
> 2022-10-28 03:33:03,611 WARN [RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020] hfile.BlockCacheUtil: Caching an already cached block: eb1c30882099
> 45f6b5006db45d5fe286_SeqId_4__1032126997. This is harmless and can happen in rare cases (see HBASE-8547)
> 2022-10-28 03:33:03,612 WARN [RpcServer.default.FPBQ.Fifo.handler=17,queue=2,port=16020] hfile.BlockCacheUtil: Caching an already cached block: eb1c30882099
> 45f6b5006db45d5fe286_SeqId_4__1032149468. This is harmless and can happen in rare cases (see HBASE-8547)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)