You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by "Adep, Karankumar (ETW - FLEX)" <Ka...@nike.com> on 2018/08/13 20:22:34 UTC
Region Server Crashes with below ERROR
Hi,
Region Server Crashes with below ERROR, Looks like some issues with GC configuration ?
2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices: Scanner 2847160913172185436 lease expired on region pi,ea000000,1519782669886.393add40963aadf9d6a3ceeceaee1106.
2018-08-09 14:54:26,104 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 6173ms
GC pool 'ParNew' had collection(s): count=2 time=114ms
GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=6262ms
2018-08-09 14:54:26,100 INFO org.apache.hadoop.hbase.ScheduledChore: Chore: CompactionChecker missed its start time
2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices: Scanner 2847160913172185429 lease expired on region pi,9c000000,1519782669886.962035d50bed0ead188dc55484587c5c.
2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices: Scanner 2847160913172185442 lease expired on region pt,b8000000,1519942253050.7cb5a194271207df2456220c1bfacda8.
Thank You,
Karan Adep | Platform Operations Team | Cell: 917-873-6163
Re: Region Server Crashes with below ERROR
Posted by Ankit Singhal <an...@gmail.com>.
bq. Region Server Crashes with below ERROR, Looks like some issues with GC
configuration ?
bq. 2018-08-09 14:54:26,104 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor:
Detected pause in JVM or host machine (eg GC): pause of approximately 6173ms
bq. GC pool 'ParNew' had collection(s): count=2 time=114ms
bq. GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=6262ms
A 6-second pause cannot result in a timeout with
zookeeper(zookeeper.session.timeout) and may not be the reason for
regionserver failure. Check for large pauses or errors(with FATAL/ERROR) in
logs as Josh said.
On Mon, Aug 13, 2018 at 2:00 PM, Josh Elser <el...@apache.org> wrote:
> Nothing in here indicates why the RegionServers actually failed.
>
> If the RegionServer crashed, there is very likely a log message at FATAL.
> You want to find that to understand what actually caused it.
>
>
> On 8/13/18 4:22 PM, Adep, Karankumar (ETW - FLEX) wrote:
>
>> Hi,
>>
>> Region Server Crashes with below ERROR, Looks like some issues with GC
>> configuration ?
>>
>> 2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices:
>> Scanner 2847160913172185436 lease expired on region
>> pi,ea000000,1519782669886.393add40963aadf9d6a3ceeceaee1106.
>> 2018-08-09 14:54:26,104 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor:
>> Detected pause in JVM or host machine (eg GC): pause of approximately 6173ms
>> GC pool 'ParNew' had collection(s): count=2 time=114ms
>> GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=6262ms
>> 2018-08-09 14:54:26,100 INFO org.apache.hadoop.hbase.ScheduledChore:
>> Chore: CompactionChecker missed its start time
>> 2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices:
>> Scanner 2847160913172185429 lease expired on region
>> pi,9c000000,1519782669886.962035d50bed0ead188dc55484587c5c.
>> 2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices:
>> Scanner 2847160913172185442 lease expired on region
>> pt,b8000000,1519942253050.7cb5a194271207df2456220c1bfacda8.
>>
>> Thank You,
>> Karan Adep | Platform Operations Team | Cell: 917-873-6163
>>
>>
>
Re: Region Server Crashes with below ERROR
Posted by Josh Elser <el...@apache.org>.
Nothing in here indicates why the RegionServers actually failed.
If the RegionServer crashed, there is very likely a log message at
FATAL. You want to find that to understand what actually caused it.
On 8/13/18 4:22 PM, Adep, Karankumar (ETW - FLEX) wrote:
> Hi,
>
> Region Server Crashes with below ERROR, Looks like some issues with GC configuration ?
>
> 2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices: Scanner 2847160913172185436 lease expired on region pi,ea000000,1519782669886.393add40963aadf9d6a3ceeceaee1106.
> 2018-08-09 14:54:26,104 INFO org.apache.hadoop.hbase.util.JvmPauseMonitor: Detected pause in JVM or host machine (eg GC): pause of approximately 6173ms
> GC pool 'ParNew' had collection(s): count=2 time=114ms
> GC pool 'ConcurrentMarkSweep' had collection(s): count=1 time=6262ms
> 2018-08-09 14:54:26,100 INFO org.apache.hadoop.hbase.ScheduledChore: Chore: CompactionChecker missed its start time
> 2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices: Scanner 2847160913172185429 lease expired on region pi,9c000000,1519782669886.962035d50bed0ead188dc55484587c5c.
> 2018-08-09 14:54:26,106 INFO org.apache.hadoop.hbase.regionserver.RSRpcServices: Scanner 2847160913172185442 lease expired on region pt,b8000000,1519942253050.7cb5a194271207df2456220c1bfacda8.
>
> Thank You,
> Karan Adep | Platform Operations Team | Cell: 917-873-6163
>
>