You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Sandeep L <sa...@outlook.com> on 2013/07/11 15:08:10 UTC
Region server going down when deleting a table from HBase
Hi,
We are using hbase-0.94.1 with hadoop-1.0.2.
Recently couple of time we faced a strange issue while deleting a table.
Whenever we are deleting a table from HBase shell using disable and drop command at least one or two region servers are going down suddenly.
I observed following error message in region server log:
INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: Unrecoverable exception while closing region REGION_NAMEjava.lang.RuntimeException: org.apache.hadoop.hbase.DroppedSnapshotException: region: REGION_NAME
Previously also we used to disable and drop tables, but we never seen this kind of issue.
Thanks,Sandeep.
Re: Region server going down when deleting a table from HBase
Posted by ramkrishna vasudevan <ra...@gmail.com>.
Yes it creates the reader and then deletes the reader if the rename fails.
On Thu, Jul 11, 2013 at 8:09 PM, Anoop John <an...@gmail.com> wrote:
> Ya as Ted said, if this warn is not there, need to see what happened
> to that file (just created)
>
> -Anoop-
>
> On Thu, Jul 11, 2013 at 8:07 PM, Anoop John <an...@gmail.com> wrote:
>
> > After the flush opening the HFile reader. You are getting
> > FileNotFoundException!
> > *
> >
> > LOG
> > *.warn("Unable to rename " + path + " to " + dstPath);
> >
> > Do you see above warn in log?
> >
> > In code I can see even if this rename got failed we try to open the file
> > for read. Also as part of the region close, if we do a flush, we might
> > not really do the file open. It will be an unwanted input stream open.
> > Some one correct me if I am wrong pls.
> > -Anoop-
> >
> > On Thu, Jul 11, 2013 at 6:54 PM, Sandeep L <sandeepvreddy@outlook.com
> >wrote:
> >
> >> Anoop,
> >> Following is stack trace
> >> ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
> >> while processing event M_RS_CLOSE_REGIONjava.lang.RuntimeException:
> >> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> >> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21.
> at
> >>
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
> >> at
> >> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >> at
> >>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >> at java.lang.Thread.run(Thread.java:722)Caused by:
> >> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> >> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21.
> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353)
> >> at
> >> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:905)
> >> at
> org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
> >> ... 4 moreCaused by: java.io.FileNotFoundException: File does not
> >> exist:
> >>
> /hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/ba8ae96c466f4559901127ca24378020
> >> at
> >>
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843)
> >> at
> >>
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834)
> >> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
> >> at
> >>
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
> >> at
> >> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
> >> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at
> >>
> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:547)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1245)
> >> at
> >> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602)
> >> at
> >> org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:791)
> >> at
> org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:108)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2256)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1455)
> >> ... 8 more
> >> INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
> >> Thanks,Sandeep.
> >>
> >> > Date: Thu, 11 Jul 2013 18:47:17 +0530
> >> > Subject: Re: Region server going down when deleting a table from HBase
> >> > From: anoop.hbase@gmail.com
> >> > To: user@hbase.apache.org
> >> >
> >> > What is the full trace for DroppedSnapshotException ? The caused by
> >> trace?
> >> >
> >> > -Anoop-
> >> >
> >> > On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sandeepvreddy@outlook.com
> >> >wrote:
> >> >
> >> > > Hi,
> >> > > We are using hbase-0.94.1 with hadoop-1.0.2.
> >> > > Recently couple of time we faced a strange issue while deleting a
> >> table.
> >> > > Whenever we are deleting a table from HBase shell using disable and
> >> drop
> >> > > command at least one or two region servers are going down suddenly.
> >> > > I observed following error message in region server log:
> >> > > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> >> > > Unrecoverable exception while closing region
> >> > > REGION_NAMEjava.lang.RuntimeException:
> >> > > org.apache.hadoop.hbase.DroppedSnapshotException: region:
> REGION_NAME
> >> > > Previously also we used to disable and drop tables, but we never
> seen
> >> this
> >> > > kind of issue.
> >> > >
> >> > > Thanks,Sandeep.
> >>
> >>
> >
> >
>
Re: Region server going down when deleting a table from HBase
Posted by Anoop John <an...@gmail.com>.
Ya as Ted said, if this warn is not there, need to see what happened
to that file (just created)
-Anoop-
On Thu, Jul 11, 2013 at 8:07 PM, Anoop John <an...@gmail.com> wrote:
> After the flush opening the HFile reader. You are getting
> FileNotFoundException!
> *
>
> LOG
> *.warn("Unable to rename " + path + " to " + dstPath);
>
> Do you see above warn in log?
>
> In code I can see even if this rename got failed we try to open the file
> for read. Also as part of the region close, if we do a flush, we might
> not really do the file open. It will be an unwanted input stream open.
> Some one correct me if I am wrong pls.
> -Anoop-
>
> On Thu, Jul 11, 2013 at 6:54 PM, Sandeep L <sa...@outlook.com>wrote:
>
>> Anoop,
>> Following is stack trace
>> ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
>> while processing event M_RS_CLOSE_REGIONjava.lang.RuntimeException:
>> org.apache.hadoop.hbase.DroppedSnapshotException: region:
>> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at
>> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
>> at
>> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> at java.lang.Thread.run(Thread.java:722)Caused by:
>> org.apache.hadoop.hbase.DroppedSnapshotException: region:
>> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:905)
>> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845)
>> at
>> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
>> ... 4 moreCaused by: java.io.FileNotFoundException: File does not
>> exist:
>> /hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/ba8ae96c466f4559901127ca24378020
>> at
>> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843)
>> at
>> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834)
>> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
>> at
>> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
>> at
>> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
>> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at
>> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:547)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1245)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602)
>> at
>> org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:791)
>> at org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:108)
>> at
>> org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2256)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1455)
>> ... 8 more
>> INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
>> Thanks,Sandeep.
>>
>> > Date: Thu, 11 Jul 2013 18:47:17 +0530
>> > Subject: Re: Region server going down when deleting a table from HBase
>> > From: anoop.hbase@gmail.com
>> > To: user@hbase.apache.org
>> >
>> > What is the full trace for DroppedSnapshotException ? The caused by
>> trace?
>> >
>> > -Anoop-
>> >
>> > On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sandeepvreddy@outlook.com
>> >wrote:
>> >
>> > > Hi,
>> > > We are using hbase-0.94.1 with hadoop-1.0.2.
>> > > Recently couple of time we faced a strange issue while deleting a
>> table.
>> > > Whenever we are deleting a table from HBase shell using disable and
>> drop
>> > > command at least one or two region servers are going down suddenly.
>> > > I observed following error message in region server log:
>> > > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
>> > > Unrecoverable exception while closing region
>> > > REGION_NAMEjava.lang.RuntimeException:
>> > > org.apache.hadoop.hbase.DroppedSnapshotException: region: REGION_NAME
>> > > Previously also we used to disable and drop tables, but we never seen
>> this
>> > > kind of issue.
>> > >
>> > > Thanks,Sandeep.
>>
>>
>
>
Re: Region server going down when deleting a table from HBase
Posted by Anoop John <an...@gmail.com>.
After the flush opening the HFile reader. You are getting
FileNotFoundException!
*
LOG*.warn("Unable to rename " + path + " to " + dstPath);
Do you see above warn in log?
In code I can see even if this rename got failed we try to open the file
for read. Also as part of the region close, if we do a flush, we might
not really do the file open. It will be an unwanted input stream open.
Some one correct me if I am wrong pls.
-Anoop-
On Thu, Jul 11, 2013 at 6:54 PM, Sandeep L <sa...@outlook.com>wrote:
> Anoop,
> Following is stack trace
> ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
> while processing event M_RS_CLOSE_REGIONjava.lang.RuntimeException:
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)Caused by:
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:905)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
> ... 4 moreCaused by: java.io.FileNotFoundException: File does not
> exist:
> /hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/ba8ae96c466f4559901127ca24378020
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
> at
> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at
> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:547)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1245)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602)
> at
> org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:791)
> at org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:108)
> at
> org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2256)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1455)
> ... 8 more
> INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
> Thanks,Sandeep.
>
> > Date: Thu, 11 Jul 2013 18:47:17 +0530
> > Subject: Re: Region server going down when deleting a table from HBase
> > From: anoop.hbase@gmail.com
> > To: user@hbase.apache.org
> >
> > What is the full trace for DroppedSnapshotException ? The caused by
> trace?
> >
> > -Anoop-
> >
> > On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sandeepvreddy@outlook.com
> >wrote:
> >
> > > Hi,
> > > We are using hbase-0.94.1 with hadoop-1.0.2.
> > > Recently couple of time we faced a strange issue while deleting a
> table.
> > > Whenever we are deleting a table from HBase shell using disable and
> drop
> > > command at least one or two region servers are going down suddenly.
> > > I observed following error message in region server log:
> > > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> > > Unrecoverable exception while closing region
> > > REGION_NAMEjava.lang.RuntimeException:
> > > org.apache.hadoop.hbase.DroppedSnapshotException: region: REGION_NAME
> > > Previously also we used to disable and drop tables, but we never seen
> this
> > > kind of issue.
> > >
> > > Thanks,Sandeep.
>
>
Re: Region server going down when deleting a table from HBase
Posted by Ted Yu <yu...@gmail.com>.
bq. Caused by: java.io.FileNotFoundException: File does not exist:
/hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/
ba8ae96c466f4559901127ca24378020
Can you check in region server / namenode logs how the above file got
deleted ?
On Thu, Jul 11, 2013 at 6:24 AM, Sandeep L <sa...@outlook.com>wrote:
> Anoop,
> Following is stack trace
> ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable
> while processing event M_RS_CLOSE_REGIONjava.lang.RuntimeException:
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)Caused by:
> org.apache.hadoop.hbase.DroppedSnapshotException: region:
> TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:905)
> at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845)
> at
> org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119)
> ... 4 moreCaused by: java.io.FileNotFoundException: File does not
> exist:
> /hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/ba8ae96c466f4559901127ca24378020
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843)
> at
> org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834)
> at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154)
> at
> org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at
> org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:547)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1245)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512)
> at
> org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602)
> at
> org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:791)
> at org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:108)
> at
> org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2256)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1455)
> ... 8 more
> INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
> Thanks,Sandeep.
>
> > Date: Thu, 11 Jul 2013 18:47:17 +0530
> > Subject: Re: Region server going down when deleting a table from HBase
> > From: anoop.hbase@gmail.com
> > To: user@hbase.apache.org
> >
> > What is the full trace for DroppedSnapshotException ? The caused by
> trace?
> >
> > -Anoop-
> >
> > On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sandeepvreddy@outlook.com
> >wrote:
> >
> > > Hi,
> > > We are using hbase-0.94.1 with hadoop-1.0.2.
> > > Recently couple of time we faced a strange issue while deleting a
> table.
> > > Whenever we are deleting a table from HBase shell using disable and
> drop
> > > command at least one or two region servers are going down suddenly.
> > > I observed following error message in region server log:
> > > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> > > Unrecoverable exception while closing region
> > > REGION_NAMEjava.lang.RuntimeException:
> > > org.apache.hadoop.hbase.DroppedSnapshotException: region: REGION_NAME
> > > Previously also we used to disable and drop tables, but we never seen
> this
> > > kind of issue.
> > >
> > > Thanks,Sandeep.
>
>
RE: Region server going down when deleting a table from HBase
Posted by Sandeep L <sa...@outlook.com>.
Anoop,
Following is stack trace
ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable while processing event M_RS_CLOSE_REGIONjava.lang.RuntimeException: org.apache.hadoop.hbase.DroppedSnapshotException: region: TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:133) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:169) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722)Caused by: org.apache.hadoop.hbase.DroppedSnapshotException: region: TableName,better,1372421736221.55b13dd9eb08790e8e93757910209c21. at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1474) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1353) at org.apache.hadoop.hbase.regionserver.HRegion.doClose(HRegion.java:905) at org.apache.hadoop.hbase.regionserver.HRegion.close(HRegion.java:845) at org.apache.hadoop.hbase.regionserver.handler.CloseRegionHandler.process(CloseRegionHandler.java:119) ... 4 moreCaused by: java.io.FileNotFoundException: File does not exist: /hbase/TableName/55b13dd9eb08790e8e93757910209c21/WPA/ba8ae96c466f4559901127ca24378020 at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1843) at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1834) at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:578) at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:154) at org.apache.hadoop.fs.FilterFileSystem.open(FilterFileSystem.java:108) at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:427) at org.apache.hadoop.hbase.io.hfile.HFile.createReaderWithEncoding(HFile.java:547) at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1245) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:512) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:602) at org.apache.hadoop.hbase.regionserver.Store.commitFile(Store.java:791) at org.apache.hadoop.hbase.regionserver.Store.access$500(Store.java:108) at org.apache.hadoop.hbase.regionserver.Store$StoreFlusherImpl.commit(Store.java:2256) at org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1455) ... 8 more
INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60020
Thanks,Sandeep.
> Date: Thu, 11 Jul 2013 18:47:17 +0530
> Subject: Re: Region server going down when deleting a table from HBase
> From: anoop.hbase@gmail.com
> To: user@hbase.apache.org
>
> What is the full trace for DroppedSnapshotException ? The caused by trace?
>
> -Anoop-
>
> On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sa...@outlook.com>wrote:
>
> > Hi,
> > We are using hbase-0.94.1 with hadoop-1.0.2.
> > Recently couple of time we faced a strange issue while deleting a table.
> > Whenever we are deleting a table from HBase shell using disable and drop
> > command at least one or two region servers are going down suddenly.
> > I observed following error message in region server log:
> > INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> > Unrecoverable exception while closing region
> > REGION_NAMEjava.lang.RuntimeException:
> > org.apache.hadoop.hbase.DroppedSnapshotException: region: REGION_NAME
> > Previously also we used to disable and drop tables, but we never seen this
> > kind of issue.
> >
> > Thanks,Sandeep.
Re: Region server going down when deleting a table from HBase
Posted by Anoop John <an...@gmail.com>.
What is the full trace for DroppedSnapshotException ? The caused by trace?
-Anoop-
On Thu, Jul 11, 2013 at 6:38 PM, Sandeep L <sa...@outlook.com>wrote:
> Hi,
> We are using hbase-0.94.1 with hadoop-1.0.2.
> Recently couple of time we faced a strange issue while deleting a table.
> Whenever we are deleting a table from HBase shell using disable and drop
> command at least one or two region servers are going down suddenly.
> I observed following error message in region server log:
> INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED:
> Unrecoverable exception while closing region
> REGION_NAMEjava.lang.RuntimeException:
> org.apache.hadoop.hbase.DroppedSnapshotException: region: REGION_NAME
> Previously also we used to disable and drop tables, but we never seen this
> kind of issue.
>
> Thanks,Sandeep.