You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Renaud Delbru <re...@deri.org> on 2008/07/18 11:11:24 UTC
Replay of hlog required, forcing regionserver restart
Hi,
I am using HBase 0.2.0-dev, Hudson Build #208.
I am experiencing a problem with HBase that looks like the issue
https://issues.apache.org/jira/browse/HBASE-15.
During an upload intensive task, one of the regionserver went down to
force rerun of hlog.
This is not the first time we have experienced this problem, it happened
already on another cluster node a few days ago, in the same settings.
I have attached the regionserver log. I don't notice something
particular in the hadoop datanode log.
Thanks in advance for any advices.
--
Renaud Delbru
Re: Replay of hlog required, forcing regionserver restart
Posted by Andrew Purtell <ap...@yahoo.com>.
Hello Renaud,
I think your cluster is overloaded.
> WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
> ten times longer than scheduled: 3000
> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We
> slept 583701ms, ten times longer than scheduled: 3000
See https://issues.apache.org/jira/browse/HBASE-616.
I think these types of messages imply excessive load, with processes stuck in disk wait, and/or threads starved for CPU time, etc. Processes can be stuck in disk wait while being swapped in or out also, so excessive RAM/heap is a third possibility. Given a 3 node cluster with a full complement of Hadoop and HBase daemons and 24 tasks, perhaps all three load-dimensions are redlined. :-)
It's hard to say what exactly was the root cause of the exception on the DFS client channel which brought down the regionserver:
> java.io.IOException: Premeture EOF from inputStream at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
This could have been a socket timeout, or a fault on the data node. Does anything useful/meaningful appear in the datanode log?
- Andy
--- On Sat, 7/19/08, Renaud Delbru <re...@deri.org> wrote:
> From: Renaud Delbru <re...@deri.org>
> Subject: Re: Replay of hlog required, forcing regionserver restart
> To: hbase-user@hadoop.apache.org
> Date: Saturday, July 19, 2008, 7:46 AM
> The cluster is currently composed of 3 nodes. The cluster is
> also used for executing hadoop jobs. One of the node acts as an
> hadoop master, an HMaster and an HRegionServer.
> The current settings are: Xmx=1024M for the Hadoop Master,
> Xmx=2048M for the HMaster, Xmx=2048M for each HRegionServer.
>
> The inserts are somehow randomised. We have 24 hadoop jobs
> uploading various documents. The row key is the URL of the document,
> and each hadoop task has to upload a range of URLs (for example,
> http://my-url/AAA - http://my-url/BBB).
> Thus, I assume that each hadoop tasks will hit a different
> region and that the insert load is distributed among the nodes.
>
> Thanks.
> --
> Renaud Delbru
>
> Jean-Daniel Cryans wrote:
> > Ok then, at least having the nodes synchronized keeps
> you safe from other
> > problems. By the way, how many nodes do you have?
> >
> > One other thing to check is your writing pattern. If
> you insert data with
> > sequential row keys, an example would be designing
> your row keys as an
> > incremented primary key equivalent but left padded
> with zeros, these inserts
> > will always hit on a single region thus the load will
> not be distributed. If
> > it's your case, try randomizing the inserts.
> >
> > I will also look at your logs monday morning.
> >
> > Thx,
> >
> > J-D
> >
> > On Sat, Jul 19, 2008 at 5:47 AM, Renaud Delbru
> <re...@deri.org>
> > wrote:
> >
> >
> >> Hi Jean-Daniel,
> >>
> >> I have synchronised all the nodes to the same ntp
> server yesterday.
> >> This morning, another regionserver went down for
> the same reason (log
> >> attached).
> >>
> >> Thanks.
> >> --
> >> Renaud Delbru
> >>
> >>
> >>
> >> Jean-Daniel Cryans wrote:
> >>
> >>
> >>> Renaud,
> >>>
> >>> Yes, the impact is important. Try fixing it
> first because typically heavy
> >>> write load problems only occur when there is a
> very few number of nodes
> >>> and
> >>> regions. We have seen this many times.
> >>>
> >>> Happy fixing!
> >>>
> >>> J-D
> >>>
> >>> On Fri, Jul 18, 2008 at 5:20 PM, Renaud Delbru
> <re...@deri.org>
> >>> wrote:
> >>>
> >>>
> >>>
> >>>
> >>>> Hi Jean-Daniel,
> >>>>
> >>>> my nodes clocks are desynchronised (each
> node has a different time).
> >>>> Could
> >>>> it have an impact on HBase ?
> >>>>
> >>>> At the moment of the failure, the node had
> about 55 regions (with
> >>>> hbase.hregion.max.filesize = 128M).
> >>>>
> >>>> Thanks.
> >>>> --
> >>>> Renaud Delbru
> >>>>
> >>>> As One node is at
> >>>>
> >>>> Jean-Daniel Cryans wrote:
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> Renaud,
> >>>>>
> >>>>> Just to make sure, are all your nodes
> clocks synchronized? Also, at the
> >>>>> moment of that failure, how many
> regions did the table have?
> >>>>>
> >>>>> Thanks,
> >>>>>
> >>>>> J-D
> >>>>>
> >>>>> On Fri, Jul 18, 2008 at 5:11 AM,
> Renaud Delbru <re...@deri.org>
> >>>>> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I am using HBase 0.2.0-dev, Hudson
> Build #208.
> >>>>>> I am experiencing a problem with
> HBase that looks like the issue
> >>>>>>
> https://issues.apache.org/jira/browse/HBASE-15.
> >>>>>> During an upload intensive task,
> one of the regionserver went down to
> >>>>>> force rerun of hlog.
> >>>>>>
> >>>>>> This is not the first time we have
> experienced this problem, it
> >>>>>> happened
> >>>>>> already on another cluster node a
> few days ago, in the same settings.
> >>>>>>
> >>>>>> I have attached the regionserver
> log. I don't notice something
> >>>>>> particular in the hadoop datanode
> log.
> >>>>>>
> >>>>>> Thanks in advance for any advices.
> >>>>>> --
> >>>>>> Renaud Delbru
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> 2008-07-18 03:40:51,861 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
> hlog. Number of
> >>>>>> entries: 30008
> >>>>>> 2008-07-18 03:40:52,073 INFO
> org.apache.hadoop.hbase.regionserver.HLog:
> >>>>>> New
> >>>>>> log writer created at
> >>>>>>
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
> >>>>>> 2008-07-18 03:43:36,257 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
> hlog. Number of
> >>>>>> entries: 30008
> >>>>>> 2008-07-18 03:43:37,020 INFO
> org.apache.hadoop.hbase.regionserver.HLog:
> >>>>>> New
> >>>>>> log writer created at
> >>>>>>
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
> >>>>>> 2008-07-18 03:45:12,963 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18
> >>>>>>
> >>>>>>> 03:45:58,868
> >>>>>>>
> >>>>>> WARN
> org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
> >>>>>>
> >>>>>> ten times longer than scheduled:
> 3000
> >>>>>> 2008-07-18 03:55:57,783 WARN
> org.apache.hadoop.hbase.util.Sleeper: We
> >>>>>> slept
> >>>>>> 583701ms, ten times longer than
> scheduled: 3000
> >>>>>> 2008-07-18 03:55:57,787 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
> hlog. Number of
> >>>>>> entries: 21011
> >>>>>> 2008-07-18 03:55:58,410 WARN
> org.apache.hadoop.dfs.DFSClient: Exception
> >>>>>> while reading from
> blk_-6424749704096696952 of
> >>>>>>
> >>>>>>
> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
> >>>>>> from 140.203.154.176:50010:
> java.io.IOException: Premeture EOF from
> >>>>>> inputStream
> >>>>>> at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
> >>>>>> at
> java.io.DataInputStream.readFully(DataInputStream.java:178)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
> >>>>>>
> >>>>>> 2008-07-18 03:55:58,966 WARN
> org.apache.hadoop.dfs.DFSClient: Exception
> >>>>>> while reading from
> blk_7305122468419183134 of
> >>>>>>
> >>>>>>
> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
> >>>>>> from 140.203.154.176:50010:
> java.io.IOException: Premeture EOF from
> >>>>>> inputStream
> >>>>>> at
> org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
> >>>>>> at
> java.io.DataInputStream.readFully(DataInputStream.java:178)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
> >>>>>>
> >>>>>> 2008-07-18 03:56:05,305 INFO
> org.apache.hadoop.hbase.regionserver.HLog:
> >>>>>> New
> >>>>>> log writer created at
> >>>>>>
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
> >>>>>> 2008-07-18 03:57:30,244 ERROR
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error
> opening
> >>>>>> scanner
> >>>>>> (fsOk: true)
> >>>>>> java.io.IOException: HStoreScanner
> failed construction
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor10.invoke(Unknown
> Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> Caused by:
> java.net.SocketTimeoutException: timed out waiting for rpc
> >>>>>> response
> >>>>>> at
> org.apache.hadoop.ipc.Client.call(Client.java:559)
> >>>>>> at
> org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> >>>>>> at
> org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> >>>>>> at
> org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> >>>>>> at
> org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
> >>>>>> at
> org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
> >>>>>> ... 10 more
> >>>>>> 2008-07-18 03:58:01,895 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 7 on 60020, call
> openScanner([B@9dd0d92, [[B@fa5f14d,
> >>>>>> [B@5784fe41,
> >>>>>> 9223372036854775807, null) from
> 140.203.154.174:48958: error:
> >>>>>> java.io.IOException: HStoreScanner
> failed construction
> >>>>>> java.io.IOException: HStoreScanner
> failed construction
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor10.invoke(Unknown
> Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> Caused by:
> java.net.SocketTimeoutException: timed out waiting for rpc
> >>>>>> response
> >>>>>> at
> org.apache.hadoop.ipc.Client.call(Client.java:559)
> >>>>>> at
> org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> >>>>>> at
> org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> >>>>>> at
> org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> >>>>>> at
> org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
> >>>>>> at
> org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
> >>>>>> ... 10 more
> >>>>>> 2008-07-18 03:58:06,728 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> compaction completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
> in 12mins,
> >>>>>> 53sec
> >>>>>> 2008-07-18 03:58:06,728 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Angus_McKay,1215890852482
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18
> >>>>>>
> >>>>>>> 03:58:06,729
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Angus_McKay,1215890852482 in
> 0sec
> >>>>>> 2008-07-18 03:58:06,729 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18
> >>>>>>
> >>>>>>> 03:58:06,730
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
> in 0sec
> >>>>>> 2008-07-18 03:58:06,730 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18
> >>>>>>
> >>>>>>> 03:58:06,760
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in0sec
> >>>>>> 2008-07-18 03:58:06,760 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18
> >>>>>>
> >>>>>>> 03:58:06,760
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in0sec
> >>>>>> 2008-07-18 03:58:06,761 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18
> >>>>>>
> >>>>>>> 03:58:06,761
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
> in 0sec
> >>>>>> 2008-07-18 03:58:06,761 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18
> >>>>>>
> >>>>>>> 03:58:06,777
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in0sec
> >>>>>> 2008-07-18 03:58:06,777 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18
> >>>>>>
> >>>>>>> 03:58:06,800
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
> in 0sec
> >>>>>> 2008-07-18 03:58:06,800 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18
> >>>>>>
> >>>>>>> 03:58:06,826
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663in
> >>>>>> 0sec
> >>>>>> 2008-07-18 03:58:06,826 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18
> >>>>>>
> >>>>>>> 03:58:06,916
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in0sec
> >>>>>> 2008-07-18 03:58:06,916 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://sws.geonames.org/3025251/,1215824259058
> >>>>>> 2008-07-18
> <http://sws.geonames.org/3025251/,12158242590582008-07-18
> >>>>>>
> >>>>>>> 03:58:06,972
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://sws.geonames.org/3025251/,1215824259058 in 0sec
> >>>>>> 2008-07-18 03:58:06,973 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18
> >>>>>>
> >>>>>>> 03:58:06,973
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046in
> >>>>>> 0sec
> >>>>>> 2008-07-18 03:58:06,973 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18
> >>>>>>
> >>>>>>> 03:58:15,979
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
> >>>>>> hlog. Number of entries: 30009
> >>>>>> 2008-07-18 03:59:32,093 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> compaction completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
> in 1mins,
> >>>>>> 25sec
> >>>>>> 2008-07-18 03:59:32,093 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://acm.rkbexplorer.com/id/person-592138,1215824878330
> >>>>>> 2008-07-18<
> >>>>>>
> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18
> >>>>>>
> >>>>>>> 03:59:32,134
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://acm.rkbexplorer.com/id/person-592138,1215824878330
> in 0sec
> >>>>>> 2008-07-18 03:59:32,134 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18
> >>>>>>
> >>>>>>> 04:00:45,884
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
> in
> >>>>>> 1mins,
> >>>>>> 13sec
> >>>>>> 2008-07-18 04:00:45,885 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18
> >>>>>>
> >>>>>>> 04:00:45,887
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
> in 0sec
> >>>>>> 2008-07-18 04:00:45,887 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18
> >>>>>>
> >>>>>>> 04:00:45,892
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in0sec
> >>>>>> 2008-07-18 04:00:45,892 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Lysander/lang2,1215922127438
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18
> >>>>>>
> >>>>>>> 04:00:45,893
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in
> 0sec
> >>>>>> 2008-07-18 04:00:45,893 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
> >>>>>> 2008-07-18<
> >>>>>>
> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18
> >>>>>>
> >>>>>>> 04:00:45,895
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in
> 0sec
> >>>>>> 2008-07-18 04:00:45,895 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://sws.geonames.org/1081116/,1215823957458
> >>>>>> 2008-07-18
> <http://sws.geonames.org/1081116/,12158239574582008-07-18
> >>>>>>
> >>>>>>> 04:00:45,895
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://sws.geonames.org/1081116/,1215823957458 in 0sec
> >>>>>> 2008-07-18 04:00:45,896 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Allomeric,1216039960663
> >>>>>> 2008-07-18 <
> >>>>>>
> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18
> >>>>>>
> >>>>>>> 04:00:45,935
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
> >>>>>> 2008-07-18 04:00:45,935 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18
> >>>>>>
> >>>>>>> 04:00:45,993
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in0sec
> >>>>>> 2008-07-18 04:00:45,994 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18
> >>>>>>
> >>>>>>> 04:00:46,004
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
> in
> >>>>>> 0sec
> >>>>>> 2008-07-18 04:00:46,005 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18
> >>>>>>
> >>>>>>> 04:00:46,005
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
> in
> >>>>>> 0sec
> >>>>>> 2008-07-18 04:00:46,005 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> starting compaction on region
> page-repository,
> >>>>>>
> http://purl.uniprot.org/taxonomy/112557,1214582151856
> >>>>>> 2008-07-18<
> >>>>>>
> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18
> >>>>>>
> >>>>>>> 04:00:46,007
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: compaction
> >>>>>>
> >>>>>> completed on region
> page-repository,
> >>>>>>
> http://purl.uniprot.org/taxonomy/112557,1214582151856 in
> 0sec
> >>>>>> 2008-07-18 04:02:16,798 FATAL
> >>>>>>
> org.apache.hadoop.hbase.regionserver.Flusher:
> >>>>>> Replay of hlog required. Forcing
> server restart
> >>>>>>
> org.apache.hadoop.hbase.DroppedSnapshotException
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
> >>>>>> at
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
> >>>>>> Caused by:
> org.apache.hadoop.ipc.RemoteException:
> >>>>>>
> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed
> to create
> >>>>>> file
> >>>>>>
> >>>>>>
> >>>>>>
> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
> >>>>>> for DFSClient_716087201 on client
> 140.203.154.176 because current
> >>>>>> leaseholder is trying to recreate
> file.
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
> >>>>>> at
> org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor44.invoke(Unknown
> Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>>
> >>>>>> at
> org.apache.hadoop.ipc.Client.call(Client.java:557)
> >>>>>> at
> org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> >>>>>> at
> org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor12.invoke(Unknown
> Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> >>>>>> at
> org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
> >>>>>> at
> org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
> >>>>>> at
> org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
> >>>>>> at
> org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
> >>>>>> ... 3 more
> >>>>>> 2008-07-18 04:02:16,798 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.Flusher:
> >>>>>>
> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
> >>>>>> 2008-07-18 04:02:16,807 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 1 on 60020, call
> batchUpdate([B@167e3c87, row =>
> >>>>>>
> http://dbpedia.org/resource/Jurrette_Sindelar, {column
> => field:class,
> >>>>>> value => '...', column
> => field:data-source, value => '...', column
> =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.174:50719: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:16,847 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 9 on 60020, call
> batchUpdate([B@525cfe10, row =>
> >>>>>>
> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column
> =>
> >>>>>> field:class,
> >>>>>> value => '...', column
> => field:data-source, value => '...', column
> =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.175:48121: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:16,848 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 0 on 60020, call
> batchUpdate([B@572bf6ae, row =>
> >>>>>>
> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29,
> >>>>>> {column
> >>>>>> => field:class, value =>
> '...', column => field:data-source, value =>
> >>>>>> '...',
> >>>>>> column => field:domain, value
> => '...', column =>
> >>>>>> field:explicit-content,
> >>>>>> value => '...', column
> => field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.175:48348: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:16,905 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 4 on 60020, call
> batchUpdate([B@27320a48, row =>
> >>>>>>
> http://dbpedia.org/resource/Karim_Benkhirane, {column =>
> field:class,
> >>>>>> value => '...', column
> => field:data-source, value => '...', column
> =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.175:48330: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:16,926 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 4 on 60020, call
> batchUpdate([B@59c7d8c6, row =>
> >>>>>>
> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column
> =>
> >>>>>> field:class,
> >>>>>> value => '...', column
> => field:data-source, value => '...', column
> =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.176:58334: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:16,959 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 8 on 60020, call
> batchUpdate([B@4f68dd36, row =>
> >>>>>>
> http://dbpedia.org/resource/Hendrik_Speuy, {column =>
> field:class,
> >>>>>> value
> >>>>>> => '...', column =>
> field:data-source, value => '...', column =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.175:48357: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:16,979 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 3 on 60020, call
> batchUpdate([B@596c7f5f, row =>
> >>>>>>
> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column
> =>
> >>>>>> field:class,
> >>>>>> value => '...', column
> => field:data-source, value => '...', column
> =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.176:58163: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:17,147 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 2 on 60020, call
> batchUpdate([B@23067cee, row =>
> >>>>>>
> http://dbpedia.org/resource/High_Littleton, {column =>
> field:class,
> >>>>>> value
> >>>>>> => '...', column =>
> field:data-source, value => '...', column =>
> >>>>>> field:domain, value =>
> '...', column => field:explicit-content, value
> >>>>>> =>
> >>>>>> '...', column =>
> field:format, value => '...', column =>
> >>>>>> field:implicit-content, value
> => '...', column => field:label, value =>
> >>>>>> '...', column =>
> field:length, value => '...', column =>
> >>>>>> field:ontology,
> >>>>>> value => '...', column
> => field:size, value => '...', column =>
> >>>>>> field:timestamp, value =>
> '...'}) from 140.203.154.174:50952: error:
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:17,180 INFO
> org.apache.hadoop.hbase.regionserver.HLog:
> >>>>>> New
> >>>>>> log writer created at
> >>>>>>
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
> >>>>>> 2008-07-18 04:02:17,181 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller
> exiting.
> >>>>>> 2008-07-18 04:02:17,376 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 7 on 60020, call
> getClosestRowBefore([B@20d09fed, [B@60b3916f)
> >>>>>> from 140.203.154.175:48464: error:
> java.io.IOException: Server not
> >>>>>> running
> >>>>>> java.io.IOException: Server not
> running
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
> >>>>>> at
> sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> >>>>>> at
> >>>>>>
> >>>>>>
> >>>>>>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >>>>>> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >>>>>> at
> >>>>>>
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> >>>>>> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>>>>> 2008-07-18 04:02:17,507 INFO
> org.apache.hadoop.ipc.Server: Stopping
> >>>>>> server
> >>>>>> on 60020
> >>>>>> 2008-07-18 04:02:17,508 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 1 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,508 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 5 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,587 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 0 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,600 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 2 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,600 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 9 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,601 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 8 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,601 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 7 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,601 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 6 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,601 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 4 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,601 INFO
> org.apache.hadoop.ipc.Server: IPC Server
> >>>>>> handler 3 on 60020: exiting
> >>>>>> 2008-07-18 04:02:17,607 INFO
> org.apache.hadoop.ipc.Server: Stopping IPC
> >>>>>> Server listener on 60020
> >>>>>> 2008-07-18 04:02:17,691 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> Stopping infoServer
> >>>>>> 2008-07-18 04:02:17,784 INFO
> org.apache.hadoop.ipc.Server: Stopping IPC
> >>>>>> Server Responder
> >>>>>> 2008-07-18 04:02:17,902 INFO
> org.mortbay.util.ThreadedServer: Stopping
> >>>>>> Acceptor
> ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
> >>>>>> 2008-07-18 04:02:18,179 INFO
> org.mortbay.http.SocketListener: Stopped
> >>>>>> SocketListener on 0.0.0.0:60030
> >>>>>> 2008-07-18 04:02:28,883 INFO
> org.mortbay.util.Container: Stopped
> >>>>>> HttpContext[/static,/static]
> >>>>>> 2008-07-18 04:02:31,116 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: worker
> thread
> >>>>>> exiting
> >>>>>> 2008-07-18 04:02:31,117 INFO
> org.mortbay.util.Container: Stopped
> >>>>>> HttpContext[/logs,/logs]
> >>>>>> 2008-07-18 04:02:31,117 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
> >>>>>>
> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
> >>>>>> 2008-07-18 04:02:31,125 INFO
> org.mortbay.util.Container: Stopped
> >>>>>>
> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
> >>>>>> 2008-07-18 04:02:34,017 INFO
> org.mortbay.util.Container: Stopped
> >>>>>> WebApplicationContext[/,/]
> >>>>>> 2008-07-18 04:02:34,017 INFO
> org.mortbay.util.Container: Stopped
> >>>>>> org.mortbay.jetty.Server@b8c3679
> >>>>>> 2008-07-18 04:02:34,309 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: On
> abort, closed
> >>>>>> hlog
> >>>>>> 2008-07-18 04:55:59,098 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -3929282808018198437 lease expired
> >>>>>> 2008-07-18 04:56:00,258 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> 8694039619106515508 lease expired
> >>>>>> 2008-07-18 04:56:02,068 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -6642383936144787497 lease expired
> >>>>>> 2008-07-18 04:56:02,178 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -8674274402855984090 lease expired
> >>>>>> 2008-07-18 04:56:02,858 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -8684213796880407868 lease expired
> >>>>>> 2008-07-18 04:56:03,478 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -8003294377021166340 lease expired
> >>>>>> 2008-07-18 04:56:05,298 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> 4741746108028229290 lease expired
> >>>>>> 2008-07-18 04:56:05,418 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -3662383214249882020 lease expired
> >>>>>> 2008-07-18 04:56:05,498 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> >>>>>> -5749397523507975233 lease expired
> >>>>>> 2008-07-18 04:56:05,499 INFO
> org.apache.hadoop.hbase.Leases:
> >>>>>>
> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing
> leases
> >>>>>> 2008-07-18 04:56:05,499 INFO
> org.apache.hadoop.hbase.Leases:
> >>>>>>
> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed
> leases
> >>>>>> 2008-07-18 04:56:05,499 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> closed .META.,,1
> >>>>>> 2008-07-18 04:56:05,499 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegion:
> >>>>>> closed page-repository,
> >>>>>>
> http://dbpedia.org/resource/CAMBIA,1215970756532
> >>>>>> 2008-07-18
> <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18
> >>>>>>
> >>>>>>> 04:56:05,499
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18
> >>>>>>
> >>>>>>> 04:56:05,499
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18
> >>>>>>
> >>>>>>> 04:56:05,499
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://sws.geonames.org/1081116/,1215823957458
> >>>>>> 2008-07-18
> <http://sws.geonames.org/1081116/,12158239574582008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://acm.rkbexplorer.com/id/person-592138,1215824878330
> >>>>>> 2008-07-18<
> >>>>>>
> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
> >>>>>> 2008-07-18
> <http://purl.uniprot.org/go/0009140,12145826028822008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18
> >>>>>>
> >>>>>>> 04:56:05,500
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://bio2rdf.org/omim:300357,1215825157318
> >>>>>> 2008-07-18
> <http://bio2rdf.org/omim:300357,12158251573182008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
> >>>>>> 2008-07-18
> <http://alcidesfonseca.com/blog/35,12158259255952008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
> >>>>>> 2008-07-18 <
> >>>>>>
> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
> >>>>>> 2008-07-18<
> >>>>>>
> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/John_J._Bernet,1216109320961
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18
> >>>>>>
> >>>>>>> 04:56:05,501
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://sws.geonames.org/3025251/,1215824259058
> >>>>>> 2008-07-18
> <http://sws.geonames.org/3025251/,12158242590582008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Billy_Forester,1216165245424
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
> >>>>>> 2008-07-18 <
> >>>>>>
> http://dbpedia.org/resource/Snatcho,12162634022752008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Template:superheroe,1216140016580
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18
> >>>>>>
> >>>>>>> 04:56:05,502
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
> >>>>>> 2008-07-18 <
> >>>>>>
> http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
> >>>>>> 2008-07-18 <
> >>>>>>
> http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18
> >>>>>>
> >>>>>>> 04:56:05,503
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
> >>>>>> 2008-07-18<
> >>>>>>
> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Harry_Girvetz,1215910087698
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://acm.rkbexplorer.com/id/person-94180,1215825860040
> >>>>>> 2008-07-18<
> >>>>>>
> http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18
> >>>>>>
> >>>>>>> 04:56:05,504
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
> >>>>>> 2008-07-18<
> >>>>>>
> >>>>>>
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18
> >>>>>>
> >>>>>>> 04:56:05,505
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Lysander/lang2,1215922127438
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18
> >>>>>>
> >>>>>>> 04:56:05,506
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://sws.geonames.org/2701307/,1215823954578
> >>>>>> 2008-07-18
> <http://sws.geonames.org/2701307/,12158239545782008-07-18
> >>>>>>
> >>>>>>> 04:56:05,506
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>> page-repository,
> >>>>>>
> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18
> >>>>>>
> >>>>>>> 04:56:05,506
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegion: closed
> >>>>>>
> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
> >>>>>> 2008-07-18<
> >>>>>>
> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18
> >>>>>>
> >>>>>>> 04:56:05,506
> >>>>>>>
> >>>>>> INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> >>>>>> aborting server at:
> 140.203.154.176:60020
> >>>>>> 2008-07-18 04:56:05,506 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> >>>>>> regionserver/0:0:0:0:0:0:0:0:60020
> exiting
> >>>>>> 2008-07-18 04:56:05,561 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> Starting shutdown
> >>>>>> thread.
> >>>>>> 2008-07-18 04:56:05,562 INFO
> >>>>>>
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> Shutdown thread
> >>>>>> complete
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >> 2008-07-19 07:31:45,505 WARN
> org.apache.hadoop.dfs.DFSClient: DataStreamer
> >> Exception: java.net.SocketTimeoutException: timed
> out waiting for rpc
> >> response
> >> 2008-07-19 07:31:45,506 WARN
> org.apache.hadoop.dfs.DFSClient: Error
> >> Recovery for block null bad datanode[0]
> >> 2008-07-19 07:31:54,868 FATAL
> org.apache.hadoop.hbase.regionserver.Flusher:
> >> Replay of hlog required. Forcing server restart
> >> org.apache.hadoop.hbase.DroppedSnapshotException
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
> >> at
> org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
> >> Caused by: org.apache.hadoop.ipc.RemoteException:
> java.io.IOException:
> >> Could not complete write to file
> >>
> /hbase/page-repository/963177038/field/info/4591605661998875281
> by
> >> DFSClient_1009756630
> >> at
> org.apache.hadoop.dfs.NameNode.complete(NameNode.java:332)
> >> at
> sun.reflect.GeneratedMethodAccessor19.invoke(Unknown
> Source)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >> at
> org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> >> at
> org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> >>
> >> at
> org.apache.hadoop.ipc.Client.call(Client.java:557)
> >> at
> org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> >> at
> org.apache.hadoop.dfs.$Proxy1.complete(Unknown Source)
> >> at
> sun.reflect.GeneratedMethodAccessor15.invoke(Unknown
> Source)
> >> at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >> at
> java.lang.reflect.Method.invoke(Method.java:597)
> >> at
> >>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> >> at
> >>
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> >> at
> org.apache.hadoop.dfs.$Proxy1.complete(Unknown Source)
> >> at
> >>
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2655)
> >> at
> >>
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2576)
> >> at
> >>
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:59)
> >> at
> >>
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:79)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HStoreFile.writeInfo(HStoreFile.java:350)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:747)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
> >> at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
> >> ... 3 more
> >> 2008-07-19 07:31:54,868 INFO
> org.apache.hadoop.hbase.regionserver.Flusher:
> >> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher
> exiting
> >>
> >>
> >>
> >
> >
Re: Replay of hlog required, forcing regionserver restart
Posted by Renaud Delbru <re...@deri.org>.
The cluster is currently composed of 3 nodes. The cluster is also used
for executing hadoop jobs. One of the node acts as an hadoop master, an
HMaster and an HRegionServer.
The current settings are: Xmx=1024M for the Hadoop Master, Xmx=2048M for
the HMaster, Xmx=2048M for each HRegionServer.
The inserts are somehow randomised. We have 24 hadoop jobs uploading
various documents. The row key is the URL of the document, and each
hadoop task has to upload a range of URLs (for example,
http://my-url/AAA - http://my-url/BBB).
Thus, I assume that each hadoop tasks will hit a different region and
that the insert load is distributed among the nodes.
Thanks.
--
Renaud Delbru
Jean-Daniel Cryans wrote:
> Ok then, at least having the nodes synchronized keeps you safe from other
> problems. By the way, how many nodes do you have?
>
> One other thing to check is your writing pattern. If you insert data with
> sequential row keys, an example would be designing your row keys as an
> incremented primary key equivalent but left padded with zeros, these inserts
> will always hit on a single region thus the load will not be distributed. If
> it's your case, try randomizing the inserts.
>
> I will also look at your logs monday morning.
>
> Thx,
>
> J-D
>
> On Sat, Jul 19, 2008 at 5:47 AM, Renaud Delbru <re...@deri.org>
> wrote:
>
>
>> Hi Jean-Daniel,
>>
>> I have synchronised all the nodes to the same ntp server yesterday.
>> This morning, another regionserver went down for the same reason (log
>> attached).
>>
>> Thanks.
>> --
>> Renaud Delbru
>>
>>
>>
>> Jean-Daniel Cryans wrote:
>>
>>
>>> Renaud,
>>>
>>> Yes, the impact is important. Try fixing it first because typically heavy
>>> write load problems only occur when there is a very few number of nodes
>>> and
>>> regions. We have seen this many times.
>>>
>>> Happy fixing!
>>>
>>> J-D
>>>
>>> On Fri, Jul 18, 2008 at 5:20 PM, Renaud Delbru <re...@deri.org>
>>> wrote:
>>>
>>>
>>>
>>>
>>>> Hi Jean-Daniel,
>>>>
>>>> my nodes clocks are desynchronised (each node has a different time).
>>>> Could
>>>> it have an impact on HBase ?
>>>>
>>>> At the moment of the failure, the node had about 55 regions (with
>>>> hbase.hregion.max.filesize = 128M).
>>>>
>>>> Thanks.
>>>> --
>>>> Renaud Delbru
>>>>
>>>> As One node is at
>>>>
>>>> Jean-Daniel Cryans wrote:
>>>>
>>>>
>>>>
>>>>
>>>>> Renaud,
>>>>>
>>>>> Just to make sure, are all your nodes clocks synchronized? Also, at the
>>>>> moment of that failure, how many regions did the table have?
>>>>>
>>>>> Thanks,
>>>>>
>>>>> J-D
>>>>>
>>>>> On Fri, Jul 18, 2008 at 5:11 AM, Renaud Delbru <re...@deri.org>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am using HBase 0.2.0-dev, Hudson Build #208.
>>>>>> I am experiencing a problem with HBase that looks like the issue
>>>>>> https://issues.apache.org/jira/browse/HBASE-15.
>>>>>> During an upload intensive task, one of the regionserver went down to
>>>>>> force rerun of hlog.
>>>>>>
>>>>>> This is not the first time we have experienced this problem, it
>>>>>> happened
>>>>>> already on another cluster node a few days ago, in the same settings.
>>>>>>
>>>>>> I have attached the regionserver log. I don't notice something
>>>>>> particular in the hadoop datanode log.
>>>>>>
>>>>>> Thanks in advance for any advices.
>>>>>> --
>>>>>> Renaud Delbru
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2008-07-18 03:40:51,861 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>>>> entries: 30008
>>>>>> 2008-07-18 03:40:52,073 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>>> New
>>>>>> log writer created at
>>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
>>>>>> 2008-07-18 03:43:36,257 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>>>> entries: 30008
>>>>>> 2008-07-18 03:43:37,020 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>>> New
>>>>>> log writer created at
>>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
>>>>>> 2008-07-18 03:45:12,963 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18
>>>>>>
>>>>>>> 03:45:58,868
>>>>>>>
>>>>>> WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
>>>>>>
>>>>>> ten times longer than scheduled: 3000
>>>>>> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We
>>>>>> slept
>>>>>> 583701ms, ten times longer than scheduled: 3000
>>>>>> 2008-07-18 03:55:57,787 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>>>> entries: 21011
>>>>>> 2008-07-18 03:55:58,410 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>>>>> while reading from blk_-6424749704096696952 of
>>>>>>
>>>>>> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
>>>>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>>>>> inputStream
>>>>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>>>>> at
>>>>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>>>>> at
>>>>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>>>>> at
>>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>>>>> at
>>>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
>>>>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>>>>
>>>>>> 2008-07-18 03:55:58,966 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>>>>> while reading from blk_7305122468419183134 of
>>>>>>
>>>>>> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
>>>>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>>>>> inputStream
>>>>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>>>>> at
>>>>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>>>>> at
>>>>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>>>>> at
>>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>>>>> at
>>>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
>>>>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>>>>
>>>>>> 2008-07-18 03:56:05,305 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>>> New
>>>>>> log writer created at
>>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
>>>>>> 2008-07-18 03:57:30,244 ERROR
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
>>>>>> scanner
>>>>>> (fsOk: true)
>>>>>> java.io.IOException: HStoreScanner failed construction
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>>>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>>>> response
>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>>>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>>>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>>>>> ... 10 more
>>>>>> 2008-07-18 03:58:01,895 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 7 on 60020, call openScanner([B@9dd0d92, [[B@fa5f14d,
>>>>>> [B@5784fe41,
>>>>>> 9223372036854775807, null) from 140.203.154.174:48958: error:
>>>>>> java.io.IOException: HStoreScanner failed construction
>>>>>> java.io.IOException: HStoreScanner failed construction
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>>>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>>>> response
>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>>>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>>>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>>>>> ... 10 more
>>>>>> 2008-07-18 03:58:06,728 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> compaction completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401 in 12mins,
>>>>>> 53sec
>>>>>> 2008-07-18 03:58:06,728 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Angus_McKay,1215890852482
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18
>>>>>>
>>>>>>> 03:58:06,729
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Angus_McKay,1215890852482 in 0sec
>>>>>> 2008-07-18 03:58:06,729 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18
>>>>>>
>>>>>>> 03:58:06,730
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425 in 0sec
>>>>>> 2008-07-18 03:58:06,730 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18
>>>>>>
>>>>>>> 03:58:06,760
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in0sec
>>>>>> 2008-07-18 03:58:06,760 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18
>>>>>>
>>>>>>> 03:58:06,760
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in0sec
>>>>>> 2008-07-18 03:58:06,761 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18
>>>>>>
>>>>>>> 03:58:06,761
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911 in 0sec
>>>>>> 2008-07-18 03:58:06,761 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18
>>>>>>
>>>>>>> 03:58:06,777
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in0sec
>>>>>> 2008-07-18 03:58:06,777 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18
>>>>>>
>>>>>>> 03:58:06,800
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137 in 0sec
>>>>>> 2008-07-18 03:58:06,800 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18
>>>>>>
>>>>>>> 03:58:06,826
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663in
>>>>>> 0sec
>>>>>> 2008-07-18 03:58:06,826 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18
>>>>>>
>>>>>>> 03:58:06,916
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in0sec
>>>>>> 2008-07-18 03:58:06,916 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://sws.geonames.org/3025251/,1215824259058
>>>>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18
>>>>>>
>>>>>>> 03:58:06,972
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://sws.geonames.org/3025251/,1215824259058 in 0sec
>>>>>> 2008-07-18 03:58:06,973 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18
>>>>>>
>>>>>>> 03:58:06,973
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046in
>>>>>> 0sec
>>>>>> 2008-07-18 03:58:06,973 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18
>>>>>>
>>>>>>> 03:58:15,979
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
>>>>>> hlog. Number of entries: 30009
>>>>>> 2008-07-18 03:59:32,093 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> compaction completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565 in 1mins,
>>>>>> 25sec
>>>>>> 2008-07-18 03:59:32,093 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>>>>> 2008-07-18<
>>>>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18
>>>>>>
>>>>>>> 03:59:32,134
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330 in 0sec
>>>>>> 2008-07-18 03:59:32,134 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18
>>>>>>
>>>>>>> 04:00:45,884
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009 in
>>>>>> 1mins,
>>>>>> 13sec
>>>>>> 2008-07-18 04:00:45,885 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18
>>>>>>
>>>>>>> 04:00:45,887
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943 in 0sec
>>>>>> 2008-07-18 04:00:45,887 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18
>>>>>>
>>>>>>> 04:00:45,892
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in0sec
>>>>>> 2008-07-18 04:00:45,892 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18
>>>>>>
>>>>>>> 04:00:45,893
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in 0sec
>>>>>> 2008-07-18 04:00:45,893 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>>>>> 2008-07-18<
>>>>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18
>>>>>>
>>>>>>> 04:00:45,895
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in 0sec
>>>>>> 2008-07-18 04:00:45,895 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://sws.geonames.org/1081116/,1215823957458
>>>>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18
>>>>>>
>>>>>>> 04:00:45,895
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://sws.geonames.org/1081116/,1215823957458 in 0sec
>>>>>> 2008-07-18 04:00:45,896 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/Allomeric,1216039960663
>>>>>> 2008-07-18 <
>>>>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18
>>>>>>
>>>>>>> 04:00:45,935
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
>>>>>> 2008-07-18 04:00:45,935 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18
>>>>>>
>>>>>>> 04:00:45,993
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in0sec
>>>>>> 2008-07-18 04:00:45,994 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18
>>>>>>
>>>>>>> 04:00:46,004
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332 in
>>>>>> 0sec
>>>>>> 2008-07-18 04:00:46,005 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18
>>>>>>
>>>>>>> 04:00:46,005
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>> completed on region page-repository,
>>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135 in
>>>>>> 0sec
>>>>>> 2008-07-18 04:00:46,005 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> starting compaction on region page-repository,
>>>>>> http://purl.uniprot.org/taxonomy/112557,1214582151856
>>>>>> 2008-07-18<
>>>>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18
>>>>>>
>>>>>>> 04:00:46,007
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>>
>>>>>> completed on region page-repository,
>>>>>> http://purl.uniprot.org/taxonomy/112557,1214582151856 in 0sec
>>>>>> 2008-07-18 04:02:16,798 FATAL
>>>>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>>>>> Replay of hlog required. Forcing server restart
>>>>>> org.apache.hadoop.hbase.DroppedSnapshotException
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>>>>>> at
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
>>>>>> Caused by: org.apache.hadoop.ipc.RemoteException:
>>>>>> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create
>>>>>> file
>>>>>>
>>>>>>
>>>>>> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
>>>>>> for DFSClient_716087201 on client 140.203.154.176 because current
>>>>>> leaseholder is trying to recreate file.
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
>>>>>> at
>>>>>> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
>>>>>> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
>>>>>> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>>
>>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>>>>> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
>>>>>> at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
>>>>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
>>>>>> at
>>>>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
>>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
>>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
>>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
>>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>>>>>> ... 3 more
>>>>>> 2008-07-18 04:02:16,798 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>>>>> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>>>>>> 2008-07-18 04:02:16,807 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 1 on 60020, call batchUpdate([B@167e3c87, row =>
>>>>>> http://dbpedia.org/resource/Jurrette_Sindelar, {column => field:class,
>>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.174:50719: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:16,847 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 9 on 60020, call batchUpdate([B@525cfe10, row =>
>>>>>> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column =>
>>>>>> field:class,
>>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48121: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:16,848 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 0 on 60020, call batchUpdate([B@572bf6ae, row =>
>>>>>> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29,
>>>>>> {column
>>>>>> => field:class, value => '...', column => field:data-source, value =>
>>>>>> '...',
>>>>>> column => field:domain, value => '...', column =>
>>>>>> field:explicit-content,
>>>>>> value => '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48348: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:16,905 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 4 on 60020, call batchUpdate([B@27320a48, row =>
>>>>>> http://dbpedia.org/resource/Karim_Benkhirane, {column => field:class,
>>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48330: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:16,926 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 4 on 60020, call batchUpdate([B@59c7d8c6, row =>
>>>>>> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column =>
>>>>>> field:class,
>>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.176:58334: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:16,959 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 8 on 60020, call batchUpdate([B@4f68dd36, row =>
>>>>>> http://dbpedia.org/resource/Hendrik_Speuy, {column => field:class,
>>>>>> value
>>>>>> => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48357: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:16,979 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 3 on 60020, call batchUpdate([B@596c7f5f, row =>
>>>>>> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column =>
>>>>>> field:class,
>>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.176:58163: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:17,147 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 2 on 60020, call batchUpdate([B@23067cee, row =>
>>>>>> http://dbpedia.org/resource/High_Littleton, {column => field:class,
>>>>>> value
>>>>>> => '...', column => field:data-source, value => '...', column =>
>>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>>> =>
>>>>>> '...', column => field:format, value => '...', column =>
>>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>>> '...', column => field:length, value => '...', column =>
>>>>>> field:ontology,
>>>>>> value => '...', column => field:size, value => '...', column =>
>>>>>> field:timestamp, value => '...'}) from 140.203.154.174:50952: error:
>>>>>> java.io.IOException: Server not running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:17,180 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>>> New
>>>>>> log writer created at
>>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
>>>>>> 2008-07-18 04:02:17,181 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
>>>>>> 2008-07-18 04:02:17,376 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 7 on 60020, call getClosestRowBefore([B@20d09fed, [B@60b3916f)
>>>>>> from 140.203.154.175:48464: error: java.io.IOException: Server not
>>>>>> running
>>>>>> java.io.IOException: Server not running
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
>>>>>> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>>>>>> at
>>>>>>
>>>>>>
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>> at
>>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>> 2008-07-18 04:02:17,507 INFO org.apache.hadoop.ipc.Server: Stopping
>>>>>> server
>>>>>> on 60020
>>>>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 1 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 5 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,587 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 0 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 2 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 9 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 8 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 7 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 6 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 4 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>>> handler 3 on 60020: exiting
>>>>>> 2008-07-18 04:02:17,607 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>>>>> Server listener on 60020
>>>>>> 2008-07-18 04:02:17,691 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
>>>>>> 2008-07-18 04:02:17,784 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>>>>> Server Responder
>>>>>> 2008-07-18 04:02:17,902 INFO org.mortbay.util.ThreadedServer: Stopping
>>>>>> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
>>>>>> 2008-07-18 04:02:18,179 INFO org.mortbay.http.SocketListener: Stopped
>>>>>> SocketListener on 0.0.0.0:60030
>>>>>> 2008-07-18 04:02:28,883 INFO org.mortbay.util.Container: Stopped
>>>>>> HttpContext[/static,/static]
>>>>>> 2008-07-18 04:02:31,116 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread
>>>>>> exiting
>>>>>> 2008-07-18 04:02:31,117 INFO org.mortbay.util.Container: Stopped
>>>>>> HttpContext[/logs,/logs]
>>>>>> 2008-07-18 04:02:31,117 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
>>>>>> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
>>>>>> 2008-07-18 04:02:31,125 INFO org.mortbay.util.Container: Stopped
>>>>>> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
>>>>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>>>>> WebApplicationContext[/,/]
>>>>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>>>>> org.mortbay.jetty.Server@b8c3679
>>>>>> 2008-07-18 04:02:34,309 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: On abort, closed
>>>>>> hlog
>>>>>> 2008-07-18 04:55:59,098 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -3929282808018198437 lease expired
>>>>>> 2008-07-18 04:56:00,258 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> 8694039619106515508 lease expired
>>>>>> 2008-07-18 04:56:02,068 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -6642383936144787497 lease expired
>>>>>> 2008-07-18 04:56:02,178 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -8674274402855984090 lease expired
>>>>>> 2008-07-18 04:56:02,858 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -8684213796880407868 lease expired
>>>>>> 2008-07-18 04:56:03,478 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -8003294377021166340 lease expired
>>>>>> 2008-07-18 04:56:05,298 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> 4741746108028229290 lease expired
>>>>>> 2008-07-18 04:56:05,418 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -3662383214249882020 lease expired
>>>>>> 2008-07-18 04:56:05,498 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>>> -5749397523507975233 lease expired
>>>>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>>>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing leases
>>>>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>>>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed leases
>>>>>> 2008-07-18 04:56:05,499 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> closed .META.,,1
>>>>>> 2008-07-18 04:56:05,499 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>>> closed page-repository,
>>>>>> http://dbpedia.org/resource/CAMBIA,1215970756532
>>>>>> 2008-07-18 <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18
>>>>>>
>>>>>>> 04:56:05,499
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18
>>>>>>
>>>>>>> 04:56:05,499
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18
>>>>>>
>>>>>>> 04:56:05,499
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://sws.geonames.org/1081116/,1215823957458
>>>>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>>>>> 2008-07-18<
>>>>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
>>>>>> 2008-07-18 <http://purl.uniprot.org/go/0009140,12145826028822008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18
>>>>>>
>>>>>>> 04:56:05,500
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://bio2rdf.org/omim:300357,1215825157318
>>>>>> 2008-07-18 <http://bio2rdf.org/omim:300357,12158251573182008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
>>>>>> 2008-07-18 <http://alcidesfonseca.com/blog/35,12158259255952008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
>>>>>> 2008-07-18 <
>>>>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>>>>> 2008-07-18<
>>>>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/John_J._Bernet,1216109320961
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18
>>>>>>
>>>>>>> 04:56:05,501
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://sws.geonames.org/3025251/,1215824259058
>>>>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Billy_Forester,1216165245424
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
>>>>>> 2008-07-18 <
>>>>>> http://dbpedia.org/resource/Snatcho,12162634022752008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Template:superheroe,1216140016580
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18
>>>>>>
>>>>>>> 04:56:05,502
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
>>>>>> 2008-07-18 <
>>>>>> http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
>>>>>> 2008-07-18 <
>>>>>> http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18
>>>>>>
>>>>>>> 04:56:05,503
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
>>>>>> 2008-07-18<
>>>>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Harry_Girvetz,1215910087698
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://acm.rkbexplorer.com/id/person-94180,1215825860040
>>>>>> 2008-07-18<
>>>>>> http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18
>>>>>>
>>>>>>> 04:56:05,504
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>>
>>>>>>
>>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>>>>> 2008-07-18<
>>>>>>
>>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18
>>>>>>
>>>>>>> 04:56:05,505
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18
>>>>>>
>>>>>>> 04:56:05,506
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://sws.geonames.org/2701307/,1215823954578
>>>>>> 2008-07-18 <http://sws.geonames.org/2701307/,12158239545782008-07-18
>>>>>>
>>>>>>> 04:56:05,506
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,
>>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18
>>>>>>
>>>>>>> 04:56:05,506
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>>> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
>>>>>> 2008-07-18<
>>>>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18
>>>>>>
>>>>>>> 04:56:05,506
>>>>>>>
>>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>>>> aborting server at: 140.203.154.176:60020
>>>>>> 2008-07-18 04:56:05,506 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>>>> regionserver/0:0:0:0:0:0:0:0:60020 exiting
>>>>>> 2008-07-18 04:56:05,561 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
>>>>>> thread.
>>>>>> 2008-07-18 04:56:05,562 INFO
>>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread
>>>>>> complete
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>> 2008-07-19 07:31:45,505 WARN org.apache.hadoop.dfs.DFSClient: DataStreamer
>> Exception: java.net.SocketTimeoutException: timed out waiting for rpc
>> response
>> 2008-07-19 07:31:45,506 WARN org.apache.hadoop.dfs.DFSClient: Error
>> Recovery for block null bad datanode[0]
>> 2008-07-19 07:31:54,868 FATAL org.apache.hadoop.hbase.regionserver.Flusher:
>> Replay of hlog required. Forcing server restart
>> org.apache.hadoop.hbase.DroppedSnapshotException
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>> at
>> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>> at org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
>> Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
>> Could not complete write to file
>> /hbase/page-repository/963177038/field/info/4591605661998875281 by
>> DFSClient_1009756630
>> at org.apache.hadoop.dfs.NameNode.complete(NameNode.java:332)
>> at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>
>> at org.apache.hadoop.ipc.Client.call(Client.java:557)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>> at org.apache.hadoop.dfs.$Proxy1.complete(Unknown Source)
>> at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>> at org.apache.hadoop.dfs.$Proxy1.complete(Unknown Source)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2655)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2576)
>> at
>> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:59)
>> at
>> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:79)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile.writeInfo(HStoreFile.java:350)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:747)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>> ... 3 more
>> 2008-07-19 07:31:54,868 INFO org.apache.hadoop.hbase.regionserver.Flusher:
>> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>>
>>
>>
>
>
Re: Replay of hlog required, forcing regionserver restart
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Ok then, at least having the nodes synchronized keeps you safe from other
problems. By the way, how many nodes do you have?
One other thing to check is your writing pattern. If you insert data with
sequential row keys, an example would be designing your row keys as an
incremented primary key equivalent but left padded with zeros, these inserts
will always hit on a single region thus the load will not be distributed. If
it's your case, try randomizing the inserts.
I will also look at your logs monday morning.
Thx,
J-D
On Sat, Jul 19, 2008 at 5:47 AM, Renaud Delbru <re...@deri.org>
wrote:
> Hi Jean-Daniel,
>
> I have synchronised all the nodes to the same ntp server yesterday.
> This morning, another regionserver went down for the same reason (log
> attached).
>
> Thanks.
> --
> Renaud Delbru
>
>
>
> Jean-Daniel Cryans wrote:
>
>> Renaud,
>>
>> Yes, the impact is important. Try fixing it first because typically heavy
>> write load problems only occur when there is a very few number of nodes
>> and
>> regions. We have seen this many times.
>>
>> Happy fixing!
>>
>> J-D
>>
>> On Fri, Jul 18, 2008 at 5:20 PM, Renaud Delbru <re...@deri.org>
>> wrote:
>>
>>
>>
>>> Hi Jean-Daniel,
>>>
>>> my nodes clocks are desynchronised (each node has a different time).
>>> Could
>>> it have an impact on HBase ?
>>>
>>> At the moment of the failure, the node had about 55 regions (with
>>> hbase.hregion.max.filesize = 128M).
>>>
>>> Thanks.
>>> --
>>> Renaud Delbru
>>>
>>> As One node is at
>>>
>>> Jean-Daniel Cryans wrote:
>>>
>>>
>>>
>>>> Renaud,
>>>>
>>>> Just to make sure, are all your nodes clocks synchronized? Also, at the
>>>> moment of that failure, how many regions did the table have?
>>>>
>>>> Thanks,
>>>>
>>>> J-D
>>>>
>>>> On Fri, Jul 18, 2008 at 5:11 AM, Renaud Delbru <re...@deri.org>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Hi,
>>>>>
>>>>> I am using HBase 0.2.0-dev, Hudson Build #208.
>>>>> I am experiencing a problem with HBase that looks like the issue
>>>>> https://issues.apache.org/jira/browse/HBASE-15.
>>>>> During an upload intensive task, one of the regionserver went down to
>>>>> force rerun of hlog.
>>>>>
>>>>> This is not the first time we have experienced this problem, it
>>>>> happened
>>>>> already on another cluster node a few days ago, in the same settings.
>>>>>
>>>>> I have attached the regionserver log. I don't notice something
>>>>> particular in the hadoop datanode log.
>>>>>
>>>>> Thanks in advance for any advices.
>>>>> --
>>>>> Renaud Delbru
>>>>>
>>>>>
>>>>>
>>>>> 2008-07-18 03:40:51,861 INFO
>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>>> entries: 30008
>>>>> 2008-07-18 03:40:52,073 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>> New
>>>>> log writer created at
>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
>>>>> 2008-07-18 03:43:36,257 INFO
>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>>> entries: 30008
>>>>> 2008-07-18 03:43:37,020 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>> New
>>>>> log writer created at
>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
>>>>> 2008-07-18 03:45:12,963 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18
>>>>> >03:45:58,868
>>>>> WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
>>>>>
>>>>> ten times longer than scheduled: 3000
>>>>> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We
>>>>> slept
>>>>> 583701ms, ten times longer than scheduled: 3000
>>>>> 2008-07-18 03:55:57,787 INFO
>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>>> entries: 21011
>>>>> 2008-07-18 03:55:58,410 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>>>> while reading from blk_-6424749704096696952 of
>>>>>
>>>>> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
>>>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>>>> inputStream
>>>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>>>> at
>>>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>>>> at
>>>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>>>> at
>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>>>> at
>>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
>>>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>>>> at
>>>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>>>
>>>>> 2008-07-18 03:55:58,966 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>>>> while reading from blk_7305122468419183134 of
>>>>>
>>>>> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
>>>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>>>> inputStream
>>>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>>>> at
>>>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>>>> at
>>>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>>>> at
>>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>>>> at
>>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
>>>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>>>> at
>>>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>>>
>>>>> 2008-07-18 03:56:05,305 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>> New
>>>>> log writer created at
>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
>>>>> 2008-07-18 03:57:30,244 ERROR
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening
>>>>> scanner
>>>>> (fsOk: true)
>>>>> java.io.IOException: HStoreScanner failed construction
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>>> response
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>>>> ... 10 more
>>>>> 2008-07-18 03:58:01,895 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 7 on 60020, call openScanner([B@9dd0d92, [[B@fa5f14d,
>>>>> [B@5784fe41,
>>>>> 9223372036854775807, null) from 140.203.154.174:48958: error:
>>>>> java.io.IOException: HStoreScanner failed construction
>>>>> java.io.IOException: HStoreScanner failed construction
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>>> response
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>>>> ... 10 more
>>>>> 2008-07-18 03:58:06,728 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> compaction completed on region page-repository,
>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401 in 12mins,
>>>>> 53sec
>>>>> 2008-07-18 03:58:06,728 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Angus_McKay,1215890852482
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18
>>>>> >03:58:06,729
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Angus_McKay,1215890852482 in 0sec
>>>>> 2008-07-18 03:58:06,729 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18
>>>>> >03:58:06,730
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425 in 0sec
>>>>> 2008-07-18 03:58:06,730 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>>
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>>>> 2008-07-18<
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18
>>>>> >03:58:06,760
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in0sec
>>>>> 2008-07-18 03:58:06,760 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>>
>>>>>
>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>>>> 2008-07-18<
>>>>>
>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18
>>>>> >03:58:06,760
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>>
>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in0sec
>>>>> 2008-07-18 03:58:06,761 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18
>>>>> >03:58:06,761
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911 in 0sec
>>>>> 2008-07-18 03:58:06,761 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>>
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>>>> 2008-07-18<
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18
>>>>> >03:58:06,777
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in0sec
>>>>> 2008-07-18 03:58:06,777 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18
>>>>> >03:58:06,800
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137 in 0sec
>>>>> 2008-07-18 03:58:06,800 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18
>>>>> >03:58:06,826
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663in
>>>>> 0sec
>>>>> 2008-07-18 03:58:06,826 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>>
>>>>>
>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>>>> 2008-07-18<
>>>>>
>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18
>>>>> >03:58:06,916
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>>
>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in0sec
>>>>> 2008-07-18 03:58:06,916 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://sws.geonames.org/3025251/,1215824259058
>>>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18
>>>>> >03:58:06,972
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://sws.geonames.org/3025251/,1215824259058 in 0sec
>>>>> 2008-07-18 03:58:06,973 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18
>>>>> >03:58:06,973
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046in
>>>>> 0sec
>>>>> 2008-07-18 03:58:06,973 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18
>>>>> >03:58:15,979
>>>>> INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
>>>>> hlog. Number of entries: 30009
>>>>> 2008-07-18 03:59:32,093 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> compaction completed on region page-repository,
>>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565 in 1mins,
>>>>> 25sec
>>>>> 2008-07-18 03:59:32,093 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>>>> 2008-07-18<
>>>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18
>>>>> >03:59:32,134
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330 in 0sec
>>>>> 2008-07-18 03:59:32,134 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18
>>>>> >04:00:45,884
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009 in
>>>>> 1mins,
>>>>> 13sec
>>>>> 2008-07-18 04:00:45,885 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18
>>>>> >04:00:45,887
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943 in 0sec
>>>>> 2008-07-18 04:00:45,887 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18
>>>>> >04:00:45,892
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in0sec
>>>>> 2008-07-18 04:00:45,892 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18
>>>>> >04:00:45,893
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in 0sec
>>>>> 2008-07-18 04:00:45,893 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>>>> 2008-07-18<
>>>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18
>>>>> >04:00:45,895
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in 0sec
>>>>> 2008-07-18 04:00:45,895 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://sws.geonames.org/1081116/,1215823957458
>>>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18
>>>>> >04:00:45,895
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://sws.geonames.org/1081116/,1215823957458 in 0sec
>>>>> 2008-07-18 04:00:45,896 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/Allomeric,1216039960663
>>>>> 2008-07-18 <
>>>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18
>>>>> >04:00:45,935
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
>>>>> 2008-07-18 04:00:45,935 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18
>>>>> >04:00:45,993
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in0sec
>>>>> 2008-07-18 04:00:45,994 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>>>> 2008-07-18<
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18
>>>>> >04:00:46,004
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332 in
>>>>> 0sec
>>>>> 2008-07-18 04:00:46,005 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18
>>>>> >04:00:46,005
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>> completed on region page-repository,
>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135 in
>>>>> 0sec
>>>>> 2008-07-18 04:00:46,005 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> starting compaction on region page-repository,
>>>>> http://purl.uniprot.org/taxonomy/112557,1214582151856
>>>>> 2008-07-18<
>>>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18
>>>>> >04:00:46,007
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>>
>>>>> completed on region page-repository,
>>>>> http://purl.uniprot.org/taxonomy/112557,1214582151856 in 0sec
>>>>> 2008-07-18 04:02:16,798 FATAL
>>>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>>>> Replay of hlog required. Forcing server restart
>>>>> org.apache.hadoop.hbase.DroppedSnapshotException
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>>>>> at
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>>>>> at
>>>>> org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
>>>>> Caused by: org.apache.hadoop.ipc.RemoteException:
>>>>> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create
>>>>> file
>>>>>
>>>>>
>>>>> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
>>>>> for DFSClient_716087201 on client 140.203.154.176 because current
>>>>> leaseholder is trying to recreate file.
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
>>>>> at
>>>>> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
>>>>> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
>>>>> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>>
>>>>> at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>>>> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
>>>>> at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
>>>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
>>>>> at
>>>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
>>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
>>>>> at
>>>>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>>>>> ... 3 more
>>>>> 2008-07-18 04:02:16,798 INFO
>>>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>>>> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>>>>> 2008-07-18 04:02:16,807 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 1 on 60020, call batchUpdate([B@167e3c87, row =>
>>>>> http://dbpedia.org/resource/Jurrette_Sindelar, {column => field:class,
>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.174:50719: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:16,847 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 9 on 60020, call batchUpdate([B@525cfe10, row =>
>>>>> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column =>
>>>>> field:class,
>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48121: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:16,848 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 0 on 60020, call batchUpdate([B@572bf6ae, row =>
>>>>> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29,
>>>>> {column
>>>>> => field:class, value => '...', column => field:data-source, value =>
>>>>> '...',
>>>>> column => field:domain, value => '...', column =>
>>>>> field:explicit-content,
>>>>> value => '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48348: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:16,905 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 4 on 60020, call batchUpdate([B@27320a48, row =>
>>>>> http://dbpedia.org/resource/Karim_Benkhirane, {column => field:class,
>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48330: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:16,926 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 4 on 60020, call batchUpdate([B@59c7d8c6, row =>
>>>>> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column =>
>>>>> field:class,
>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.176:58334: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:16,959 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 8 on 60020, call batchUpdate([B@4f68dd36, row =>
>>>>> http://dbpedia.org/resource/Hendrik_Speuy, {column => field:class,
>>>>> value
>>>>> => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.175:48357: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:16,979 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 3 on 60020, call batchUpdate([B@596c7f5f, row =>
>>>>> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column =>
>>>>> field:class,
>>>>> value => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.176:58163: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:17,147 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 2 on 60020, call batchUpdate([B@23067cee, row =>
>>>>> http://dbpedia.org/resource/High_Littleton, {column => field:class,
>>>>> value
>>>>> => '...', column => field:data-source, value => '...', column =>
>>>>> field:domain, value => '...', column => field:explicit-content, value
>>>>> =>
>>>>> '...', column => field:format, value => '...', column =>
>>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>>> '...', column => field:length, value => '...', column =>
>>>>> field:ontology,
>>>>> value => '...', column => field:size, value => '...', column =>
>>>>> field:timestamp, value => '...'}) from 140.203.154.174:50952: error:
>>>>> java.io.IOException: Server not running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:17,180 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>>> New
>>>>> log writer created at
>>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
>>>>> 2008-07-18 04:02:17,181 INFO
>>>>> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
>>>>> 2008-07-18 04:02:17,376 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 7 on 60020, call getClosestRowBefore([B@20d09fed, [B@60b3916f)
>>>>> from 140.203.154.175:48464: error: java.io.IOException: Server not
>>>>> running
>>>>> java.io.IOException: Server not running
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>>> at
>>>>>
>>>>>
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
>>>>> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>>>>> at
>>>>>
>>>>>
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>>> at
>>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>> 2008-07-18 04:02:17,507 INFO org.apache.hadoop.ipc.Server: Stopping
>>>>> server
>>>>> on 60020
>>>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 1 on 60020: exiting
>>>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 5 on 60020: exiting
>>>>> 2008-07-18 04:02:17,587 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 0 on 60020: exiting
>>>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 2 on 60020: exiting
>>>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 9 on 60020: exiting
>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 8 on 60020: exiting
>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 7 on 60020: exiting
>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 6 on 60020: exiting
>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 4 on 60020: exiting
>>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>>> handler 3 on 60020: exiting
>>>>> 2008-07-18 04:02:17,607 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>>>> Server listener on 60020
>>>>> 2008-07-18 04:02:17,691 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
>>>>> 2008-07-18 04:02:17,784 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>>>> Server Responder
>>>>> 2008-07-18 04:02:17,902 INFO org.mortbay.util.ThreadedServer: Stopping
>>>>> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
>>>>> 2008-07-18 04:02:18,179 INFO org.mortbay.http.SocketListener: Stopped
>>>>> SocketListener on 0.0.0.0:60030
>>>>> 2008-07-18 04:02:28,883 INFO org.mortbay.util.Container: Stopped
>>>>> HttpContext[/static,/static]
>>>>> 2008-07-18 04:02:31,116 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread
>>>>> exiting
>>>>> 2008-07-18 04:02:31,117 INFO org.mortbay.util.Container: Stopped
>>>>> HttpContext[/logs,/logs]
>>>>> 2008-07-18 04:02:31,117 INFO
>>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
>>>>> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
>>>>> 2008-07-18 04:02:31,125 INFO org.mortbay.util.Container: Stopped
>>>>> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
>>>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>>>> WebApplicationContext[/,/]
>>>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>>>> org.mortbay.jetty.Server@b8c3679
>>>>> 2008-07-18 04:02:34,309 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: On abort, closed
>>>>> hlog
>>>>> 2008-07-18 04:55:59,098 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -3929282808018198437 lease expired
>>>>> 2008-07-18 04:56:00,258 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> 8694039619106515508 lease expired
>>>>> 2008-07-18 04:56:02,068 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -6642383936144787497 lease expired
>>>>> 2008-07-18 04:56:02,178 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -8674274402855984090 lease expired
>>>>> 2008-07-18 04:56:02,858 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -8684213796880407868 lease expired
>>>>> 2008-07-18 04:56:03,478 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -8003294377021166340 lease expired
>>>>> 2008-07-18 04:56:05,298 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> 4741746108028229290 lease expired
>>>>> 2008-07-18 04:56:05,418 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -3662383214249882020 lease expired
>>>>> 2008-07-18 04:56:05,498 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>>> -5749397523507975233 lease expired
>>>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing leases
>>>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed leases
>>>>> 2008-07-18 04:56:05,499 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> closed .META.,,1
>>>>> 2008-07-18 04:56:05,499 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>>> closed page-repository,
>>>>> http://dbpedia.org/resource/CAMBIA,1215970756532
>>>>> 2008-07-18 <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18
>>>>> >04:56:05,499
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18
>>>>> >04:56:05,499
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
>>>>> 2008-07-18<
>>>>>
>>>>> http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18
>>>>> >04:56:05,499
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://sws.geonames.org/1081116/,1215823957458
>>>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>>>> 2008-07-18<
>>>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
>>>>> 2008-07-18 <http://purl.uniprot.org/go/0009140,12145826028822008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>>>> 2008-07-18<
>>>>>
>>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18
>>>>> >04:56:05,500
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://bio2rdf.org/omim:300357,1215825157318
>>>>> 2008-07-18 <http://bio2rdf.org/omim:300357,12158251573182008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
>>>>> 2008-07-18 <http://alcidesfonseca.com/blog/35,12158259255952008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
>>>>> 2008-07-18 <
>>>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>>>> 2008-07-18<
>>>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/John_J._Bernet,1216109320961
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18
>>>>> >04:56:05,501
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://sws.geonames.org/3025251/,1215824259058
>>>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Billy_Forester,1216165245424
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
>>>>> 2008-07-18 <
>>>>> http://dbpedia.org/resource/Snatcho,12162634022752008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Template:superheroe,1216140016580
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>>>> 2008-07-18<
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>>>> 2008-07-18<
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18
>>>>> >04:56:05,502
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>>>> 2008-07-18<
>>>>>
>>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
>>>>> 2008-07-18 <
>>>>> http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
>>>>> 2008-07-18 <
>>>>> http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18
>>>>> >04:56:05,503
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
>>>>> 2008-07-18<
>>>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Harry_Girvetz,1215910087698
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
>>>>> 2008-07-18<
>>>>>
>>>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://acm.rkbexplorer.com/id/person-94180,1215825860040
>>>>> 2008-07-18<
>>>>> http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
>>>>> 2008-07-18<
>>>>>
>>>>> http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18
>>>>> >04:56:05,504
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>>>> 2008-07-18<
>>>>>
>>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>>
>>>>>
>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>>>> 2008-07-18<
>>>>>
>>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18
>>>>> >04:56:05,505
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18
>>>>> >04:56:05,506
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://sws.geonames.org/2701307/,1215823954578
>>>>> 2008-07-18 <http://sws.geonames.org/2701307/,12158239545782008-07-18
>>>>> >04:56:05,506
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,
>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18
>>>>> >04:56:05,506
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>>> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
>>>>> 2008-07-18<
>>>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18
>>>>> >04:56:05,506
>>>>> INFO org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>>> aborting server at: 140.203.154.176:60020
>>>>> 2008-07-18 04:56:05,506 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>>> regionserver/0:0:0:0:0:0:0:0:60020 exiting
>>>>> 2008-07-18 04:56:05,561 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
>>>>> thread.
>>>>> 2008-07-18 04:56:05,562 INFO
>>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread
>>>>> complete
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>
>
> 2008-07-19 07:31:45,505 WARN org.apache.hadoop.dfs.DFSClient: DataStreamer
> Exception: java.net.SocketTimeoutException: timed out waiting for rpc
> response
> 2008-07-19 07:31:45,506 WARN org.apache.hadoop.dfs.DFSClient: Error
> Recovery for block null bad datanode[0]
> 2008-07-19 07:31:54,868 FATAL org.apache.hadoop.hbase.regionserver.Flusher:
> Replay of hlog required. Forcing server restart
> org.apache.hadoop.hbase.DroppedSnapshotException
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
> at
> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
> at org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
> Caused by: org.apache.hadoop.ipc.RemoteException: java.io.IOException:
> Could not complete write to file
> /hbase/page-repository/963177038/field/info/4591605661998875281 by
> DFSClient_1009756630
> at org.apache.hadoop.dfs.NameNode.complete(NameNode.java:332)
> at sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:557)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> at org.apache.hadoop.dfs.$Proxy1.complete(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy1.complete(Unknown Source)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:2655)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:2576)
> at
> org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:59)
> at
> org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:79)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile.writeInfo(HStoreFile.java:350)
> at
> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:747)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
> ... 3 more
> 2008-07-19 07:31:54,868 INFO org.apache.hadoop.hbase.regionserver.Flusher:
> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>
>
Re: Replay of hlog required, forcing regionserver restart
Posted by Renaud Delbru <re...@deri.org>.
Hi Jean-Daniel,
I have synchronised all the nodes to the same ntp server yesterday.
This morning, another regionserver went down for the same reason (log
attached).
Thanks.
--
Renaud Delbru
Jean-Daniel Cryans wrote:
> Renaud,
>
> Yes, the impact is important. Try fixing it first because typically heavy
> write load problems only occur when there is a very few number of nodes and
> regions. We have seen this many times.
>
> Happy fixing!
>
> J-D
>
> On Fri, Jul 18, 2008 at 5:20 PM, Renaud Delbru <re...@deri.org>
> wrote:
>
>
>> Hi Jean-Daniel,
>>
>> my nodes clocks are desynchronised (each node has a different time). Could
>> it have an impact on HBase ?
>>
>> At the moment of the failure, the node had about 55 regions (with
>> hbase.hregion.max.filesize = 128M).
>>
>> Thanks.
>> --
>> Renaud Delbru
>>
>> As One node is at
>>
>> Jean-Daniel Cryans wrote:
>>
>>
>>> Renaud,
>>>
>>> Just to make sure, are all your nodes clocks synchronized? Also, at the
>>> moment of that failure, how many regions did the table have?
>>>
>>> Thanks,
>>>
>>> J-D
>>>
>>> On Fri, Jul 18, 2008 at 5:11 AM, Renaud Delbru <re...@deri.org>
>>> wrote:
>>>
>>>
>>>
>>>
>>>> Hi,
>>>>
>>>> I am using HBase 0.2.0-dev, Hudson Build #208.
>>>> I am experiencing a problem with HBase that looks like the issue
>>>> https://issues.apache.org/jira/browse/HBASE-15.
>>>> During an upload intensive task, one of the regionserver went down to
>>>> force rerun of hlog.
>>>>
>>>> This is not the first time we have experienced this problem, it happened
>>>> already on another cluster node a few days ago, in the same settings.
>>>>
>>>> I have attached the regionserver log. I don't notice something
>>>> particular in the hadoop datanode log.
>>>>
>>>> Thanks in advance for any advices.
>>>> --
>>>> Renaud Delbru
>>>>
>>>>
>>>>
>>>> 2008-07-18 03:40:51,861 INFO
>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>> entries: 30008
>>>> 2008-07-18 03:40:52,073 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>> New
>>>> log writer created at
>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
>>>> 2008-07-18 03:43:36,257 INFO
>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>> entries: 30008
>>>> 2008-07-18 03:43:37,020 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>> New
>>>> log writer created at
>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
>>>> 2008-07-18 03:45:12,963 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>03:45:58,868
>>>> WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
>>>>
>>>> ten times longer than scheduled: 3000
>>>> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We
>>>> slept
>>>> 583701ms, ten times longer than scheduled: 3000
>>>> 2008-07-18 03:55:57,787 INFO
>>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>>> entries: 21011
>>>> 2008-07-18 03:55:58,410 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>>> while reading from blk_-6424749704096696952 of
>>>> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
>>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>>> inputStream
>>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>>> at
>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>>> at
>>>>
>>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>>> at
>>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>>> at
>>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>>> at
>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>>> at
>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>>> at
>>>>
>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>>> at
>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>>> at
>>>>
>>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>>> at
>>>>
>>>> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
>>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>>
>>>> 2008-07-18 03:55:58,966 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>>> while reading from blk_7305122468419183134 of
>>>> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
>>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>>> inputStream
>>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>>> at
>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>>> at
>>>>
>>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>>> at
>>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>>> at
>>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>>> at
>>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>>> at
>>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>>> at
>>>>
>>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>>> at
>>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>>> at
>>>>
>>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>>> at
>>>>
>>>> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
>>>> at
>>>>
>>>> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
>>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>>
>>>> 2008-07-18 03:56:05,305 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>> New
>>>> log writer created at
>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
>>>> 2008-07-18 03:57:30,244 ERROR
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening scanner
>>>> (fsOk: true)
>>>> java.io.IOException: HStoreScanner failed construction
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>> response
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>>> at
>>>>
>>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>>> ... 10 more
>>>> 2008-07-18 03:58:01,895 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 7 on 60020, call openScanner([B@9dd0d92, [[B@fa5f14d,
>>>> [B@5784fe41,
>>>> 9223372036854775807, null) from 140.203.154.174:48958: error:
>>>> java.io.IOException: HStoreScanner failed construction
>>>> java.io.IOException: HStoreScanner failed construction
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>> response
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>>> at
>>>>
>>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>>> ... 10 more
>>>> 2008-07-18 03:58:06,728 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> compaction completed on region page-repository,
>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401 in 12mins,
>>>> 53sec
>>>> 2008-07-18 03:58:06,728 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Angus_McKay,1215890852482
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>03:58:06,729
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Angus_McKay,1215890852482 in 0sec
>>>> 2008-07-18 03:58:06,729 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>03:58:06,730
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425 in 0sec
>>>> 2008-07-18 03:58:06,730 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>>
>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>>> 2008-07-18<
>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>03:58:06,760
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>>
>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in0sec
>>>> 2008-07-18 03:58:06,760 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>>
>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>>> 2008-07-18<
>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>03:58:06,760
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>>
>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in0sec
>>>> 2008-07-18 03:58:06,761 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>03:58:06,761
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911 in 0sec
>>>> 2008-07-18 03:58:06,761 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>>
>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>>> 2008-07-18<
>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>03:58:06,777
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>>
>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in0sec
>>>> 2008-07-18 03:58:06,777 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>03:58:06,800
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137 in 0sec
>>>> 2008-07-18 03:58:06,800 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>03:58:06,826
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663in
>>>> 0sec
>>>> 2008-07-18 03:58:06,826 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>>
>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>>> 2008-07-18<
>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>03:58:06,916
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>>
>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in0sec
>>>> 2008-07-18 03:58:06,916 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://sws.geonames.org/3025251/,1215824259058
>>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>03:58:06,972
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://sws.geonames.org/3025251/,1215824259058 in 0sec
>>>> 2008-07-18 03:58:06,973 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>03:58:06,973
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046in
>>>> 0sec
>>>> 2008-07-18 03:58:06,973 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>03:58:15,979
>>>> INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
>>>> hlog. Number of entries: 30009
>>>> 2008-07-18 03:59:32,093 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> compaction completed on region page-repository,
>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565 in 1mins,
>>>> 25sec
>>>> 2008-07-18 03:59:32,093 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>>> 2008-07-18<
>>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>03:59:32,134
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330 in 0sec
>>>> 2008-07-18 03:59:32,134 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:00:45,884
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009 in
>>>> 1mins,
>>>> 13sec
>>>> 2008-07-18 04:00:45,885 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:00:45,887
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943 in 0sec
>>>> 2008-07-18 04:00:45,887 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>>
>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:00:45,892
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>>
>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in0sec
>>>> 2008-07-18 04:00:45,892 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:00:45,893
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in 0sec
>>>> 2008-07-18 04:00:45,893 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>>> 2008-07-18<
>>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:00:45,895
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in 0sec
>>>> 2008-07-18 04:00:45,895 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://sws.geonames.org/1081116/,1215823957458
>>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:00:45,895
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://sws.geonames.org/1081116/,1215823957458 in 0sec
>>>> 2008-07-18 04:00:45,896 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/Allomeric,1216039960663
>>>> 2008-07-18 <
>>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:00:45,935
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
>>>> 2008-07-18 04:00:45,935 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>>
>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:00:45,993
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>>
>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in0sec
>>>> 2008-07-18 04:00:45,994 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>>> 2008-07-18<
>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:00:46,004
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332 in
>>>> 0sec
>>>> 2008-07-18 04:00:46,005 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:00:46,005
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>> completed on region page-repository,
>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135 in 0sec
>>>> 2008-07-18 04:00:46,005 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> starting compaction on region page-repository,
>>>> http://purl.uniprot.org/taxonomy/112557,1214582151856
>>>> 2008-07-18<
>>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:00:46,007
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>>
>>>> completed on region page-repository,
>>>> http://purl.uniprot.org/taxonomy/112557,1214582151856 in 0sec
>>>> 2008-07-18 04:02:16,798 FATAL
>>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>>> Replay of hlog required. Forcing server restart
>>>> org.apache.hadoop.hbase.DroppedSnapshotException
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
>>>> Caused by: org.apache.hadoop.ipc.RemoteException:
>>>> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file
>>>>
>>>> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
>>>> for DFSClient_716087201 on client 140.203.154.176 because current
>>>> leaseholder is trying to recreate file.
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
>>>> at
>>>> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
>>>> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
>>>> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>>
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>>> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>>> at
>>>>
>>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
>>>> at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
>>>> at
>>>>
>>>> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
>>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
>>>> at
>>>>
>>>> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
>>>> at
>>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
>>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
>>>> at
>>>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>>>> ... 3 more
>>>> 2008-07-18 04:02:16,798 INFO
>>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>>> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>>>> 2008-07-18 04:02:16,807 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 1 on 60020, call batchUpdate([B@167e3c87, row =>
>>>> http://dbpedia.org/resource/Jurrette_Sindelar, {column => field:class,
>>>> value => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.174:50719: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:16,847 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 9 on 60020, call batchUpdate([B@525cfe10, row =>
>>>> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column => field:class,
>>>> value => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.175:48121: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:16,848 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 0 on 60020, call batchUpdate([B@572bf6ae, row =>
>>>> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29, {column
>>>> => field:class, value => '...', column => field:data-source, value =>
>>>> '...',
>>>> column => field:domain, value => '...', column => field:explicit-content,
>>>> value => '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.175:48348: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:16,905 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 4 on 60020, call batchUpdate([B@27320a48, row =>
>>>> http://dbpedia.org/resource/Karim_Benkhirane, {column => field:class,
>>>> value => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.175:48330: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:16,926 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 4 on 60020, call batchUpdate([B@59c7d8c6, row =>
>>>> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column =>
>>>> field:class,
>>>> value => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.176:58334: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:16,959 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 8 on 60020, call batchUpdate([B@4f68dd36, row =>
>>>> http://dbpedia.org/resource/Hendrik_Speuy, {column => field:class, value
>>>> => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.175:48357: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:16,979 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 3 on 60020, call batchUpdate([B@596c7f5f, row =>
>>>> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column =>
>>>> field:class,
>>>> value => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.176:58163: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:17,147 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 2 on 60020, call batchUpdate([B@23067cee, row =>
>>>> http://dbpedia.org/resource/High_Littleton, {column => field:class,
>>>> value
>>>> => '...', column => field:data-source, value => '...', column =>
>>>> field:domain, value => '...', column => field:explicit-content, value =>
>>>> '...', column => field:format, value => '...', column =>
>>>> field:implicit-content, value => '...', column => field:label, value =>
>>>> '...', column => field:length, value => '...', column => field:ontology,
>>>> value => '...', column => field:size, value => '...', column =>
>>>> field:timestamp, value => '...'}) from 140.203.154.174:50952: error:
>>>> java.io.IOException: Server not running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:17,180 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>>> New
>>>> log writer created at
>>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
>>>> 2008-07-18 04:02:17,181 INFO
>>>> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
>>>> 2008-07-18 04:02:17,376 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 7 on 60020, call getClosestRowBefore([B@20d09fed, [B@60b3916f)
>>>> from 140.203.154.175:48464: error: java.io.IOException: Server not
>>>> running
>>>> java.io.IOException: Server not running
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>>> at
>>>>
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
>>>> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>>>> at
>>>>
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>> 2008-07-18 04:02:17,507 INFO org.apache.hadoop.ipc.Server: Stopping
>>>> server
>>>> on 60020
>>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 1 on 60020: exiting
>>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 5 on 60020: exiting
>>>> 2008-07-18 04:02:17,587 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 0 on 60020: exiting
>>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 2 on 60020: exiting
>>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 9 on 60020: exiting
>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 8 on 60020: exiting
>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 7 on 60020: exiting
>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 6 on 60020: exiting
>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 4 on 60020: exiting
>>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>>> handler 3 on 60020: exiting
>>>> 2008-07-18 04:02:17,607 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>>> Server listener on 60020
>>>> 2008-07-18 04:02:17,691 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
>>>> 2008-07-18 04:02:17,784 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>>> Server Responder
>>>> 2008-07-18 04:02:17,902 INFO org.mortbay.util.ThreadedServer: Stopping
>>>> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
>>>> 2008-07-18 04:02:18,179 INFO org.mortbay.http.SocketListener: Stopped
>>>> SocketListener on 0.0.0.0:60030
>>>> 2008-07-18 04:02:28,883 INFO org.mortbay.util.Container: Stopped
>>>> HttpContext[/static,/static]
>>>> 2008-07-18 04:02:31,116 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread exiting
>>>> 2008-07-18 04:02:31,117 INFO org.mortbay.util.Container: Stopped
>>>> HttpContext[/logs,/logs]
>>>> 2008-07-18 04:02:31,117 INFO
>>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
>>>> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
>>>> 2008-07-18 04:02:31,125 INFO org.mortbay.util.Container: Stopped
>>>> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
>>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>>> WebApplicationContext[/,/]
>>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>>> org.mortbay.jetty.Server@b8c3679
>>>> 2008-07-18 04:02:34,309 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: On abort, closed hlog
>>>> 2008-07-18 04:55:59,098 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -3929282808018198437 lease expired
>>>> 2008-07-18 04:56:00,258 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> 8694039619106515508 lease expired
>>>> 2008-07-18 04:56:02,068 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -6642383936144787497 lease expired
>>>> 2008-07-18 04:56:02,178 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -8674274402855984090 lease expired
>>>> 2008-07-18 04:56:02,858 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -8684213796880407868 lease expired
>>>> 2008-07-18 04:56:03,478 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -8003294377021166340 lease expired
>>>> 2008-07-18 04:56:05,298 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> 4741746108028229290 lease expired
>>>> 2008-07-18 04:56:05,418 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -3662383214249882020 lease expired
>>>> 2008-07-18 04:56:05,498 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>>> -5749397523507975233 lease expired
>>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing leases
>>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed leases
>>>> 2008-07-18 04:56:05,499 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> closed .META.,,1
>>>> 2008-07-18 04:56:05,499 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>>> closed page-repository,http://dbpedia.org/resource/CAMBIA,1215970756532
>>>> 2008-07-18 <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18>04:56:05,499
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:56:05,499
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
>>>> 2008-07-18<
>>>> http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18>04:56:05,499
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://sws.geonames.org/1081116/,1215823957458
>>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>>> 2008-07-18<
>>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
>>>> 2008-07-18 <http://purl.uniprot.org/go/0009140,12145826028822008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>>> 2008-07-18<
>>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>04:56:05,500
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://bio2rdf.org/omim:300357,1215825157318
>>>> 2008-07-18 <http://bio2rdf.org/omim:300357,12158251573182008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
>>>> 2008-07-18 <http://alcidesfonseca.com/blog/35,12158259255952008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
>>>> 2008-07-18 <
>>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>>> 2008-07-18<
>>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/John_J._Bernet,1216109320961
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>04:56:05,501
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://sws.geonames.org/3025251/,1215824259058
>>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Billy_Forester,1216165245424
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
>>>> 2008-07-18 <http://dbpedia.org/resource/Snatcho,12162634022752008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Template:superheroe,1216140016580
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>>> 2008-07-18<
>>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>>> 2008-07-18<
>>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>04:56:05,502
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>>> 2008-07-18<
>>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
>>>> 2008-07-18 <
>>>> http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
>>>> 2008-07-18 <http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18>04:56:05,503
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
>>>> 2008-07-18<
>>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Harry_Girvetz,1215910087698
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
>>>> 2008-07-18<
>>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://acm.rkbexplorer.com/id/person-94180,1215825860040
>>>> 2008-07-18<
>>>> http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
>>>> 2008-07-18<
>>>> http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18>04:56:05,504
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>>> 2008-07-18<
>>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>>
>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:56:05,505
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:56:05,506
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://sws.geonames.org/2701307/,1215823954578
>>>> 2008-07-18 <http://sws.geonames.org/2701307/,12158239545782008-07-18>04:56:05,506
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,
>>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>04:56:05,506
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>>> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
>>>> 2008-07-18<
>>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>04:56:05,506
>>>> INFO org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>> aborting server at: 140.203.154.176:60020
>>>> 2008-07-18 04:56:05,506 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>>> regionserver/0:0:0:0:0:0:0:0:60020 exiting
>>>> 2008-07-18 04:56:05,561 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
>>>> thread.
>>>> 2008-07-18 04:56:05,562 INFO
>>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread
>>>> complete
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>
>
Re: Replay of hlog required, forcing regionserver restart
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Renaud,
Yes, the impact is important. Try fixing it first because typically heavy
write load problems only occur when there is a very few number of nodes and
regions. We have seen this many times.
Happy fixing!
J-D
On Fri, Jul 18, 2008 at 5:20 PM, Renaud Delbru <re...@deri.org>
wrote:
> Hi Jean-Daniel,
>
> my nodes clocks are desynchronised (each node has a different time). Could
> it have an impact on HBase ?
>
> At the moment of the failure, the node had about 55 regions (with
> hbase.hregion.max.filesize = 128M).
>
> Thanks.
> --
> Renaud Delbru
>
> As One node is at
>
> Jean-Daniel Cryans wrote:
>
>> Renaud,
>>
>> Just to make sure, are all your nodes clocks synchronized? Also, at the
>> moment of that failure, how many regions did the table have?
>>
>> Thanks,
>>
>> J-D
>>
>> On Fri, Jul 18, 2008 at 5:11 AM, Renaud Delbru <re...@deri.org>
>> wrote:
>>
>>
>>
>>> Hi,
>>>
>>> I am using HBase 0.2.0-dev, Hudson Build #208.
>>> I am experiencing a problem with HBase that looks like the issue
>>> https://issues.apache.org/jira/browse/HBASE-15.
>>> During an upload intensive task, one of the regionserver went down to
>>> force rerun of hlog.
>>>
>>> This is not the first time we have experienced this problem, it happened
>>> already on another cluster node a few days ago, in the same settings.
>>>
>>> I have attached the regionserver log. I don't notice something
>>> particular in the hadoop datanode log.
>>>
>>> Thanks in advance for any advices.
>>> --
>>> Renaud Delbru
>>>
>>>
>>>
>>> 2008-07-18 03:40:51,861 INFO
>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>> entries: 30008
>>> 2008-07-18 03:40:52,073 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>> New
>>> log writer created at
>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
>>> 2008-07-18 03:43:36,257 INFO
>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>> entries: 30008
>>> 2008-07-18 03:43:37,020 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>> New
>>> log writer created at
>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
>>> 2008-07-18 03:45:12,963 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>03:45:58,868
>>> WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
>>>
>>> ten times longer than scheduled: 3000
>>> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We
>>> slept
>>> 583701ms, ten times longer than scheduled: 3000
>>> 2008-07-18 03:55:57,787 INFO
>>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>>> entries: 21011
>>> 2008-07-18 03:55:58,410 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>> while reading from blk_-6424749704096696952 of
>>> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>> inputStream
>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>> at
>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>> at
>>>
>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>> at
>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>> at
>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>> at
>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>> at
>>>
>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>> at
>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>> at
>>>
>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>> at
>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>> at
>>>
>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>> at
>>>
>>> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>> at
>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>
>>> 2008-07-18 03:55:58,966 WARN org.apache.hadoop.dfs.DFSClient: Exception
>>> while reading from blk_7305122468419183134 of
>>> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
>>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>>> inputStream
>>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>>> at
>>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>>> at
>>>
>>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>>> at
>>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>>> at
>>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>>> at
>>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>>> at
>>>
>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>>> at
>>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>>> at
>>>
>>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>>> at
>>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>>> at
>>>
>>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>>> at
>>>
>>> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
>>> at
>>>
>>> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
>>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>>> at
>>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>>
>>> 2008-07-18 03:56:05,305 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>> New
>>> log writer created at
>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
>>> 2008-07-18 03:57:30,244 ERROR
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening scanner
>>> (fsOk: true)
>>> java.io.IOException: HStoreScanner failed construction
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>> at
>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>> response
>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>>
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>> at
>>>
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>> at
>>>
>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>> at
>>>
>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>> ... 10 more
>>> 2008-07-18 03:58:01,895 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 7 on 60020, call openScanner([B@9dd0d92, [[B@fa5f14d,
>>> [B@5784fe41,
>>> 9223372036854775807, null) from 140.203.154.174:48958: error:
>>> java.io.IOException: HStoreScanner failed construction
>>> java.io.IOException: HStoreScanner failed construction
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>>> at
>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>> response
>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>>
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>> at
>>>
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>>> at
>>>
>>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>> at
>>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>> at
>>>
>>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>>> ... 10 more
>>> 2008-07-18 03:58:06,728 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> compaction completed on region page-repository,
>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401 in 12mins,
>>> 53sec
>>> 2008-07-18 03:58:06,728 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Angus_McKay,1215890852482
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>03:58:06,729
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Angus_McKay,1215890852482 in 0sec
>>> 2008-07-18 03:58:06,729 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>03:58:06,730
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425 in 0sec
>>> 2008-07-18 03:58:06,730 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>>
>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>> 2008-07-18<
>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>03:58:06,760
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>>
>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in0sec
>>> 2008-07-18 03:58:06,760 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>>
>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>> 2008-07-18<
>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>03:58:06,760
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>>
>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in0sec
>>> 2008-07-18 03:58:06,761 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>> 2008-07-18<
>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>03:58:06,761
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911 in 0sec
>>> 2008-07-18 03:58:06,761 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>>
>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>> 2008-07-18<
>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>03:58:06,777
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>>
>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in0sec
>>> 2008-07-18 03:58:06,777 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>03:58:06,800
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137 in 0sec
>>> 2008-07-18 03:58:06,800 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>03:58:06,826
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663in
>>> 0sec
>>> 2008-07-18 03:58:06,826 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>>
>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>> 2008-07-18<
>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>03:58:06,916
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>>
>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in0sec
>>> 2008-07-18 03:58:06,916 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://sws.geonames.org/3025251/,1215824259058
>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>03:58:06,972
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://sws.geonames.org/3025251/,1215824259058 in 0sec
>>> 2008-07-18 03:58:06,973 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>03:58:06,973
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046in
>>> 0sec
>>> 2008-07-18 03:58:06,973 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>03:58:15,979
>>> INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
>>> hlog. Number of entries: 30009
>>> 2008-07-18 03:59:32,093 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> compaction completed on region page-repository,
>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565 in 1mins,
>>> 25sec
>>> 2008-07-18 03:59:32,093 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>> 2008-07-18<
>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>03:59:32,134
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330 in 0sec
>>> 2008-07-18 03:59:32,134 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:00:45,884
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009 in
>>> 1mins,
>>> 13sec
>>> 2008-07-18 04:00:45,885 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:00:45,887
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943 in 0sec
>>> 2008-07-18 04:00:45,887 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>>
>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:00:45,892
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>>
>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in0sec
>>> 2008-07-18 04:00:45,892 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:00:45,893
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in 0sec
>>> 2008-07-18 04:00:45,893 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>> 2008-07-18<
>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:00:45,895
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in 0sec
>>> 2008-07-18 04:00:45,895 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://sws.geonames.org/1081116/,1215823957458
>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:00:45,895
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://sws.geonames.org/1081116/,1215823957458 in 0sec
>>> 2008-07-18 04:00:45,896 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/Allomeric,1216039960663
>>> 2008-07-18 <
>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:00:45,935
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
>>> 2008-07-18 04:00:45,935 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>>
>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>> 2008-07-18<
>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:00:45,993
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>>
>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in0sec
>>> 2008-07-18 04:00:45,994 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>> 2008-07-18<
>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:00:46,004
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332 in
>>> 0sec
>>> 2008-07-18 04:00:46,005 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>> 2008-07-18<
>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:00:46,005
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>> completed on region page-repository,
>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135 in 0sec
>>> 2008-07-18 04:00:46,005 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> starting compaction on region page-repository,
>>> http://purl.uniprot.org/taxonomy/112557,1214582151856
>>> 2008-07-18<
>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:00:46,007
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>>>
>>> completed on region page-repository,
>>> http://purl.uniprot.org/taxonomy/112557,1214582151856 in 0sec
>>> 2008-07-18 04:02:16,798 FATAL
>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>> Replay of hlog required. Forcing server restart
>>> org.apache.hadoop.hbase.DroppedSnapshotException
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>>> at
>>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>>> at
>>> org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
>>> Caused by: org.apache.hadoop.ipc.RemoteException:
>>> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file
>>>
>>> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
>>> for DFSClient_716087201 on client 140.203.154.176 because current
>>> leaseholder is trying to recreate file.
>>> at
>>>
>>> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
>>> at
>>> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
>>> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
>>> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>>
>>> at org.apache.hadoop.ipc.Client.call(Client.java:557)
>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>>
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>>> at
>>>
>>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>>> at
>>>
>>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
>>> at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
>>> at
>>>
>>> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
>>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
>>> at
>>>
>>> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
>>> at
>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
>>> at
>>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
>>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
>>> at
>>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>>> ... 3 more
>>> 2008-07-18 04:02:16,798 INFO
>>> org.apache.hadoop.hbase.regionserver.Flusher:
>>> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>>> 2008-07-18 04:02:16,807 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 1 on 60020, call batchUpdate([B@167e3c87, row =>
>>> http://dbpedia.org/resource/Jurrette_Sindelar, {column => field:class,
>>> value => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.174:50719: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:16,847 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 9 on 60020, call batchUpdate([B@525cfe10, row =>
>>> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column => field:class,
>>> value => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.175:48121: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:16,848 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 0 on 60020, call batchUpdate([B@572bf6ae, row =>
>>> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29, {column
>>> => field:class, value => '...', column => field:data-source, value =>
>>> '...',
>>> column => field:domain, value => '...', column => field:explicit-content,
>>> value => '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.175:48348: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:16,905 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 4 on 60020, call batchUpdate([B@27320a48, row =>
>>> http://dbpedia.org/resource/Karim_Benkhirane, {column => field:class,
>>> value => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.175:48330: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:16,926 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 4 on 60020, call batchUpdate([B@59c7d8c6, row =>
>>> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column =>
>>> field:class,
>>> value => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.176:58334: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:16,959 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 8 on 60020, call batchUpdate([B@4f68dd36, row =>
>>> http://dbpedia.org/resource/Hendrik_Speuy, {column => field:class, value
>>> => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.175:48357: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:16,979 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 3 on 60020, call batchUpdate([B@596c7f5f, row =>
>>> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column =>
>>> field:class,
>>> value => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.176:58163: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:17,147 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 2 on 60020, call batchUpdate([B@23067cee, row =>
>>> http://dbpedia.org/resource/High_Littleton, {column => field:class,
>>> value
>>> => '...', column => field:data-source, value => '...', column =>
>>> field:domain, value => '...', column => field:explicit-content, value =>
>>> '...', column => field:format, value => '...', column =>
>>> field:implicit-content, value => '...', column => field:label, value =>
>>> '...', column => field:length, value => '...', column => field:ontology,
>>> value => '...', column => field:size, value => '...', column =>
>>> field:timestamp, value => '...'}) from 140.203.154.174:50952: error:
>>> java.io.IOException: Server not running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:17,180 INFO org.apache.hadoop.hbase.regionserver.HLog:
>>> New
>>> log writer created at
>>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
>>> 2008-07-18 04:02:17,181 INFO
>>> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
>>> 2008-07-18 04:02:17,376 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 7 on 60020, call getClosestRowBefore([B@20d09fed, [B@60b3916f)
>>> from 140.203.154.175:48464: error: java.io.IOException: Server not
>>> running
>>> java.io.IOException: Server not running
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>>> at
>>>
>>> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
>>> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>>> at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>> 2008-07-18 04:02:17,507 INFO org.apache.hadoop.ipc.Server: Stopping
>>> server
>>> on 60020
>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 1 on 60020: exiting
>>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 5 on 60020: exiting
>>> 2008-07-18 04:02:17,587 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 0 on 60020: exiting
>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 2 on 60020: exiting
>>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 9 on 60020: exiting
>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 8 on 60020: exiting
>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 7 on 60020: exiting
>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 6 on 60020: exiting
>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 4 on 60020: exiting
>>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> handler 3 on 60020: exiting
>>> 2008-07-18 04:02:17,607 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>> Server listener on 60020
>>> 2008-07-18 04:02:17,691 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
>>> 2008-07-18 04:02:17,784 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>>> Server Responder
>>> 2008-07-18 04:02:17,902 INFO org.mortbay.util.ThreadedServer: Stopping
>>> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
>>> 2008-07-18 04:02:18,179 INFO org.mortbay.http.SocketListener: Stopped
>>> SocketListener on 0.0.0.0:60030
>>> 2008-07-18 04:02:28,883 INFO org.mortbay.util.Container: Stopped
>>> HttpContext[/static,/static]
>>> 2008-07-18 04:02:31,116 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread exiting
>>> 2008-07-18 04:02:31,117 INFO org.mortbay.util.Container: Stopped
>>> HttpContext[/logs,/logs]
>>> 2008-07-18 04:02:31,117 INFO
>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
>>> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
>>> 2008-07-18 04:02:31,125 INFO org.mortbay.util.Container: Stopped
>>> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>> WebApplicationContext[/,/]
>>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>>> org.mortbay.jetty.Server@b8c3679
>>> 2008-07-18 04:02:34,309 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: On abort, closed hlog
>>> 2008-07-18 04:55:59,098 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -3929282808018198437 lease expired
>>> 2008-07-18 04:56:00,258 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> 8694039619106515508 lease expired
>>> 2008-07-18 04:56:02,068 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -6642383936144787497 lease expired
>>> 2008-07-18 04:56:02,178 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -8674274402855984090 lease expired
>>> 2008-07-18 04:56:02,858 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -8684213796880407868 lease expired
>>> 2008-07-18 04:56:03,478 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -8003294377021166340 lease expired
>>> 2008-07-18 04:56:05,298 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> 4741746108028229290 lease expired
>>> 2008-07-18 04:56:05,418 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -3662383214249882020 lease expired
>>> 2008-07-18 04:56:05,498 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>>> -5749397523507975233 lease expired
>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing leases
>>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed leases
>>> 2008-07-18 04:56:05,499 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> closed .META.,,1
>>> 2008-07-18 04:56:05,499 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion:
>>> closed page-repository,http://dbpedia.org/resource/CAMBIA,1215970756532
>>> 2008-07-18 <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18>04:56:05,499
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:56:05,499
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
>>> 2008-07-18<
>>> http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18>04:56:05,499
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>>> 2008-07-18<
>>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://sws.geonames.org/1081116/,1215823957458
>>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>>> 2008-07-18<
>>> http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
>>> 2008-07-18 <http://purl.uniprot.org/go/0009140,12145826028822008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>>> 2008-07-18<
>>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>04:56:05,500
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://bio2rdf.org/omim:300357,1215825157318
>>> 2008-07-18 <http://bio2rdf.org/omim:300357,12158251573182008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
>>> 2008-07-18 <http://alcidesfonseca.com/blog/35,12158259255952008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
>>> 2008-07-18 <
>>> http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>>> 2008-07-18<
>>> http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/John_J._Bernet,1216109320961
>>> 2008-07-18<
>>> http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>04:56:05,501
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://sws.geonames.org/3025251/,1215824259058
>>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Billy_Forester,1216165245424
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
>>> 2008-07-18 <http://dbpedia.org/resource/Snatcho,12162634022752008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>>> 2008-07-18<
>>> http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Template:superheroe,1216140016580
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>>> 2008-07-18<
>>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>>> 2008-07-18<
>>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>04:56:05,502
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>>> 2008-07-18<
>>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
>>> 2008-07-18 <
>>> http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
>>> 2008-07-18<
>>> http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
>>> 2008-07-18 <http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18>04:56:05,503
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
>>> 2008-07-18<
>>> http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Harry_Girvetz,1215910087698
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
>>> 2008-07-18<
>>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://acm.rkbexplorer.com/id/person-94180,1215825860040
>>> 2008-07-18<
>>> http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
>>> 2008-07-18<
>>> http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18>04:56:05,504
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
>>> 2008-07-18<
>>> http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>>> 2008-07-18<
>>> http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>>> 2008-07-18<
>>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>>
>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:56:05,505
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Lysander/lang2,1215922127438
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:56:05,506
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://sws.geonames.org/2701307/,1215823954578
>>> 2008-07-18 <http://sws.geonames.org/2701307/,12158239545782008-07-18>04:56:05,506
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,
>>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>04:56:05,506
>>> INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>>> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
>>> 2008-07-18<
>>> http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>04:56:05,506
>>> INFO org.apache.hadoop.hbase.regionserver.HRegionServer:
>>> aborting server at: 140.203.154.176:60020
>>> 2008-07-18 04:56:05,506 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>>> regionserver/0:0:0:0:0:0:0:0:60020 exiting
>>> 2008-07-18 04:56:05,561 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
>>> thread.
>>> 2008-07-18 04:56:05,562 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread
>>> complete
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
Re: Replay of hlog required, forcing regionserver restart
Posted by Renaud Delbru <re...@deri.org>.
Hi Jean-Daniel,
my nodes clocks are desynchronised (each node has a different time).
Could it have an impact on HBase ?
At the moment of the failure, the node had about 55 regions (with
hbase.hregion.max.filesize = 128M).
Thanks.
--
Renaud Delbru
As One node is at
Jean-Daniel Cryans wrote:
> Renaud,
>
> Just to make sure, are all your nodes clocks synchronized? Also, at the
> moment of that failure, how many regions did the table have?
>
> Thanks,
>
> J-D
>
> On Fri, Jul 18, 2008 at 5:11 AM, Renaud Delbru <re...@deri.org>
> wrote:
>
>
>> Hi,
>>
>> I am using HBase 0.2.0-dev, Hudson Build #208.
>> I am experiencing a problem with HBase that looks like the issue
>> https://issues.apache.org/jira/browse/HBASE-15.
>> During an upload intensive task, one of the regionserver went down to
>> force rerun of hlog.
>>
>> This is not the first time we have experienced this problem, it happened
>> already on another cluster node a few days ago, in the same settings.
>>
>> I have attached the regionserver log. I don't notice something
>> particular in the hadoop datanode log.
>>
>> Thanks in advance for any advices.
>> --
>> Renaud Delbru
>>
>>
>>
>> 2008-07-18 03:40:51,861 INFO
>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>> entries: 30008
>> 2008-07-18 03:40:52,073 INFO org.apache.hadoop.hbase.regionserver.HLog: New
>> log writer created at
>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
>> 2008-07-18 03:43:36,257 INFO
>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>> entries: 30008
>> 2008-07-18 03:43:37,020 INFO org.apache.hadoop.hbase.regionserver.HLog: New
>> log writer created at
>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
>> 2008-07-18 03:45:12,963 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>> 2008-07-18<http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>03:45:58,868 WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
>> ten times longer than scheduled: 3000
>> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We slept
>> 583701ms, ten times longer than scheduled: 3000
>> 2008-07-18 03:55:57,787 INFO
>> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
>> entries: 21011
>> 2008-07-18 03:55:58,410 WARN org.apache.hadoop.dfs.DFSClient: Exception
>> while reading from blk_-6424749704096696952 of
>> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>> inputStream
>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>> at
>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>> at
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>> at
>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>> at
>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>> at
>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>> at
>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>> at
>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>> at
>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>
>> 2008-07-18 03:55:58,966 WARN org.apache.hadoop.dfs.DFSClient: Exception
>> while reading from blk_7305122468419183134 of
>> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
>> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
>> inputStream
>> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
>> at
>> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
>> at
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
>> at
>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
>> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
>> at
>> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
>> at java.io.DataInputStream.readFully(DataInputStream.java:178)
>> at
>> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
>> at
>> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
>> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
>> at
>> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
>> at
>> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>>
>> 2008-07-18 03:56:05,305 INFO org.apache.hadoop.hbase.regionserver.HLog: New
>> log writer created at
>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
>> 2008-07-18 03:57:30,244 ERROR
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening scanner
>> (fsOk: true)
>> java.io.IOException: HStoreScanner failed construction
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>> response
>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>> at
>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>> at
>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>> ... 10 more
>> 2008-07-18 03:58:01,895 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 7 on 60020, call openScanner([B@9dd0d92, [[B@fa5f14d, [B@5784fe41,
>> 9223372036854775807, null) from 140.203.154.174:48958: error:
>> java.io.IOException: HStoreScanner failed construction
>> java.io.IOException: HStoreScanner failed construction
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
>> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>> response
>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
>> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
>> at
>> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
>> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>> at
>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>> at
>> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
>> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
>> at
>> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
>> ... 10 more
>> 2008-07-18 03:58:06,728 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> compaction completed on region page-repository,
>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401 in 12mins,
>> 53sec
>> 2008-07-18 03:58:06,728 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Angus_McKay,1215890852482
>> 2008-07-18<http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>03:58:06,729 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Angus_McKay,1215890852482 in 0sec
>> 2008-07-18 03:58:06,729 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>> 2008-07-18<http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>03:58:06,730 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425 in 0sec
>> 2008-07-18 03:58:06,730 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>> 2008-07-18<http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>03:58:06,760 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in 0sec
>> 2008-07-18 03:58:06,760 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>> 2008-07-18<http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>03:58:06,760 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in 0sec
>> 2008-07-18 03:58:06,761 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>> 2008-07-18<http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>03:58:06,761 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911 in 0sec
>> 2008-07-18 03:58:06,761 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>> 2008-07-18<http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>03:58:06,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in 0sec
>> 2008-07-18 03:58:06,777 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>> 2008-07-18<http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>03:58:06,800 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137 in 0sec
>> 2008-07-18 03:58:06,800 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>> 2008-07-18<http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>03:58:06,826 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663 in
>> 0sec
>> 2008-07-18 03:58:06,826 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>> 2008-07-18<http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>03:58:06,916 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in 0sec
>> 2008-07-18 03:58:06,916 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://sws.geonames.org/3025251/,1215824259058
>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>03:58:06,972 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://sws.geonames.org/3025251/,1215824259058 in 0sec
>> 2008-07-18 03:58:06,973 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>> 2008-07-18<http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>03:58:06,973 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046 in
>> 0sec
>> 2008-07-18 03:58:06,973 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>> 2008-07-18<http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>03:58:15,979 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
>> hlog. Number of entries: 30009
>> 2008-07-18 03:59:32,093 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> compaction completed on region page-repository,
>> http://dbpedia.org/resource/Flowers_of_Life,1216008459565 in 1mins, 25sec
>> 2008-07-18 03:59:32,093 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://acm.rkbexplorer.com/id/person-592138,1215824878330
>> 2008-07-18<http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>03:59:32,134 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://acm.rkbexplorer.com/id/person-592138,1215824878330 in 0sec
>> 2008-07-18 03:59:32,134 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>> 2008-07-18<http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:00:45,884 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009 in 1mins,
>> 13sec
>> 2008-07-18 04:00:45,885 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>> 2008-07-18<http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:00:45,887 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943 in 0sec
>> 2008-07-18 04:00:45,887 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>> 2008-07-18<http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:00:45,892 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in 0sec
>> 2008-07-18 04:00:45,892 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Lysander/lang2,1215922127438
>> 2008-07-18<http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:00:45,893 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in 0sec
>> 2008-07-18 04:00:45,893 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>> 2008-07-18<http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:00:45,895 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in 0sec
>> 2008-07-18 04:00:45,895 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://sws.geonames.org/1081116/,1215823957458
>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:00:45,895 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://sws.geonames.org/1081116/,1215823957458 in 0sec
>> 2008-07-18 04:00:45,896 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/Allomeric,1216039960663
>> 2008-07-18 <http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:00:45,935 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
>> 2008-07-18 04:00:45,935 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>> 2008-07-18<http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:00:45,993 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in 0sec
>> 2008-07-18 04:00:45,994 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>> 2008-07-18<http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:00:46,004 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332 in 0sec
>> 2008-07-18 04:00:46,005 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>> 2008-07-18<http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:00:46,005 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135 in 0sec
>> 2008-07-18 04:00:46,005 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> starting compaction on region page-repository,
>> http://purl.uniprot.org/taxonomy/112557,1214582151856
>> 2008-07-18<http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:00:46,007 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
>> completed on region page-repository,
>> http://purl.uniprot.org/taxonomy/112557,1214582151856 in 0sec
>> 2008-07-18 04:02:16,798 FATAL org.apache.hadoop.hbase.regionserver.Flusher:
>> Replay of hlog required. Forcing server restart
>> org.apache.hadoop.hbase.DroppedSnapshotException
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
>> at
>> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
>> at org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
>> Caused by: org.apache.hadoop.ipc.RemoteException:
>> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file
>> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
>> for DFSClient_716087201 on client 140.203.154.176 because current
>> leaseholder is trying to recreate file.
>> at
>> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
>> at
>> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
>> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
>> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>>
>> at org.apache.hadoop.ipc.Client.call(Client.java:557)
>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
>> at
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
>> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
>> at
>> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
>> at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
>> at
>> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
>> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
>> at
>> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
>> at
>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
>> at
>> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
>> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
>> at
>> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
>> at
>> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
>> ... 3 more
>> 2008-07-18 04:02:16,798 INFO org.apache.hadoop.hbase.regionserver.Flusher:
>> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
>> 2008-07-18 04:02:16,807 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 1 on 60020, call batchUpdate([B@167e3c87, row =>
>> http://dbpedia.org/resource/Jurrette_Sindelar, {column => field:class,
>> value => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.174:50719: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:16,847 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 9 on 60020, call batchUpdate([B@525cfe10, row =>
>> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column => field:class,
>> value => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.175:48121: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:16,848 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 0 on 60020, call batchUpdate([B@572bf6ae, row =>
>> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29, {column
>> => field:class, value => '...', column => field:data-source, value => '...',
>> column => field:domain, value => '...', column => field:explicit-content,
>> value => '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.175:48348: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:16,905 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 4 on 60020, call batchUpdate([B@27320a48, row =>
>> http://dbpedia.org/resource/Karim_Benkhirane, {column => field:class,
>> value => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.175:48330: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:16,926 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 4 on 60020, call batchUpdate([B@59c7d8c6, row =>
>> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column => field:class,
>> value => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.176:58334: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:16,959 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 8 on 60020, call batchUpdate([B@4f68dd36, row =>
>> http://dbpedia.org/resource/Hendrik_Speuy, {column => field:class, value
>> => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.175:48357: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:16,979 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 3 on 60020, call batchUpdate([B@596c7f5f, row =>
>> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column => field:class,
>> value => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.176:58163: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:17,147 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 2 on 60020, call batchUpdate([B@23067cee, row =>
>> http://dbpedia.org/resource/High_Littleton, {column => field:class, value
>> => '...', column => field:data-source, value => '...', column =>
>> field:domain, value => '...', column => field:explicit-content, value =>
>> '...', column => field:format, value => '...', column =>
>> field:implicit-content, value => '...', column => field:label, value =>
>> '...', column => field:length, value => '...', column => field:ontology,
>> value => '...', column => field:size, value => '...', column =>
>> field:timestamp, value => '...'}) from 140.203.154.174:50952: error:
>> java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
>> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:17,180 INFO org.apache.hadoop.hbase.regionserver.HLog: New
>> log writer created at
>> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
>> 2008-07-18 04:02:17,181 INFO
>> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
>> 2008-07-18 04:02:17,376 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 7 on 60020, call getClosestRowBefore([B@20d09fed, [B@60b3916f)
>> from 140.203.154.175:48464: error: java.io.IOException: Server not running
>> java.io.IOException: Server not running
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
>> at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
>> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
>> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>> 2008-07-18 04:02:17,507 INFO org.apache.hadoop.ipc.Server: Stopping server
>> on 60020
>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 1 on 60020: exiting
>> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 5 on 60020: exiting
>> 2008-07-18 04:02:17,587 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 0 on 60020: exiting
>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 2 on 60020: exiting
>> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 9 on 60020: exiting
>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 8 on 60020: exiting
>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 7 on 60020: exiting
>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 6 on 60020: exiting
>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 4 on 60020: exiting
>> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
>> handler 3 on 60020: exiting
>> 2008-07-18 04:02:17,607 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>> Server listener on 60020
>> 2008-07-18 04:02:17,691 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
>> 2008-07-18 04:02:17,784 INFO org.apache.hadoop.ipc.Server: Stopping IPC
>> Server Responder
>> 2008-07-18 04:02:17,902 INFO org.mortbay.util.ThreadedServer: Stopping
>> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
>> 2008-07-18 04:02:18,179 INFO org.mortbay.http.SocketListener: Stopped
>> SocketListener on 0.0.0.0:60030
>> 2008-07-18 04:02:28,883 INFO org.mortbay.util.Container: Stopped
>> HttpContext[/static,/static]
>> 2008-07-18 04:02:31,116 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread exiting
>> 2008-07-18 04:02:31,117 INFO org.mortbay.util.Container: Stopped
>> HttpContext[/logs,/logs]
>> 2008-07-18 04:02:31,117 INFO
>> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
>> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
>> 2008-07-18 04:02:31,125 INFO org.mortbay.util.Container: Stopped
>> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>> WebApplicationContext[/,/]
>> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
>> org.mortbay.jetty.Server@b8c3679
>> 2008-07-18 04:02:34,309 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: On abort, closed hlog
>> 2008-07-18 04:55:59,098 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -3929282808018198437 lease expired
>> 2008-07-18 04:56:00,258 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> 8694039619106515508 lease expired
>> 2008-07-18 04:56:02,068 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -6642383936144787497 lease expired
>> 2008-07-18 04:56:02,178 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -8674274402855984090 lease expired
>> 2008-07-18 04:56:02,858 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -8684213796880407868 lease expired
>> 2008-07-18 04:56:03,478 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -8003294377021166340 lease expired
>> 2008-07-18 04:56:05,298 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> 4741746108028229290 lease expired
>> 2008-07-18 04:56:05,418 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -3662383214249882020 lease expired
>> 2008-07-18 04:56:05,498 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
>> -5749397523507975233 lease expired
>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing leases
>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
>> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed leases
>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> closed .META.,,1
>> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion:
>> closed page-repository,http://dbpedia.org/resource/CAMBIA,1215970756532
>> 2008-07-18 <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18>04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
>> 2008-07-18<http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
>> 2008-07-18<http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18>04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
>> 2008-07-18<http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
>> 2008-07-18<http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://sws.geonames.org/1081116/,1215823957458
>> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://acm.rkbexplorer.com/id/person-592138,1215824878330
>> 2008-07-18<http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
>> 2008-07-18<http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
>> 2008-07-18<http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
>> 2008-07-18 <http://purl.uniprot.org/go/0009140,12145826028822008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
>> 2008-07-18<http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://bio2rdf.org/omim:300357,1215825157318
>> 2008-07-18 <http://bio2rdf.org/omim:300357,12158251573182008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
>> 2008-07-18 <http://alcidesfonseca.com/blog/35,12158259255952008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
>> 2008-07-18 <http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://sws.geonames.org/5197685/nearby.rdf,1215824259791
>> 2008-07-18<http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
>> 2008-07-18<http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/John_J._Bernet,1216109320961
>> 2008-07-18<http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Flowers_of_Life,1216008459565
>> 2008-07-18<http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://sws.geonames.org/3025251/,1215824259058
>> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
>> 2008-07-18<http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Billy_Forester,1216165245424
>> 2008-07-18<http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
>> 2008-07-18<http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
>> 2008-07-18 <http://dbpedia.org/resource/Snatcho,12162634022752008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
>> 2008-07-18<http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Template:superheroe,1216140016580
>> 2008-07-18<http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
>> 2008-07-18<http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
>> 2008-07-18<http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
>> 2008-07-18<http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
>> 2008-07-18 <http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
>> 2008-07-18<http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
>> 2008-07-18<http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
>> 2008-07-18<http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
>> 2008-07-18<http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
>> 2008-07-18 <http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
>> 2008-07-18<http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Harry_Girvetz,1215910087698
>> 2008-07-18<http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
>> 2008-07-18<http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
>> 2008-07-18<http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
>> 2008-07-18<http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://acm.rkbexplorer.com/id/person-94180,1215825860040
>> 2008-07-18<http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
>> 2008-07-18<http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
>> 2008-07-18<http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
>> 2008-07-18<http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
>> 2008-07-18<http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
>> 2008-07-18<http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
>> 2008-07-18<http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
>> 2008-07-18<http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
>> 2008-07-18<http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,
>> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
>> 2008-07-18<http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Lysander/lang2,1215922127438
>> 2008-07-18<http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://sws.geonames.org/2701307/,1215823954578
>> 2008-07-18 <http://sws.geonames.org/2701307/,12158239545782008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
>> 2008-07-18<http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
>> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
>> 2008-07-18<http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegionServer:
>> aborting server at: 140.203.154.176:60020
>> 2008-07-18 04:56:05,506 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer:
>> regionserver/0:0:0:0:0:0:0:0:60020 exiting
>> 2008-07-18 04:56:05,561 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
>> thread.
>> 2008-07-18 04:56:05,562 INFO
>> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread complete
>>
>>
>>
>>
>
>
Re: Replay of hlog required, forcing regionserver restart
Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Renaud,
Just to make sure, are all your nodes clocks synchronized? Also, at the
moment of that failure, how many regions did the table have?
Thanks,
J-D
On Fri, Jul 18, 2008 at 5:11 AM, Renaud Delbru <re...@deri.org>
wrote:
> Hi,
>
> I am using HBase 0.2.0-dev, Hudson Build #208.
> I am experiencing a problem with HBase that looks like the issue
> https://issues.apache.org/jira/browse/HBASE-15.
> During an upload intensive task, one of the regionserver went down to
> force rerun of hlog.
>
> This is not the first time we have experienced this problem, it happened
> already on another cluster node a few days ago, in the same settings.
>
> I have attached the regionserver log. I don't notice something
> particular in the hadoop datanode log.
>
> Thanks in advance for any advices.
> --
> Renaud Delbru
>
>
>
> 2008-07-18 03:40:51,861 INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
> entries: 30008
> 2008-07-18 03:40:52,073 INFO org.apache.hadoop.hbase.regionserver.HLog: New
> log writer created at
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216348851959
> 2008-07-18 03:43:36,257 INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
> entries: 30008
> 2008-07-18 03:43:37,020 INFO org.apache.hadoop.hbase.regionserver.HLog: New
> log writer created at
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349016755
> 2008-07-18 03:45:12,963 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
> 2008-07-18<http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>03:45:58,868 WARN org.apache.hadoop.hbase.util.Sleeper: We slept 30837ms,
> ten times longer than scheduled: 3000
> 2008-07-18 03:55:57,783 WARN org.apache.hadoop.hbase.util.Sleeper: We slept
> 583701ms, ten times longer than scheduled: 3000
> 2008-07-18 03:55:57,787 INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: Rolling hlog. Number of
> entries: 21011
> 2008-07-18 03:55:58,410 WARN org.apache.hadoop.dfs.DFSClient: Exception
> while reading from blk_-6424749704096696952 of
> /hbase/page-repository/942413588/field/mapfiles/7570604533190594376/data
> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
> inputStream
> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
> at
> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
> at
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
> at
> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
> at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
> at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
> at java.io.DataInputStream.readFully(DataInputStream.java:178)
> at
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
> at
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
> at
> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
> at
> org.apache.hadoop.io.SequenceFile$Reader.readBlock(SequenceFile.java:1682)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1799)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1828)
> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>
> 2008-07-18 03:55:58,966 WARN org.apache.hadoop.dfs.DFSClient: Exception
> while reading from blk_7305122468419183134 of
> /hbase/page-repository/942413588/field/mapfiles/4454700123966493614/data
> from 140.203.154.176:50010: java.io.IOException: Premeture EOF from
> inputStream
> at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:100)
> at
> org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:967)
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
> at
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
> at
> org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:829)
> at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1352)
> at
> org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1388)
> at java.io.DataInputStream.readFully(DataInputStream.java:178)
> at
> org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:56)
> at
> org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:90)
> at
> org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1643)
> at
> org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1709)
> at
> org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1745)
> at
> org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1831)
> at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:499)
> at
> org.apache.hadoop.hbase.regionserver.MapFileCompactionReader.next(MapFileCompactionReader.java:45)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compactHStoreFiles(HStore.java:1029)
> at
> org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:880)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:840)
> at
> org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:84)
>
> 2008-07-18 03:56:05,305 INFO org.apache.hadoop.hbase.regionserver.HLog: New
> log writer created at
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216349764848
> 2008-07-18 03:57:30,244 ERROR
> org.apache.hadoop.hbase.regionserver.HRegionServer: Error opening scanner
> (fsOk: true)
> java.io.IOException: HStoreScanner failed construction
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
> at
> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
> response
> at org.apache.hadoop.ipc.Client.call(Client.java:559)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
> at
> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
> at
> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
> ... 10 more
> 2008-07-18 03:58:01,895 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 7 on 60020, call openScanner([B@9dd0d92, [[B@fa5f14d, [B@5784fe41,
> 9223372036854775807, null) from 140.203.154.174:48958: error:
> java.io.IOException: HStoreScanner failed construction
> java.io.IOException: HStoreScanner failed construction
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:69)
> at
> org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:68)
> at
> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1896)
> at
> org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1767)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1293)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1186)
> at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
> response
> at org.apache.hadoop.ipc.Client.call(Client.java:559)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy1.getFileInfo(Unknown Source)
> at org.apache.hadoop.dfs.DFSClient.getFileInfo(DFSClient.java:572)
> at
> org.apache.hadoop.dfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:365)
> at org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:628)
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
> at
> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
> at
> org.apache.hadoop.io.MapFile$Reader.createDataFileReader(MapFile.java:301)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.createDataFileReader(HStoreFile.java:677)
> at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:283)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:659)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:644)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:734)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:409)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:95)
> at
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:66)
> ... 10 more
> 2008-07-18 03:58:06,728 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> compaction completed on region page-repository,
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401 in 12mins,
> 53sec
> 2008-07-18 03:58:06,728 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Angus_McKay,1215890852482
> 2008-07-18<http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>03:58:06,729 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Angus_McKay,1215890852482 in 0sec
> 2008-07-18 03:58:06,729 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
> 2008-07-18<http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>03:58:06,730 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425 in 0sec
> 2008-07-18 03:58:06,730 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
> 2008-07-18<http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>03:58:06,760 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539in 0sec
> 2008-07-18 03:58:06,760 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
> 2008-07-18<http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>03:58:06,760 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811in 0sec
> 2008-07-18 03:58:06,761 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
> 2008-07-18<http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>03:58:06,761 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911 in 0sec
> 2008-07-18 03:58:06,761 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
> 2008-07-18<http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>03:58:06,777 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411in 0sec
> 2008-07-18 03:58:06,777 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
> 2008-07-18<http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>03:58:06,800 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137 in 0sec
> 2008-07-18 03:58:06,800 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
> 2008-07-18<http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>03:58:06,826 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663 in
> 0sec
> 2008-07-18 03:58:06,826 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
> 2008-07-18<http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>03:58:06,916 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814in 0sec
> 2008-07-18 03:58:06,916 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://sws.geonames.org/3025251/,1215824259058
> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>03:58:06,972 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://sws.geonames.org/3025251/,1215824259058 in 0sec
> 2008-07-18 03:58:06,973 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
> 2008-07-18<http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>03:58:06,973 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046 in
> 0sec
> 2008-07-18 03:58:06,973 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Flowers_of_Life,1216008459565
> 2008-07-18<http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>03:58:15,979 INFO org.apache.hadoop.hbase.regionserver.LogRoller: Rolling
> hlog. Number of entries: 30009
> 2008-07-18 03:59:32,093 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> compaction completed on region page-repository,
> http://dbpedia.org/resource/Flowers_of_Life,1216008459565 in 1mins, 25sec
> 2008-07-18 03:59:32,093 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://acm.rkbexplorer.com/id/person-592138,1215824878330
> 2008-07-18<http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>03:59:32,134 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://acm.rkbexplorer.com/id/person-592138,1215824878330 in 0sec
> 2008-07-18 03:59:32,134 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
> 2008-07-18<http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:00:45,884 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009 in 1mins,
> 13sec
> 2008-07-18 04:00:45,885 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
> 2008-07-18<http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:00:45,887 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943 in 0sec
> 2008-07-18 04:00:45,887 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
> 2008-07-18<http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:00:45,892 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842in 0sec
> 2008-07-18 04:00:45,892 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Lysander/lang2,1215922127438
> 2008-07-18<http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:00:45,893 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Lysander/lang2,1215922127438 in 0sec
> 2008-07-18 04:00:45,893 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://sws.geonames.org/5197685/nearby.rdf,1215824259791
> 2008-07-18<http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:00:45,895 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://sws.geonames.org/5197685/nearby.rdf,1215824259791 in 0sec
> 2008-07-18 04:00:45,895 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://sws.geonames.org/1081116/,1215823957458
> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:00:45,895 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://sws.geonames.org/1081116/,1215823957458 in 0sec
> 2008-07-18 04:00:45,896 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/Allomeric,1216039960663
> 2008-07-18 <http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:00:45,935 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/Allomeric,1216039960663 in 0sec
> 2008-07-18 04:00:45,935 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
> 2008-07-18<http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:00:45,993 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259in 0sec
> 2008-07-18 04:00:45,994 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
> 2008-07-18<http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:00:46,004 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332 in 0sec
> 2008-07-18 04:00:46,005 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
> 2008-07-18<http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:00:46,005 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135 in 0sec
> 2008-07-18 04:00:46,005 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> starting compaction on region page-repository,
> http://purl.uniprot.org/taxonomy/112557,1214582151856
> 2008-07-18<http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:00:46,007 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction
> completed on region page-repository,
> http://purl.uniprot.org/taxonomy/112557,1214582151856 in 0sec
> 2008-07-18 04:02:16,798 FATAL org.apache.hadoop.hbase.regionserver.Flusher:
> Replay of hlog required. Forcing server restart
> org.apache.hadoop.hbase.DroppedSnapshotException
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1040)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.flushcache(HRegion.java:942)
> at
> org.apache.hadoop.hbase.regionserver.Flusher.flushRegion(Flusher.java:174)
> at org.apache.hadoop.hbase.regionserver.Flusher.run(Flusher.java:93)
> Caused by: org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.dfs.AlreadyBeingCreatedException: failed to create file
> /hbase/page-repository/1103034897/field/mapfiles/2990615865592052391/index
> for DFSClient_716087201 on client 140.203.154.176 because current
> leaseholder is trying to recreate file.
> at
> org.apache.hadoop.dfs.FSNamesystem.startFileInternal(FSNamesystem.java:1010)
> at
> org.apache.hadoop.dfs.FSNamesystem.startFile(FSNamesystem.java:967)
> at org.apache.hadoop.dfs.NameNode.create(NameNode.java:269)
> at sun.reflect.GeneratedMethodAccessor44.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:446)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
>
> at org.apache.hadoop.ipc.Client.call(Client.java:557)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
> at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
> at org.apache.hadoop.dfs.$Proxy1.create(Unknown Source)
> at
> org.apache.hadoop.dfs.DFSClient$DFSOutputStream.<init>(DFSClient.java:2189)
> at org.apache.hadoop.dfs.DFSClient.create(DFSClient.java:479)
> at
> org.apache.hadoop.dfs.DistributedFileSystem.create(DistributedFileSystem.java:138)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:508)
> at
> org.apache.hadoop.io.SequenceFile$BlockCompressWriter.<init>(SequenceFile.java:1208)
> at
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:402)
> at
> org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:307)
> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:159)
> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:133)
> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:125)
> at org.apache.hadoop.io.MapFile$Writer.<init>(MapFile.java:110)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseWriter.<init>(HStoreFile.java:706)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:810)
> at
> org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:453)
> at
> org.apache.hadoop.hbase.regionserver.HStore.internalFlushCache(HStore.java:706)
> at
> org.apache.hadoop.hbase.regionserver.HStore.flushCache(HStore.java:682)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.internalFlushcache(HRegion.java:1030)
> ... 3 more
> 2008-07-18 04:02:16,798 INFO org.apache.hadoop.hbase.regionserver.Flusher:
> regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher exiting
> 2008-07-18 04:02:16,807 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 1 on 60020, call batchUpdate([B@167e3c87, row =>
> http://dbpedia.org/resource/Jurrette_Sindelar, {column => field:class,
> value => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.174:50719: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:16,847 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 9 on 60020, call batchUpdate([B@525cfe10, row =>
> http://dbpedia.org/resource/Josef_Cyril_Sychra, {column => field:class,
> value => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.175:48121: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:16,848 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 0 on 60020, call batchUpdate([B@572bf6ae, row =>
> http://dbpedia.org/resource/John_Johnston_%28Royal_Household%29, {column
> => field:class, value => '...', column => field:data-source, value => '...',
> column => field:domain, value => '...', column => field:explicit-content,
> value => '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.175:48348: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:16,905 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 4 on 60020, call batchUpdate([B@27320a48, row =>
> http://dbpedia.org/resource/Karim_Benkhirane, {column => field:class,
> value => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.175:48330: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:16,926 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 4 on 60020, call batchUpdate([B@59c7d8c6, row =>
> http://dbpedia.org/resource/Fort_Pierce_%28FL%29, {column => field:class,
> value => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.176:58334: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:16,959 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 60020, call batchUpdate([B@4f68dd36, row =>
> http://dbpedia.org/resource/Hendrik_Speuy, {column => field:class, value
> => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.175:48357: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:16,979 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 3 on 60020, call batchUpdate([B@596c7f5f, row =>
> http://dbpedia.org/resource/Harry_Stuck_Sullivan, {column => field:class,
> value => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.176:58163: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:17,147 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 2 on 60020, call batchUpdate([B@23067cee, row =>
> http://dbpedia.org/resource/High_Littleton, {column => field:class, value
> => '...', column => field:data-source, value => '...', column =>
> field:domain, value => '...', column => field:explicit-content, value =>
> '...', column => field:format, value => '...', column =>
> field:implicit-content, value => '...', column => field:label, value =>
> '...', column => field:length, value => '...', column => field:ontology,
> value => '...', column => field:size, value => '...', column =>
> field:timestamp, value => '...'}) from 140.203.154.174:50952: error:
> java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.batchUpdate(HRegionServer.java:1146)
> at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:17,180 INFO org.apache.hadoop.hbase.regionserver.HLog: New
> log writer created at
> /hbase/log_140.203.154.176_1216023782181_60020/hlog.dat.1216350137145
> 2008-07-18 04:02:17,181 INFO
> org.apache.hadoop.hbase.regionserver.LogRoller: LogRoller exiting.
> 2008-07-18 04:02:17,376 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 7 on 60020, call getClosestRowBefore([B@20d09fed, [B@60b3916f)
> from 140.203.154.175:48464: error: java.io.IOException: Server not running
> java.io.IOException: Server not running
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.checkOpen(HRegionServer.java:1447)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getClosestRowBefore(HRegionServer.java:1092)
> at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:438)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:896)
> 2008-07-18 04:02:17,507 INFO org.apache.hadoop.ipc.Server: Stopping server
> on 60020
> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 1 on 60020: exiting
> 2008-07-18 04:02:17,508 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 5 on 60020: exiting
> 2008-07-18 04:02:17,587 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 0 on 60020: exiting
> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 2 on 60020: exiting
> 2008-07-18 04:02:17,600 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 9 on 60020: exiting
> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 8 on 60020: exiting
> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 7 on 60020: exiting
> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 6 on 60020: exiting
> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 4 on 60020: exiting
> 2008-07-18 04:02:17,601 INFO org.apache.hadoop.ipc.Server: IPC Server
> handler 3 on 60020: exiting
> 2008-07-18 04:02:17,607 INFO org.apache.hadoop.ipc.Server: Stopping IPC
> Server listener on 60020
> 2008-07-18 04:02:17,691 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Stopping infoServer
> 2008-07-18 04:02:17,784 INFO org.apache.hadoop.ipc.Server: Stopping IPC
> Server Responder
> 2008-07-18 04:02:17,902 INFO org.mortbay.util.ThreadedServer: Stopping
> Acceptor ServerSocket[addr=0.0.0.0/0.0.0.0,port=0,localport=60030]
> 2008-07-18 04:02:18,179 INFO org.mortbay.http.SocketListener: Stopped
> SocketListener on 0.0.0.0:60030
> 2008-07-18 04:02:28,883 INFO org.mortbay.util.Container: Stopped
> HttpContext[/static,/static]
> 2008-07-18 04:02:31,116 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: worker thread exiting
> 2008-07-18 04:02:31,117 INFO org.mortbay.util.Container: Stopped
> HttpContext[/logs,/logs]
> 2008-07-18 04:02:31,117 INFO
> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
> regionserver/0:0:0:0:0:0:0:0:60020.compactor exiting
> 2008-07-18 04:02:31,125 INFO org.mortbay.util.Container: Stopped
> org.mortbay.jetty.servlet.WebApplicationHandler@53797795
> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
> WebApplicationContext[/,/]
> 2008-07-18 04:02:34,017 INFO org.mortbay.util.Container: Stopped
> org.mortbay.jetty.Server@b8c3679
> 2008-07-18 04:02:34,309 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: On abort, closed hlog
> 2008-07-18 04:55:59,098 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -3929282808018198437 lease expired
> 2008-07-18 04:56:00,258 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> 8694039619106515508 lease expired
> 2008-07-18 04:56:02,068 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -6642383936144787497 lease expired
> 2008-07-18 04:56:02,178 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -8674274402855984090 lease expired
> 2008-07-18 04:56:02,858 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -8684213796880407868 lease expired
> 2008-07-18 04:56:03,478 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -8003294377021166340 lease expired
> 2008-07-18 04:56:05,298 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> 4741746108028229290 lease expired
> 2008-07-18 04:56:05,418 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -3662383214249882020 lease expired
> 2008-07-18 04:56:05,498 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -5749397523507975233 lease expired
> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closing leases
> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.Leases:
> regionserver/0:0:0:0:0:0:0:0:60020.leaseChecker closed leases
> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> closed .META.,,1
> 2008-07-18 04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> closed page-repository,http://dbpedia.org/resource/CAMBIA,1215970756532
> 2008-07-18 <http://dbpedia.org/resource/CAMBIA,12159707565322008-07-18>04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Brachyotum_fictum,1216044594943
> 2008-07-18<http://dbpedia.org/resource/Brachyotum_fictum,12160445949432008-07-18>04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://users.livejournal.com/_alicesprings/data/foaf,1215824861383
> 2008-07-18<http://users.livejournal.com/_alicesprings/data/foaf,12158248613832008-07-18>04:56:05,499 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Monrad_Charles_Wallgren,1216118586621
> 2008-07-18<http://dbpedia.org/resource/Monrad_Charles_Wallgren,12161185866212008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,1215783796259
> 2008-07-18<http://dbpedia.org/resource/%E7%8E%96%E7%8F%82%E3%83%91%E3%83%BC%E3%82%AD%E3%83%B3%E3%82%B0%E3%82%A8%E3%83%AA%E3%82%A2,12157837962592008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://sws.geonames.org/1081116/,1215823957458
> 2008-07-18 <http://sws.geonames.org/1081116/,12158239574582008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://acm.rkbexplorer.com/id/person-592138,1215824878330
> 2008-07-18<http://acm.rkbexplorer.com/id/person-592138,12158248783302008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,1216118586621
> 2008-07-18<http://dbpedia.org/resource/Microrregi%C3%A3o_de_Ribeir%C3%A3o_Preto,12161185866212008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Jens-Paul_Wollenberg,1216109320961
> 2008-07-18<http://dbpedia.org/resource/Jens-Paul_Wollenberg,12161093209612008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://purl.uniprot.org/go/0009140,1214582602882
> 2008-07-18 <http://purl.uniprot.org/go/0009140,12145826028822008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,1214547593814
> 2008-07-18<http://riese.joanneum.at/data/dca17168/2001_wrl_rest_fgdp_555_de,12145475938142008-07-18>04:56:05,500 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://bio2rdf.org/omim:300357,1215825157318
> 2008-07-18 <http://bio2rdf.org/omim:300357,12158251573182008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://alcidesfonseca.com/blog/35,1215825925595
> 2008-07-18 <http://alcidesfonseca.com/blog/35,12158259255952008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Allomeric,1216039960663
> 2008-07-18 <http://dbpedia.org/resource/Allomeric,12160399606632008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://sws.geonames.org/5197685/nearby.rdf,1215824259791
> 2008-07-18<http://sws.geonames.org/5197685/nearby.rdf,12158242597912008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Portland__Garden_Club,1216125563046
> 2008-07-18<http://dbpedia.org/resource/Portland__Garden_Club,12161255630462008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/John_J._Bernet,1216109320961
> 2008-07-18<http://dbpedia.org/resource/John_J._Bernet,12161093209612008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Flowers_of_Life,1216008459565
> 2008-07-18<http://dbpedia.org/resource/Flowers_of_Life,12160084595652008-07-18>04:56:05,501 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://sws.geonames.org/3025251/,1215824259058
> 2008-07-18 <http://sws.geonames.org/3025251/,12158242590582008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,1216125563046
> 2008-07-18<http://dbpedia.org/resource/Philippe_Le_Hardy_de_Beaulieu,12161255630462008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Billy_Forester,1216165245424
> 2008-07-18<http://dbpedia.org/resource/Billy_Forester,12161652454242008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Judah_ibn_Ghayyat,1215919264401
> 2008-07-18<http://dbpedia.org/resource/Judah_ibn_Ghayyat,12159192644012008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Snatcho,1216263402275
> 2008-07-18 <http://dbpedia.org/resource/Snatcho,12162634022752008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/%2837715%29_1996_RN31,1215824255135
> 2008-07-18<http://dbpedia.org/resource/%2837715%29_1996_RN31,12158242551352008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Template:superheroe,1216140016580
> 2008-07-18<http://dbpedia.org/resource/Template:superheroe,12161400165802008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,1215823897332
> 2008-07-18<http://dblp.rkbexplorer.com/id/conf/cocoon/CohenCF01,12158238973322008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,1215823896411
> 2008-07-18<http://dblp.rkbexplorer.com/id/people-42151d70d8c1a0a74307093179b4a1f4-c2bbeaffc138db617c870e19b352ce16,12158238964112008-07-18>04:56:05,502 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,1214596399811
> 2008-07-18<http://paguntaka.org/2007/08/09/polar-star-mining-corporation-reverse-take-over-transaction/,12145963998112008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Bob_Errey,1216165245424
> 2008-07-18 <http://dbpedia.org/resource/Bob_Errey,12161652454242008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/The_Way_To_Fight,1216138073477
> 2008-07-18<http://dbpedia.org/resource/The_Way_To_Fight,12161380734772008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Al_Fateh-universitetet,1215959690520
> 2008-07-18<http://dbpedia.org/resource/Al_Fateh-universitetet,12159596905202008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Syed_Koteb,1216140016580
> 2008-07-18<http://dbpedia.org/resource/Syed_Koteb,12161400165802008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Humboldt-Universiteit,1215915770009
> 2008-07-18<http://dbpedia.org/resource/Humboldt-Universiteit,12159157700092008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/1er_RCP,1215980565468
> 2008-07-18 <http://dbpedia.org/resource/1er_RCP,12159805654682008-07-18>04:56:05,503 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://purl.uniprot.org/taxonomy/112557,1214582151856
> 2008-07-18<http://purl.uniprot.org/taxonomy/112557,12145821518562008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Harry_Girvetz,1215910087698
> 2008-07-18<http://dbpedia.org/resource/Harry_Girvetz,12159100876982008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,1215922127438
> 2008-07-18<http://dbpedia.org/resource/Liste_der_Autobahnen_in_Spanien,12159221274382008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,1215825925595
> 2008-07-18<http://apassant.net/home/2007/12/flickrdf/data/groups/355509@N23,12158259255952008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Category:1948_in_law,1215896373168
> 2008-07-18<http://dbpedia.org/resource/Category:1948_in_law,12158963731682008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://acm.rkbexplorer.com/id/person-94180,1215825860040
> 2008-07-18<http://acm.rkbexplorer.com/id/person-94180,12158258600402008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://www.akropolix.net/rik0/blogs/category/mac/,1215823941299
> 2008-07-18<http://www.akropolix.net/rik0/blogs/category/mac/,12158239412992008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Trevor_Ashmore,1216138073477
> 2008-07-18<http://dbpedia.org/resource/Trevor_Ashmore,12161380734772008-07-18>04:56:05,504 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,1215929139300
> 2008-07-18<http://dbpedia.org/resource/Papa_New_Guinea_Baseball_Federation,12159291393002008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Ursoaia%2C_Vaslui,1215947224425
> 2008-07-18<http://dbpedia.org/resource/Ursoaia%2C_Vaslui,12159472244252008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,1216039960663
> 2008-07-18<http://dbpedia.org/resource/Amour%2C_honneur_et_trahison,12160399606632008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/1960-09-03,1215980565468
> 2008-07-18<http://dbpedia.org/resource/1960-09-03,12159805654682008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/A38_%28Frankrijk%29,1215958955911
> 2008-07-18<http://dbpedia.org/resource/A38_%28Frankrijk%29,12159589559112008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,1215983091539
> 2008-07-18<http://dblp.rkbexplorer.com/id/people-bbf1b15b69c241e01979fb4c3d8d573f-20e639fc904f03ed0d61aa1255bc3235,12159830915392008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,
> http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,1216004671842
> 2008-07-18<http://dbpedia.org/resource/Cumbernauld_railway_station/rail_line3,12160046718422008-07-18>04:56:05,505 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Lysander/lang2,1215922127438
> 2008-07-18<http://dbpedia.org/resource/Lysander/lang2,12159221274382008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://sws.geonames.org/2701307/,1215823954578
> 2008-07-18 <http://sws.geonames.org/2701307/,12158239545782008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Bayerische_S_2/6,1215823981137
> 2008-07-18<http://dbpedia.org/resource/Bayerische_S_2/6,12158239811372008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegion: closed
> page-repository,http://dbpedia.org/resource/Angus_McKay,1215890852482
> 2008-07-18<http://dbpedia.org/resource/Angus_McKay,12158908524822008-07-18>04:56:05,506 INFO org.apache.hadoop.hbase.regionserver.HRegionServer:
> aborting server at: 140.203.154.176:60020
> 2008-07-18 04:56:05,506 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer:
> regionserver/0:0:0:0:0:0:0:0:60020 exiting
> 2008-07-18 04:56:05,561 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Starting shutdown
> thread.
> 2008-07-18 04:56:05,562 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Shutdown thread complete
>
>
>