You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Li Li <fa...@gmail.com> on 2014/04/15 02:32:07 UTC
hbase exception: Could not reseek StoreFileScanner
Mon Apr 14 23:54:40 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
3968 Mon Apr 14 23:55:50 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
3969
3970 at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
3971 at
org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
...............
3975 14-04-14 23:58:14,993 ERROR Thread-8
com.founder.extractor.ExtractWorker Failed after attempts=14,
exceptions:
3976 Mon Apr 14 23:51:52 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
3977 Mon Apr 14 23:52:02 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
Re: hbase exception: Could not reseek StoreFileScanner
Posted by Ted Yu <yu...@gmail.com>.
bq. HFileScanner for reader reader=hdfs://
192.168.11.150:8020/hbase/vc2.in_link/6<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75>
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75>
You can find, from master log, which region server hosted
6<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75>
b879cb43205cdae084a280c38fab34a<http://192.168.11.150:8020/hbase/vc2.in_link/6b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75>
Then you can check the region server log on that server.
On Mon, Apr 14, 2014 at 10:11 PM, Li Li <fa...@gmail.com> wrote:
> where to find the server log? I mean there are many region servers.
> should I find one by one?
>
> On Tue, Apr 15, 2014 at 12:31 PM, lars hofhansl <la...@apache.org> wrote:
> > Thanks.
> > Was there anything in the server logs at that time?
> > The client does not report the full stack trace.
> >
> > I have not seen this one before. I assume HDFS was running at the time...
> >
> >
> > -- Lars
> >
> >
> >
> > ________________________________
> > From: Li Li <fa...@gmail.com>
> > To: user@hbase.apache.org; lars hofhansl <la...@apache.org>
> > Sent: Monday, April 14, 2014 9:09 PM
> > Subject: Re: hbase exception: Could not reseek StoreFileScanner
> >
> >
> > Version 0.94.11, r1513697, Wed Aug 14 04:54:46 UTC 2013
> >
> >
> > On Tue, Apr 15, 2014 at 12:03 PM, lars hofhansl <la...@apache.org>
> wrote:
> >> Hi Li,
> >>
> >> please always tell us which version of HBase/Hadoop you are using and
> what it is that you were trying to do.
> >> Thanks.
> >>
> >> -- Lars
> >>
> >>
> >>
> >> ________________________________
> >> From: Li Li <fa...@gmail.com>
> >> To: user@hbase.apache.org
> >> Sent: Monday, April 14, 2014 5:32 PM
> >> Subject: hbase exception: Could not reseek StoreFileScanner
> >>
> >>
> >> Mon Apr 14 23:54:40 CST 2014,
> >> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> >> [cacheCompressed=false],
> >>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> >>
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> >>
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> >> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> >>
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> >> 3968 Mon Apr 14 23:55:50 CST 2014,
> >> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> >> [cacheCompressed=false],
> >>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> >>
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> >>
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> >> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> >>
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> >> 3969
> >> 3970 at
> >>
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
> >> 3971 at
> >> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
> >> ...............
> >>
> >> 3975 14-04-14 23:58:14,993 ERROR Thread-8
> >> com.founder.extractor.ExtractWorker Failed after attempts=14,
> >> exceptions:
> >> 3976 Mon Apr 14 23:51:52 CST 2014,
> >> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> >> [cacheCompressed=false],
> >>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> >>
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> >>
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> >> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> >>
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> >> 3977 Mon Apr 14 23:52:02 CST 2014,
> >> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> >> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> >> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> >> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> >> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> >> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> >> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> >> [cacheCompressed=false],
> >>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> >> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> >>
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> >> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> >>
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> >> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> >>
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>
Re: hbase exception: Could not reseek StoreFileScanner
Posted by Li Li <fa...@gmail.com>.
where to find the server log? I mean there are many region servers.
should I find one by one?
On Tue, Apr 15, 2014 at 12:31 PM, lars hofhansl <la...@apache.org> wrote:
> Thanks.
> Was there anything in the server logs at that time?
> The client does not report the full stack trace.
>
> I have not seen this one before. I assume HDFS was running at the time...
>
>
> -- Lars
>
>
>
> ________________________________
> From: Li Li <fa...@gmail.com>
> To: user@hbase.apache.org; lars hofhansl <la...@apache.org>
> Sent: Monday, April 14, 2014 9:09 PM
> Subject: Re: hbase exception: Could not reseek StoreFileScanner
>
>
> Version 0.94.11, r1513697, Wed Aug 14 04:54:46 UTC 2013
>
>
> On Tue, Apr 15, 2014 at 12:03 PM, lars hofhansl <la...@apache.org> wrote:
>> Hi Li,
>>
>> please always tell us which version of HBase/Hadoop you are using and what it is that you were trying to do.
>> Thanks.
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>> From: Li Li <fa...@gmail.com>
>> To: user@hbase.apache.org
>> Sent: Monday, April 14, 2014 5:32 PM
>> Subject: hbase exception: Could not reseek StoreFileScanner
>>
>>
>> Mon Apr 14 23:54:40 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
>> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>> 3968 Mon Apr 14 23:55:50 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
>> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>> 3969
>> 3970 at
>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
>> 3971 at
>> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
>> ...............
>>
>> 3975 14-04-14 23:58:14,993 ERROR Thread-8
>> com.founder.extractor.ExtractWorker Failed after attempts=14,
>> exceptions:
>> 3976 Mon Apr 14 23:51:52 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
>> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>> 3977 Mon Apr 14 23:52:02 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
>> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
Re: hbase exception: Could not reseek StoreFileScanner
Posted by lars hofhansl <la...@apache.org>.
Thanks.
Was there anything in the server logs at that time?
The client does not report the full stack trace.
I have not seen this one before. I assume HDFS was running at the time...
-- Lars
________________________________
From: Li Li <fa...@gmail.com>
To: user@hbase.apache.org; lars hofhansl <la...@apache.org>
Sent: Monday, April 14, 2014 9:09 PM
Subject: Re: hbase exception: Could not reseek StoreFileScanner
Version 0.94.11, r1513697, Wed Aug 14 04:54:46 UTC 2013
On Tue, Apr 15, 2014 at 12:03 PM, lars hofhansl <la...@apache.org> wrote:
> Hi Li,
>
> please always tell us which version of HBase/Hadoop you are using and what it is that you were trying to do.
> Thanks.
>
> -- Lars
>
>
>
> ________________________________
> From: Li Li <fa...@gmail.com>
> To: user@hbase.apache.org
> Sent: Monday, April 14, 2014 5:32 PM
> Subject: hbase exception: Could not reseek StoreFileScanner
>
>
> Mon Apr 14 23:54:40 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3968 Mon Apr 14 23:55:50 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3969
> 3970 at
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
> 3971 at
> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
> ...............
>
> 3975 14-04-14 23:58:14,993 ERROR Thread-8
> com.founder.extractor.ExtractWorker Failed after attempts=14,
> exceptions:
> 3976 Mon Apr 14 23:51:52 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3977 Mon Apr 14 23:52:02 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
Re: hbase exception: Could not reseek StoreFileScanner
Posted by Li Li <fa...@gmail.com>.
Version 0.94.11, r1513697, Wed Aug 14 04:54:46 UTC 2013
On Tue, Apr 15, 2014 at 12:03 PM, lars hofhansl <la...@apache.org> wrote:
> Hi Li,
>
> please always tell us which version of HBase/Hadoop you are using and what it is that you were trying to do.
> Thanks.
>
> -- Lars
>
>
>
> ________________________________
> From: Li Li <fa...@gmail.com>
> To: user@hbase.apache.org
> Sent: Monday, April 14, 2014 5:32 PM
> Subject: hbase exception: Could not reseek StoreFileScanner
>
>
> Mon Apr 14 23:54:40 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3968 Mon Apr 14 23:55:50 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3969
> 3970 at
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
> 3971 at
> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
> ...............
>
> 3975 14-04-14 23:58:14,993 ERROR Thread-8
> com.founder.extractor.ExtractWorker Failed after attempts=14,
> exceptions:
> 3976 Mon Apr 14 23:51:52 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3977 Mon Apr 14 23:52:02 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
Re: hbase exception: Could not reseek StoreFileScanner
Posted by lars hofhansl <la...@apache.org>.
Hi Li,
please always tell us which version of HBase/Hadoop you are using and what it is that you were trying to do.
Thanks.
-- Lars
________________________________
From: Li Li <fa...@gmail.com>
To: user@hbase.apache.org
Sent: Monday, April 14, 2014 5:32 PM
Subject: hbase exception: Could not reseek StoreFileScanner
Mon Apr 14 23:54:40 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
3968 Mon Apr 14 23:55:50 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
3969
3970 at
org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
3971 at
org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
...............
3975 14-04-14 23:58:14,993 ERROR Thread-8
com.founder.extractor.ExtractWorker Failed after attempts=14,
exceptions:
3976 Mon Apr 14 23:51:52 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
3977 Mon Apr 14 23:52:02 CST 2014,
org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
[cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
[cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
[cacheCompressed=false],
firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
Re: hbase exception: Could not reseek StoreFileScanner
Posted by Li Li <fa...@gmail.com>.
there is something wrong with hdfs. I'll ask admin for help
On Tue, Apr 15, 2014 at 8:57 AM, Ted Yu <yu...@gmail.com> wrote:
> Can you provide a bit more context ?
> Such as: version of HBase you're running, whether the exception happened in
> multiple HFiles or a single HFile, etc.
>
> Cheers
>
>
> On Mon, Apr 14, 2014 at 5:32 PM, Li Li <fa...@gmail.com> wrote:
>
>> Mon Apr 14 23:54:40 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>>
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>>
>> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
>> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>>
>> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>> 3968 Mon Apr 14 23:55:50 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>>
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>>
>> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
>> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>>
>> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>> 3969
>> 3970 at
>>
>> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
>> 3971 at
>> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
>> ...............
>>
>> 3975 14-04-14 23:58:14,993 ERROR Thread-8
>> com.founder.extractor.ExtractWorker Failed after attempts=14,
>> exceptions:
>> 3976 Mon Apr 14 23:51:52 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>>
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>>
>> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
>> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>>
>> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>> 3977 Mon Apr 14 23:52:02 CST 2014,
>> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
>> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
>> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
>> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
>> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
>> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
>> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
>> [cacheCompressed=false],
>>
>> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
>> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
>> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
>> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>>
>> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
>> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>>
>> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>>
Re: hbase exception: Could not reseek StoreFileScanner
Posted by Ted Yu <yu...@gmail.com>.
Can you provide a bit more context ?
Such as: version of HBase you're running, whether the exception happened in
multiple HFiles or a single HFile, etc.
Cheers
On Mon, Apr 14, 2014 at 5:32 PM, Li Li <fa...@gmail.com> wrote:
> Mon Apr 14 23:54:40 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3968 Mon Apr 14 23:55:50 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@14923f6b, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>
> cur=\xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDC
> F/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>
> \xEC5cA\xF1\x03Y\x01!\xD6\x86\x15\x13"\xD6\xC9\xBDb:#A\x08\x86\x14j\xA0)\xA8\x85\x11\xDCF/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3969
> 3970 at
>
> org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:188)
> 3971 at
> org.apache.hadoop.hbase.client.HTable.checkAndPut(HTable.java:918)
> ...............
>
> 3975 14-04-14 23:58:14,993 ERROR Thread-8
> com.founder.extractor.ExtractWorker Failed after attempts=14,
> exceptions:
> 3976 Mon Apr 14 23:51:52 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
> 3977 Mon Apr 14 23:52:02 CST 2014,
> org.apache.hadoop.hbase.client.HTable$9@17b86244, java.io.IOException:
> java.io.IOException: Could not reseek StoreFileScanner[HFileScanner
> for reader reader=hdfs://192.168.11.150:8020/hbase/vc2.in_link/6
> b879cb43205cdae084a280c38fab34a/cf/4dc235709de44f53b2484d2903f1bb75,
> compression=none, cacheConf=CacheConfig:enabled [cacheDataOnRead=true]
> [cacheDataOnWrite=false] [cacheIndexesOnWrite=false]
> [cacheBloomsOnWrite=false] [cacheEvictOn Close=false]
> [cacheCompressed=false],
>
> firstKey=\xE82\x14\xFF/\xF04\xA4\xBC\xB0X\xEB\xB4\xE9\xD1\x11\x93h\xD3\xAA\xC4\xAB\x99\xC3\x09\x874\x16VZ\x05\x10/cf:an/1397117856840/Put,
> lastKey=\xF0\x1F\xA7\xF7u\x9E.\xB2\x8EZ\xD5\xEB\xD6h\x03
> W\x0F\x8A\xA0\x9B>\x0A\xE8\xEC\x9ELu5o\xFE\x03\xCE/cf:an/1397131734218/Put,
> avgKeyLen=48, avgValueLen=14, entries=3712302, length=260849569,
>
> cur=\xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x1
> 6\x16\x0D&/cf:an/1397454753471/Maximum/vlen=0/ts=0] to key
>
> \xED8\xE3R\x1C\xD4\xB5\xB9\xBE\x93\xC2r\xF7\x0461\xBB\xD0\xA7\xF3\xDB\xC8\x85q}^\x8D=\x16\x16\x0D&/cf:an/LATEST_TIMESTAMP/Maximum/vlen=0/ts=0
>