You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Stanislaw Kogut <sk...@sistyma.net> on 2010/07/01 14:03:10 UTC
Re: HBase 0.20.5 issues
Completely changed all hadoop configuration to almost default, PE completes
writing for 1000000 rows, but regions still come assigned to multiple RS's
hbase(main):001:0> status 'detailed'
version 0.20.5
0 regionsInTransition
6 live servers
uasstse005.ua.sistyma.com:60020 1277985198620
requests=0, regions=2, usedHeap=25, maxHeap=1196
.META.,,1
stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
storefileIndexSizeMB=0
-ROOT-,,0
stores=1, storefiles=3, storefileSizeMB=0, memstoreSizeMB=0,
storefileIndexSizeMB=0
stas-node.ua.sistyma.com:60020 1277985198573
requests=0, regions=1, usedHeap=22, maxHeap=1996
.META.,,1
stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
storefileIndexSizeMB=0
uasstse004.ua.sistyma.com:60020 1277985198572
requests=0, regions=1, usedHeap=23, maxHeap=1996
.META.,,1
stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
storefileIndexSizeMB=0
uasstse006.ua.sistyma.com:60020 1277985198554
requests=0, regions=0, usedHeap=33, maxHeap=1196
uasstse002.ua.sistyma.com:60020 1277985198667
requests=0, regions=1, usedHeap=34, maxHeap=1996
.META.,,1
stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
storefileIndexSizeMB=0
uasstse003.ua.sistyma.com:60020 1277985198550
requests=0, regions=1, usedHeap=22, maxHeap=1996
.META.,,1
stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
storefileIndexSizeMB=0
0 dead servers
On Wed, Jun 30, 2010 at 2:49 PM, Stanislaw Kogut <sk...@sistyma.net> wrote:
> See clean logs from scratch for hadoop and hbase after start with clean
> hbase rootdir.
>
> http://sp.sistyma.com/hbase_logs.tar.gz
>
>
> On Tue, Jun 29, 2010 at 8:46 PM, Stack <sa...@gmail.com> wrote:
>
>> Something is seriously wrong with your setup. Please put your master logs
>> somewhere we can pull from. Enable debug too. Thanks
>>
>>
>>
>> On Jun 29, 2010, at 10:29 AM, Stanislaw Kogut <sk...@sistyma.net> wrote:
>>
>> > 1. Stopping hbase
>> > 2. Removing hbase.root.dir from hdfs
>> > 3. Starting hbase
>> > 4. Doing major_compact on .META.
>> > 5. Starting PE
>> >
>> > 10/06/29 20:17:30 INFO hbase.PerformanceEvaluation: Table {NAME =>
>> > 'TestTable', FAMILIES => [{NAME => 'info', COMPRESSION => 'NONE',
>> VERSIONS
>> > => '3', TTL => '2147483647', BLOCKSIZE => '65536', IN_MEMORY => 'false',
>> > BLOCKCACHE => 'true'}]} created
>> > 10/06/29 20:17:30 INFO hbase.PerformanceEvaluation: Start class
>> > org.apache.hadoop.hbase.PerformanceEvaluation$SequentialWriteTest at
>> offset
>> > 0 for 1048576 rows
>> > 10/06/29 20:17:42 INFO hbase.PerformanceEvaluation: 0/104857/1048576
>> > 10/06/29 20:17:55 INFO hbase.PerformanceEvaluation: 0/209714/1048576
>> > 10/06/29 20:18:13 INFO hbase.PerformanceEvaluation: 0/314571/1048576
>> > 10/06/29 20:18:29 INFO hbase.PerformanceEvaluation: 0/419428/1048576
>> > 10/06/29 20:22:37 ERROR hbase.PerformanceEvaluation: Failed
>> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
>> contact
>> > region server -- nothing found, no 'location' returned,
>> > tableName=TestTable, reload=true -- for region , row '0000511450', but
>> > failed after 11 attempts.
>> > Exceptions:
>> > java.io.IOException: HRegionInfo was null or empty in .META.
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> > org.apache.hadoop.hbase.TableNotFoundException: TestTable
>> >
>> > at
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocationForRowWithRetries(HConnectionManager.java:1087)
>> > at
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.access$200(HConnectionManager.java:240)
>> > at
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.getRegionName(HConnectionManager.java:1183)
>> > at
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers$Batch.process(HConnectionManager.java:1160)
>> > at
>> >
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.processBatchOfRows(HConnectionManager.java:1230)
>> > at
>> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:666)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation$Test.testTakedown(PerformanceEvaluation.java:621)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation$Test.test(PerformanceEvaluation.java:637)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation.runOneClient(PerformanceEvaluation.java:889)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation.runNIsOne(PerformanceEvaluation.java:907)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation.runTest(PerformanceEvaluation.java:939)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation.doCommandLine(PerformanceEvaluation.java:1036)
>> > at
>> >
>> org.apache.hadoop.hbase.PerformanceEvaluation.main(PerformanceEvaluation.java:1061)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
>> >
>> >
>> > On Tue, Jun 29, 2010 at 8:03 PM, Stack <st...@duboce.net> wrote:
>> >
>> >> For sure you are removing the hbase dir in hdfs?
>> >>
>> >> Try major compaction of your .META. table?
>> >>
>> >> hbase> major_compact ".META."
>> >>
>> >> You seem to be suffering HBASE-1880 but if you are removing the hbase
>> >> dir, you shouldn't be running into this.
>> >>
>> >> St.Ack
>> >>
>> >>
>> > --
>> > Regards,
>> > Stanislaw Kogut
>> > Sistyma LLC
>>
>
>
>
> --
> Regards,
> Stanislaw Kogut
> Sistyma LLC
>
--
Regards,
Stanislaw Kogut
Sistyma LLC
Re: HBase 0.20.5 issues
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I removed that part of the code in
http://svn.apache.org/viewvc/hbase/branches/0.20/src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java?r1=948360&r2=948631
So you don't need to apply that hunk :)
J-D
On Tue, Jul 6, 2010 at 5:14 PM, Ted Yu <yu...@gmail.com> wrote:
> Here is the output from patch:
>
> tyumac:hbase-0.20.5 tyu$ patch -p0 --dry-run < 2599-0.20.txt
> patching file src/java/org/apache/hadoop/hbase/ClusterStatus.java
> patching file
> src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> Hunk #1 FAILED at 520.
> Hunk #2 succeeded at 616 (offset -28 lines).
> Hunk #3 succeeded at 638 (offset -28 lines).
> Hunk #4 succeeded at 1037 (offset -11 lines).
> Hunk #5 succeeded at 1173 (offset -9 lines).
> Hunk #6 succeeded at 1335 (offset -9 lines).
> Hunk #7 succeeded at 1358 (offset -9 lines).
> 1 out of 7 hunks FAILED -- saving rejects to file
> src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java.rej
> patching file src/java/org/apache/hadoop/hbase/HServerInfo.java
> patching file src/java/org/apache/hadoop/hbase/master/ServerManager.java
> Hunk #2 succeeded at 73 (offset -2 lines).
> Hunk #3 succeeded at 105 (offset -2 lines).
> Hunk #4 succeeded at 215 (offset -2 lines).
> Hunk #5 succeeded at 309 (offset -2 lines).
> Hunk #6 succeeded at 332 (offset -2 lines).
> Hunk #7 succeeded at 391 (offset -2 lines).
> Hunk #8 succeeded at 597 (offset -2 lines).
> Hunk #9 succeeded at 686 (offset -2 lines).
> Hunk #10 succeeded at 769 (offset -2 lines).
> Hunk #11 succeeded at 839 (offset -2 lines).
> Hunk #12 succeeded at 903 (offset -2 lines).
> patching file src/java/org/apache/hadoop/hbase/master/HMaster.java
> patching file
> src/java/org/apache/hadoop/hbase/master/ProcessRegionOpen.java
> patching file src/java/org/apache/hadoop/hbase/master/BaseScanner.java
> patching file src/webapps/master/table.jsp
>
> I looked at
> src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
> and the only serverInfo.setStartCode() call is at line 1337.
>
> Please advise how to apply the patch to 0.20.5
>
> Thanks
>
> On Fri, Jul 2, 2010 at 7:57 AM, Stanislaw Kogut <sk...@sistyma.net>
> wrote:
>
> > Thanks, I have applied this patch and issue went out.
> >
> > On Thu, Jul 1, 2010 at 9:18 PM, Jean-Daniel Cryans <jdcryans@apache.org
> > >wrote:
> >
> > > (sorry it took so long to answer, we were all busy with the various
> > > meetings around the Bay Area)
> > >
> > > I can see the issue:
> > >
> > > 2010-06-30 13:48:16,135 DEBUG master.BaseScanner
> > > (BaseScanner.java:checkAssigned(580)) - Current assignment of
> > > .META.,,1 is not valid; serverAddress=, startCode=0 unknown.
> > > ...
> > > 2010-06-30 13:48:26,967 INFO master.RegionServerOperation
> > > (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
> > > -ROOT-,,0 with startcode=1277894894819, server=192.168.242.142:60020
> > > 2010-06-30 13:48:26,968 DEBUG master.RegionServerOperation
> > > (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
> > > {server: 192.168.242.142:60020, regionname: .META.,,1, startKey: <>}
> > > ...
> > > 2010-06-30 13:49:13,050 DEBUG master.BaseScanner
> > > (BaseScanner.java:checkAssigned(580)) - Current assignment of
> > > .META.,,1 is not valid; serverAddress=192.168.242.142:60020,
> > > startCode=1277894894819 unknown.
> > > ...
> > > 2010-06-30 13:49:13,176 INFO master.RegionServerOperation
> > > (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
> > > -ROOT-,,0 with startcode=1277894893999, server=192.168.242.145:60020
> > > 2010-06-30 13:49:13,176 DEBUG master.RegionServerOperation
> > > (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
> > > {server: 192.168.242.145:60020, regionname: .META.,,1, startKey: <>}
> > > ...
> > >
> > > This is https://issues.apache.org/jira/browse/hbase-2599. We didn't
> > > apply it on branch because it was breaking rolling restarts but you
> > > can safely apply the patch before a cluster startup.
> > >
> > > J-D
> > >
> > > On Thu, Jul 1, 2010 at 5:03 AM, Stanislaw Kogut <sk...@sistyma.net>
> > > wrote:
> > > > Completely changed all hadoop configuration to almost default, PE
> > > completes
> > > > writing for 1000000 rows, but regions still come assigned to multiple
> > > RS's
> > > >
> > > > hbase(main):001:0> status 'detailed'
> > > > version 0.20.5
> > > > 0 regionsInTransition
> > > > 6 live servers
> > > > uasstse005.ua.sistyma.com:60020 1277985198620
> > > > requests=0, regions=2, usedHeap=25, maxHeap=1196
> > > > .META.,,1
> > > > stores=2, storefiles=0, storefileSizeMB=0,
> memstoreSizeMB=0,
> > > > storefileIndexSizeMB=0
> > > > -ROOT-,,0
> > > > stores=1, storefiles=3, storefileSizeMB=0,
> memstoreSizeMB=0,
> > > > storefileIndexSizeMB=0
> > > > stas-node.ua.sistyma.com:60020 1277985198573
> > > > requests=0, regions=1, usedHeap=22, maxHeap=1996
> > > > .META.,,1
> > > > stores=2, storefiles=0, storefileSizeMB=0,
> memstoreSizeMB=0,
> > > > storefileIndexSizeMB=0
> > > > uasstse004.ua.sistyma.com:60020 1277985198572
> > > > requests=0, regions=1, usedHeap=23, maxHeap=1996
> > > > .META.,,1
> > > > stores=2, storefiles=0, storefileSizeMB=0,
> memstoreSizeMB=0,
> > > > storefileIndexSizeMB=0
> > > > uasstse006.ua.sistyma.com:60020 1277985198554
> > > > requests=0, regions=0, usedHeap=33, maxHeap=1196
> > > > uasstse002.ua.sistyma.com:60020 1277985198667
> > > > requests=0, regions=1, usedHeap=34, maxHeap=1996
> > > > .META.,,1
> > > > stores=2, storefiles=0, storefileSizeMB=0,
> memstoreSizeMB=0,
> > > > storefileIndexSizeMB=0
> > > > uasstse003.ua.sistyma.com:60020 1277985198550
> > > > requests=0, regions=1, usedHeap=22, maxHeap=1996
> > > > .META.,,1
> > > > stores=2, storefiles=0, storefileSizeMB=0,
> memstoreSizeMB=0,
> > > > storefileIndexSizeMB=0
> > > > 0 dead servers
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Regards,
> > Stanislaw Kogut
> > Sistyma LLC
> >
>
Re: HBase 0.20.5 issues
Posted by Ted Yu <yu...@gmail.com>.
Here is the output from patch:
tyumac:hbase-0.20.5 tyu$ patch -p0 --dry-run < 2599-0.20.txt
patching file src/java/org/apache/hadoop/hbase/ClusterStatus.java
patching file
src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
Hunk #1 FAILED at 520.
Hunk #2 succeeded at 616 (offset -28 lines).
Hunk #3 succeeded at 638 (offset -28 lines).
Hunk #4 succeeded at 1037 (offset -11 lines).
Hunk #5 succeeded at 1173 (offset -9 lines).
Hunk #6 succeeded at 1335 (offset -9 lines).
Hunk #7 succeeded at 1358 (offset -9 lines).
1 out of 7 hunks FAILED -- saving rejects to file
src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java.rej
patching file src/java/org/apache/hadoop/hbase/HServerInfo.java
patching file src/java/org/apache/hadoop/hbase/master/ServerManager.java
Hunk #2 succeeded at 73 (offset -2 lines).
Hunk #3 succeeded at 105 (offset -2 lines).
Hunk #4 succeeded at 215 (offset -2 lines).
Hunk #5 succeeded at 309 (offset -2 lines).
Hunk #6 succeeded at 332 (offset -2 lines).
Hunk #7 succeeded at 391 (offset -2 lines).
Hunk #8 succeeded at 597 (offset -2 lines).
Hunk #9 succeeded at 686 (offset -2 lines).
Hunk #10 succeeded at 769 (offset -2 lines).
Hunk #11 succeeded at 839 (offset -2 lines).
Hunk #12 succeeded at 903 (offset -2 lines).
patching file src/java/org/apache/hadoop/hbase/master/HMaster.java
patching file src/java/org/apache/hadoop/hbase/master/ProcessRegionOpen.java
patching file src/java/org/apache/hadoop/hbase/master/BaseScanner.java
patching file src/webapps/master/table.jsp
I looked at src/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
and the only serverInfo.setStartCode() call is at line 1337.
Please advise how to apply the patch to 0.20.5
Thanks
On Fri, Jul 2, 2010 at 7:57 AM, Stanislaw Kogut <sk...@sistyma.net> wrote:
> Thanks, I have applied this patch and issue went out.
>
> On Thu, Jul 1, 2010 at 9:18 PM, Jean-Daniel Cryans <jdcryans@apache.org
> >wrote:
>
> > (sorry it took so long to answer, we were all busy with the various
> > meetings around the Bay Area)
> >
> > I can see the issue:
> >
> > 2010-06-30 13:48:16,135 DEBUG master.BaseScanner
> > (BaseScanner.java:checkAssigned(580)) - Current assignment of
> > .META.,,1 is not valid; serverAddress=, startCode=0 unknown.
> > ...
> > 2010-06-30 13:48:26,967 INFO master.RegionServerOperation
> > (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
> > -ROOT-,,0 with startcode=1277894894819, server=192.168.242.142:60020
> > 2010-06-30 13:48:26,968 DEBUG master.RegionServerOperation
> > (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
> > {server: 192.168.242.142:60020, regionname: .META.,,1, startKey: <>}
> > ...
> > 2010-06-30 13:49:13,050 DEBUG master.BaseScanner
> > (BaseScanner.java:checkAssigned(580)) - Current assignment of
> > .META.,,1 is not valid; serverAddress=192.168.242.142:60020,
> > startCode=1277894894819 unknown.
> > ...
> > 2010-06-30 13:49:13,176 INFO master.RegionServerOperation
> > (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
> > -ROOT-,,0 with startcode=1277894893999, server=192.168.242.145:60020
> > 2010-06-30 13:49:13,176 DEBUG master.RegionServerOperation
> > (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
> > {server: 192.168.242.145:60020, regionname: .META.,,1, startKey: <>}
> > ...
> >
> > This is https://issues.apache.org/jira/browse/hbase-2599. We didn't
> > apply it on branch because it was breaking rolling restarts but you
> > can safely apply the patch before a cluster startup.
> >
> > J-D
> >
> > On Thu, Jul 1, 2010 at 5:03 AM, Stanislaw Kogut <sk...@sistyma.net>
> > wrote:
> > > Completely changed all hadoop configuration to almost default, PE
> > completes
> > > writing for 1000000 rows, but regions still come assigned to multiple
> > RS's
> > >
> > > hbase(main):001:0> status 'detailed'
> > > version 0.20.5
> > > 0 regionsInTransition
> > > 6 live servers
> > > uasstse005.ua.sistyma.com:60020 1277985198620
> > > requests=0, regions=2, usedHeap=25, maxHeap=1196
> > > .META.,,1
> > > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > > storefileIndexSizeMB=0
> > > -ROOT-,,0
> > > stores=1, storefiles=3, storefileSizeMB=0, memstoreSizeMB=0,
> > > storefileIndexSizeMB=0
> > > stas-node.ua.sistyma.com:60020 1277985198573
> > > requests=0, regions=1, usedHeap=22, maxHeap=1996
> > > .META.,,1
> > > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > > storefileIndexSizeMB=0
> > > uasstse004.ua.sistyma.com:60020 1277985198572
> > > requests=0, regions=1, usedHeap=23, maxHeap=1996
> > > .META.,,1
> > > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > > storefileIndexSizeMB=0
> > > uasstse006.ua.sistyma.com:60020 1277985198554
> > > requests=0, regions=0, usedHeap=33, maxHeap=1196
> > > uasstse002.ua.sistyma.com:60020 1277985198667
> > > requests=0, regions=1, usedHeap=34, maxHeap=1996
> > > .META.,,1
> > > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > > storefileIndexSizeMB=0
> > > uasstse003.ua.sistyma.com:60020 1277985198550
> > > requests=0, regions=1, usedHeap=22, maxHeap=1996
> > > .META.,,1
> > > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > > storefileIndexSizeMB=0
> > > 0 dead servers
> > >
> > >
> >
>
>
>
> --
> Regards,
> Stanislaw Kogut
> Sistyma LLC
>
Re: HBase 0.20.5 issues
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Awesome!
Have fun HBase'ing!
J-D
On Fri, Jul 2, 2010 at 7:57 AM, Stanislaw Kogut <sk...@sistyma.net> wrote:
> Thanks, I have applied this patch and issue went out.
>
> On Thu, Jul 1, 2010 at 9:18 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> (sorry it took so long to answer, we were all busy with the various
>> meetings around the Bay Area)
>>
>> I can see the issue:
>>
>> 2010-06-30 13:48:16,135 DEBUG master.BaseScanner
>> (BaseScanner.java:checkAssigned(580)) - Current assignment of
>> .META.,,1 is not valid; serverAddress=, startCode=0 unknown.
>> ...
>> 2010-06-30 13:48:26,967 INFO master.RegionServerOperation
>> (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
>> -ROOT-,,0 with startcode=1277894894819, server=192.168.242.142:60020
>> 2010-06-30 13:48:26,968 DEBUG master.RegionServerOperation
>> (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
>> {server: 192.168.242.142:60020, regionname: .META.,,1, startKey: <>}
>> ...
>> 2010-06-30 13:49:13,050 DEBUG master.BaseScanner
>> (BaseScanner.java:checkAssigned(580)) - Current assignment of
>> .META.,,1 is not valid; serverAddress=192.168.242.142:60020,
>> startCode=1277894894819 unknown.
>> ...
>> 2010-06-30 13:49:13,176 INFO master.RegionServerOperation
>> (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
>> -ROOT-,,0 with startcode=1277894893999, server=192.168.242.145:60020
>> 2010-06-30 13:49:13,176 DEBUG master.RegionServerOperation
>> (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
>> {server: 192.168.242.145:60020, regionname: .META.,,1, startKey: <>}
>> ...
>>
>> This is https://issues.apache.org/jira/browse/hbase-2599. We didn't
>> apply it on branch because it was breaking rolling restarts but you
>> can safely apply the patch before a cluster startup.
>>
>> J-D
>>
>> On Thu, Jul 1, 2010 at 5:03 AM, Stanislaw Kogut <sk...@sistyma.net>
>> wrote:
>> > Completely changed all hadoop configuration to almost default, PE
>> completes
>> > writing for 1000000 rows, but regions still come assigned to multiple
>> RS's
>> >
>> > hbase(main):001:0> status 'detailed'
>> > version 0.20.5
>> > 0 regionsInTransition
>> > 6 live servers
>> > uasstse005.ua.sistyma.com:60020 1277985198620
>> > requests=0, regions=2, usedHeap=25, maxHeap=1196
>> > .META.,,1
>> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
>> > storefileIndexSizeMB=0
>> > -ROOT-,,0
>> > stores=1, storefiles=3, storefileSizeMB=0, memstoreSizeMB=0,
>> > storefileIndexSizeMB=0
>> > stas-node.ua.sistyma.com:60020 1277985198573
>> > requests=0, regions=1, usedHeap=22, maxHeap=1996
>> > .META.,,1
>> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
>> > storefileIndexSizeMB=0
>> > uasstse004.ua.sistyma.com:60020 1277985198572
>> > requests=0, regions=1, usedHeap=23, maxHeap=1996
>> > .META.,,1
>> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
>> > storefileIndexSizeMB=0
>> > uasstse006.ua.sistyma.com:60020 1277985198554
>> > requests=0, regions=0, usedHeap=33, maxHeap=1196
>> > uasstse002.ua.sistyma.com:60020 1277985198667
>> > requests=0, regions=1, usedHeap=34, maxHeap=1996
>> > .META.,,1
>> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
>> > storefileIndexSizeMB=0
>> > uasstse003.ua.sistyma.com:60020 1277985198550
>> > requests=0, regions=1, usedHeap=22, maxHeap=1996
>> > .META.,,1
>> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
>> > storefileIndexSizeMB=0
>> > 0 dead servers
>> >
>> >
>>
>
>
>
> --
> Regards,
> Stanislaw Kogut
> Sistyma LLC
>
Re: HBase 0.20.5 issues
Posted by Stanislaw Kogut <sk...@sistyma.net>.
Thanks, I have applied this patch and issue went out.
On Thu, Jul 1, 2010 at 9:18 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> (sorry it took so long to answer, we were all busy with the various
> meetings around the Bay Area)
>
> I can see the issue:
>
> 2010-06-30 13:48:16,135 DEBUG master.BaseScanner
> (BaseScanner.java:checkAssigned(580)) - Current assignment of
> .META.,,1 is not valid; serverAddress=, startCode=0 unknown.
> ...
> 2010-06-30 13:48:26,967 INFO master.RegionServerOperation
> (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
> -ROOT-,,0 with startcode=1277894894819, server=192.168.242.142:60020
> 2010-06-30 13:48:26,968 DEBUG master.RegionServerOperation
> (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
> {server: 192.168.242.142:60020, regionname: .META.,,1, startKey: <>}
> ...
> 2010-06-30 13:49:13,050 DEBUG master.BaseScanner
> (BaseScanner.java:checkAssigned(580)) - Current assignment of
> .META.,,1 is not valid; serverAddress=192.168.242.142:60020,
> startCode=1277894894819 unknown.
> ...
> 2010-06-30 13:49:13,176 INFO master.RegionServerOperation
> (ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
> -ROOT-,,0 with startcode=1277894893999, server=192.168.242.145:60020
> 2010-06-30 13:49:13,176 DEBUG master.RegionServerOperation
> (ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
> {server: 192.168.242.145:60020, regionname: .META.,,1, startKey: <>}
> ...
>
> This is https://issues.apache.org/jira/browse/hbase-2599. We didn't
> apply it on branch because it was breaking rolling restarts but you
> can safely apply the patch before a cluster startup.
>
> J-D
>
> On Thu, Jul 1, 2010 at 5:03 AM, Stanislaw Kogut <sk...@sistyma.net>
> wrote:
> > Completely changed all hadoop configuration to almost default, PE
> completes
> > writing for 1000000 rows, but regions still come assigned to multiple
> RS's
> >
> > hbase(main):001:0> status 'detailed'
> > version 0.20.5
> > 0 regionsInTransition
> > 6 live servers
> > uasstse005.ua.sistyma.com:60020 1277985198620
> > requests=0, regions=2, usedHeap=25, maxHeap=1196
> > .META.,,1
> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > -ROOT-,,0
> > stores=1, storefiles=3, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > stas-node.ua.sistyma.com:60020 1277985198573
> > requests=0, regions=1, usedHeap=22, maxHeap=1996
> > .META.,,1
> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > uasstse004.ua.sistyma.com:60020 1277985198572
> > requests=0, regions=1, usedHeap=23, maxHeap=1996
> > .META.,,1
> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > uasstse006.ua.sistyma.com:60020 1277985198554
> > requests=0, regions=0, usedHeap=33, maxHeap=1196
> > uasstse002.ua.sistyma.com:60020 1277985198667
> > requests=0, regions=1, usedHeap=34, maxHeap=1996
> > .META.,,1
> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > uasstse003.ua.sistyma.com:60020 1277985198550
> > requests=0, regions=1, usedHeap=22, maxHeap=1996
> > .META.,,1
> > stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> > storefileIndexSizeMB=0
> > 0 dead servers
> >
> >
>
--
Regards,
Stanislaw Kogut
Sistyma LLC
Re: HBase 0.20.5 issues
Posted by Jean-Daniel Cryans <jd...@apache.org>.
(sorry it took so long to answer, we were all busy with the various
meetings around the Bay Area)
I can see the issue:
2010-06-30 13:48:16,135 DEBUG master.BaseScanner
(BaseScanner.java:checkAssigned(580)) - Current assignment of
.META.,,1 is not valid; serverAddress=, startCode=0 unknown.
...
2010-06-30 13:48:26,967 INFO master.RegionServerOperation
(ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
-ROOT-,,0 with startcode=1277894894819, server=192.168.242.142:60020
2010-06-30 13:48:26,968 DEBUG master.RegionServerOperation
(ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
{server: 192.168.242.142:60020, regionname: .META.,,1, startKey: <>}
...
2010-06-30 13:49:13,050 DEBUG master.BaseScanner
(BaseScanner.java:checkAssigned(580)) - Current assignment of
.META.,,1 is not valid; serverAddress=192.168.242.142:60020,
startCode=1277894894819 unknown.
...
2010-06-30 13:49:13,176 INFO master.RegionServerOperation
(ProcessRegionOpen.java:process(80)) - Updated row .META.,,1 in region
-ROOT-,,0 with startcode=1277894893999, server=192.168.242.145:60020
2010-06-30 13:49:13,176 DEBUG master.RegionServerOperation
(ProcessRegionOpen.java:process(98)) - Adding to onlineMetaRegions:
{server: 192.168.242.145:60020, regionname: .META.,,1, startKey: <>}
...
This is https://issues.apache.org/jira/browse/hbase-2599. We didn't
apply it on branch because it was breaking rolling restarts but you
can safely apply the patch before a cluster startup.
J-D
On Thu, Jul 1, 2010 at 5:03 AM, Stanislaw Kogut <sk...@sistyma.net> wrote:
> Completely changed all hadoop configuration to almost default, PE completes
> writing for 1000000 rows, but regions still come assigned to multiple RS's
>
> hbase(main):001:0> status 'detailed'
> version 0.20.5
> 0 regionsInTransition
> 6 live servers
> uasstse005.ua.sistyma.com:60020 1277985198620
> requests=0, regions=2, usedHeap=25, maxHeap=1196
> .META.,,1
> stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> -ROOT-,,0
> stores=1, storefiles=3, storefileSizeMB=0, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> stas-node.ua.sistyma.com:60020 1277985198573
> requests=0, regions=1, usedHeap=22, maxHeap=1996
> .META.,,1
> stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> uasstse004.ua.sistyma.com:60020 1277985198572
> requests=0, regions=1, usedHeap=23, maxHeap=1996
> .META.,,1
> stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> uasstse006.ua.sistyma.com:60020 1277985198554
> requests=0, regions=0, usedHeap=33, maxHeap=1196
> uasstse002.ua.sistyma.com:60020 1277985198667
> requests=0, regions=1, usedHeap=34, maxHeap=1996
> .META.,,1
> stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> uasstse003.ua.sistyma.com:60020 1277985198550
> requests=0, regions=1, usedHeap=22, maxHeap=1996
> .META.,,1
> stores=2, storefiles=0, storefileSizeMB=0, memstoreSizeMB=0,
> storefileIndexSizeMB=0
> 0 dead servers
>
>