You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by stack <st...@duboce.net> on 2008/07/23 00:24:21 UTC

[ANN] hbase-0.2.0 Release Candidate 1

The first 0.2.0 release candidate is available for download:

  http://people.apache.org/~stack/hbase-0.2.0-candidate-1/

Please take this release candidate for a spin. Check the documentation, 
that unit tests all complete on your platform, etc.

Should we release this candidate as hbase 0.2.0?  Vote yes or no before 
Friday, July 25th.

Release 0.2.0 has over 240 issues resolved [1] since the branch for 0.1 
hbase was made.  Be warned that hbase 0.2.0 is not backward compatible 
with the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level 
API differences between 0.1 and 0.2.  For notes on how to migrate your 
0.1 era hbase data to 0.2, see Izaak's migration guide [3].

Yours,
The HBase Team

1. 
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Rick Hangartner <ha...@strands.com>.
Hi St.Ack,

Thanks for the additional info.

A few additional comments inserted below.

Rick

On Jul 23, 2008, at 11:46 AM, stack wrote:

> Rick:
>
> Thanks for the great feedback.
>
> Other comments interspersed:
>
> Rick Hangartner wrote:
>> ...
>> 2) In both approaches, when we tried to do the data migration from  
>> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to  
>> "unrecovered region server logs". Following the 'Redo Logs'  
>> comments in the "http://wiki.apache.org/hadoop/Hbase/HowToMigrate"  
>> doc, starting afresh with a  new copy of our virtualized system  
>> each time, we tried these methods of getting rid of the those logs  
>> and the fatal error:
>>
>>   a) deleting just the log files in the "/hbase" directory
>
> Did this not work?  How'd you do it?

We used "bin/hadoop dfs -rm" command with a file spec, but it's  
possible we got more than the log files in that delete (I can't  
remember the file spec we used, but it was more than just "/hbase").   
Unfortunately, this was the one case we didn't get a chance to repeat  
to absolutely confirm.

>
>
>
>>   b) deleting the entire contents of the "/hbase" directory (which  
>> means we lost our data, but we are just investigating the upgrade  
>> path, after all)
>>   c) deleting the "/hbase" directory entirely and creating a new "/ 
>> hbase" directory.
>>
>> I should also note that we would need to repeat approach a) to be  
>> 100% certain of our results for that case.  (We've already repeated  
>> approaches b) and c) and have just run out of time for these  
>> upgrade tests because we need to get to other things).
>>
>> In all cases, the migrate then failed as:
>>
>> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
>> available...
>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not  
>> running...
>> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
>> savory1/10.0.0.45:60000. Already tried 1 time(s).
>> ...
>> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
>> savory1/10.0.0.45:60000. Already tried 10 time(s).
>> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
>> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
>> java.io.IOException: Install 0.1.x of hbase and run its migration  
>> first
>>       at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>       at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)
>
> This would seem to indicate that the hbase.version file is missing  
> from under /hbase directory.
>
>>
>> As you can see, we were then in a bit of a Catch-22.  To re-install  
>> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried  
>> reinstalling HBase-0.1.x without doing that!) so there was no way  
>> to proceed.  Attempting to start up HBase-0.2.0 just resulted in an  
>> error message that we needed to do the migrate.
>>
> Hmm.  I suppose this error message is kinda useless.  If you've  
> already made the committment to 0.17.x, you can't really go back.
>
> Luo Ning made a patch so you can run 0.1.3 hbase on 0.17.1 hadoop: https://issues.apache.org/jira/browse/HBASE-749 
> .  Maybe this is what we should be recommending folks do in the  
> migration doc. and in the migration emission?

It certainly seems to me this is a good option to have if one has any  
migration propblems.  And maybe it is just smart even for people to  
run a migrate on 0.1.3 running on 0.17.1 as part of the upgrade  
procedure even if there is no apparent reason to do so, if it doesn't  
involve a tremendous amount of extra work during a upgrade.

Even if it did involve a lot of work, it would only be once and I know  
I'd do it just to be safe.  But then I've always been of the school  
that it's always best to "measure twice, cut once" when real data is  
involved.

>
>
>
>> 3) Since this was just a test, we then blew away the disk used by  
>> Hadoop and re-built the namenode per a standard Hadoop new  
>> install.  Hadoop-0.17.1 and Hbase-0.2.0 then started up just fine.   
>> We only ran a few tests with the new Hbase command line shell in  
>> some ways we used the old HQL shell for sanity checks, and  
>> everything seems copacetic.
>>
>> A few other comments:
>>
>> - The new shell takes a bit of getting used to, but seems quite  
>> functional (we're not the biggest Ruby fans, but hey, someone took  
>> this on and upgraded the shell so we just say: Thanks!)
>>
> Smile.
>
>> - We really like how timestamps have become first-class objects in  
>> the HBase-0.2.0 API .  Although, we were in the middle of  
>> developing some code under HBase-0.1.3 with workarounds for  
>> timestamps not being first-class objects and we will have to decide  
>> whether we should backup and re-develop for HBase-0.2.0 (we know we  
>> should), or plunge ahead with what we were doing under HBase-0.1.3  
>> just to discard it in the near future because of the other  
>> advantages of HBase-0.2.0.  Is there anything we should consider in  
>> making this decision, perhaps about timing of any bug fixes and an  
>> official release of HBase-0.2.0? (HBase-0.2.1?).
> We're sick of looking at 0.1.x hbase.  Can you factor that into your  
> decision regards hbase 0.2 or 0.1?

That's actually pretty much exactly what I was hoping to hear --- that  
going forward with Hbase-0.2.0 is where your interests lie ...

>
>
> Joking aside, a stable offering is our #1 priority ahead of all else  
> whether new features, performance, etc.  In some ways, I'd guess  
> 0.2.0 will probably be less stable than 0.1.3 being new but in  
> others it will be more so (e.g. it has region balancing so no more  
> will you start a cluster and see on node carrying 2 regions and then  
> its partner 200).  Its hard to say.  Best thing to do is do as your  
> doing, testing the 0.2.0 release candidate.  With enough folks  
> banging on it, its possible that it will not only have more features  
> but also be as stable if not more so than our current 0.1.3.

I think working with 0.2.0 release candidate(s) as you suggest best  
fits our plan right now.  And I re-visited our table design today with  
the intent of coming up with a re-design that can be implemented in  
0.1.3 or 0.2.0 so we can go either way --- particularly since we can  
run 0.1.3 on hadoop-0.17.1.

Thanks again for providing the community with a really good system.

Rick

>
>
> Thanks again for the feedback.
>
> St.Ack

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
Rick:

Thanks for the great feedback.

Other comments interspersed:

Rick Hangartner wrote:
> ...
> 2) In both approaches, when we tried to do the data migration from 
> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to 
> "unrecovered region server logs". Following the 'Redo Logs' comments 
> in the "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc, 
> starting afresh with a  new copy of our virtualized system each time, 
> we tried these methods of getting rid of the those logs and the fatal 
> error:
>
>    a) deleting just the log files in the "/hbase" directory

Did this not work?  How'd you do it?


>    b) deleting the entire contents of the "/hbase" directory (which 
> means we lost our data, but we are just investigating the upgrade 
> path, after all)
>    c) deleting the "/hbase" directory entirely and creating a new 
> "/hbase" directory.
>
> I should also note that we would need to repeat approach a) to be 100% 
> certain of our results for that case.  (We've already repeated 
> approaches b) and c) and have just run out of time for these upgrade 
> tests because we need to get to other things).
>
> In all cases, the migrate then failed as:
>
> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
> available...
> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not 
> running...
> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
> savory1/10.0.0.45:60000. Already tried 1 time(s).
>  ...
> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
> savory1/10.0.0.45:60000. Already tried 10 time(s).
> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
> java.io.IOException: Install 0.1.x of hbase and run its migration first
>        at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>        at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)

This would seem to indicate that the hbase.version file is missing from 
under /hbase directory.

>
> As you can see, we were then in a bit of a Catch-22.  To re-install 
> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried 
> reinstalling HBase-0.1.x without doing that!) so there was no way to 
> proceed.  Attempting to start up HBase-0.2.0 just resulted in an error 
> message that we needed to do the migrate.
>
Hmm.  I suppose this error message is kinda useless.  If you've already 
made the committment to 0.17.x, you can't really go back.

Luo Ning made a patch so you can run 0.1.3 hbase on 0.17.1 hadoop: 
https://issues.apache.org/jira/browse/HBASE-749.  Maybe this is what we 
should be recommending folks do in the migration doc. and in the 
migration emission?


> 3) Since this was just a test, we then blew away the disk used by 
> Hadoop and re-built the namenode per a standard Hadoop new install.  
> Hadoop-0.17.1 and Hbase-0.2.0 then started up just fine.  We only ran 
> a few tests with the new Hbase command line shell in some ways we used 
> the old HQL shell for sanity checks, and everything seems copacetic.
>
> A few other comments:
>
> - The new shell takes a bit of getting used to, but seems quite 
> functional (we're not the biggest Ruby fans, but hey, someone took 
> this on and upgraded the shell so we just say: Thanks!)
>
Smile.

> - We really like how timestamps have become first-class objects in the 
> HBase-0.2.0 API .  Although, we were in the middle of developing some 
> code under HBase-0.1.3 with workarounds for timestamps not being 
> first-class objects and we will have to decide whether we should 
> backup and re-develop for HBase-0.2.0 (we know we should), or plunge 
> ahead with what we were doing under HBase-0.1.3 just to discard it in 
> the near future because of the other advantages of HBase-0.2.0.  Is 
> there anything we should consider in making this decision, perhaps 
> about timing of any bug fixes and an official release of HBase-0.2.0? 
> (HBase-0.2.1?).
We're sick of looking at 0.1.x hbase.  Can you factor that into your 
decision regards hbase 0.2 or 0.1?

Joking aside, a stable offering is our #1 priority ahead of all else 
whether new features, performance, etc.  In some ways, I'd guess 0.2.0 
will probably be less stable than 0.1.3 being new but in others it will 
be more so (e.g. it has region balancing so no more will you start a 
cluster and see on node carrying 2 regions and then its partner 200).  
Its hard to say.  Best thing to do is do as your doing, testing the 
0.2.0 release candidate.  With enough folks banging on it, its possible 
that it will not only have more features but also be as stable if not 
more so than our current 0.1.3.

Thanks again for the feedback.

St.Ack

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
Rick:

Thanks for the great feedback.

Other comments interspersed:

Rick Hangartner wrote:
> ...
> 2) In both approaches, when we tried to do the data migration from 
> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to 
> "unrecovered region server logs". Following the 'Redo Logs' comments 
> in the "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc, 
> starting afresh with a  new copy of our virtualized system each time, 
> we tried these methods of getting rid of the those logs and the fatal 
> error:
>
>    a) deleting just the log files in the "/hbase" directory

Did this not work?  How'd you do it?


>    b) deleting the entire contents of the "/hbase" directory (which 
> means we lost our data, but we are just investigating the upgrade 
> path, after all)
>    c) deleting the "/hbase" directory entirely and creating a new 
> "/hbase" directory.
>
> I should also note that we would need to repeat approach a) to be 100% 
> certain of our results for that case.  (We've already repeated 
> approaches b) and c) and have just run out of time for these upgrade 
> tests because we need to get to other things).
>
> In all cases, the migrate then failed as:
>
> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
> available...
> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not 
> running...
> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
> savory1/10.0.0.45:60000. Already tried 1 time(s).
>  ...
> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
> savory1/10.0.0.45:60000. Already tried 10 time(s).
> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
> java.io.IOException: Install 0.1.x of hbase and run its migration first
>        at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>        at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)

This would seem to indicate that the hbase.version file is missing from 
under /hbase directory.

>
> As you can see, we were then in a bit of a Catch-22.  To re-install 
> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried 
> reinstalling HBase-0.1.x without doing that!) so there was no way to 
> proceed.  Attempting to start up HBase-0.2.0 just resulted in an error 
> message that we needed to do the migrate.
>
Hmm.  I suppose this error message is kinda useless.  If you've already 
made the committment to 0.17.x, you can't really go back.

Luo Ning made a patch so you can run 0.1.3 hbase on 0.17.1 hadoop: 
https://issues.apache.org/jira/browse/HBASE-749.  Maybe this is what we 
should be recommending folks do in the migration doc. and in the 
migration emission?


> 3) Since this was just a test, we then blew away the disk used by 
> Hadoop and re-built the namenode per a standard Hadoop new install.  
> Hadoop-0.17.1 and Hbase-0.2.0 then started up just fine.  We only ran 
> a few tests with the new Hbase command line shell in some ways we used 
> the old HQL shell for sanity checks, and everything seems copacetic.
>
> A few other comments:
>
> - The new shell takes a bit of getting used to, but seems quite 
> functional (we're not the biggest Ruby fans, but hey, someone took 
> this on and upgraded the shell so we just say: Thanks!)
>
Smile.

> - We really like how timestamps have become first-class objects in the 
> HBase-0.2.0 API .  Although, we were in the middle of developing some 
> code under HBase-0.1.3 with workarounds for timestamps not being 
> first-class objects and we will have to decide whether we should 
> backup and re-develop for HBase-0.2.0 (we know we should), or plunge 
> ahead with what we were doing under HBase-0.1.3 just to discard it in 
> the near future because of the other advantages of HBase-0.2.0.  Is 
> there anything we should consider in making this decision, perhaps 
> about timing of any bug fixes and an official release of HBase-0.2.0? 
> (HBase-0.2.1?).
We're sick of looking at 0.1.x hbase.  Can you factor that into your 
decision regards hbase 0.2 or 0.1?

Joking aside, a stable offering is our #1 priority ahead of all else 
whether new features, performance, etc.  In some ways, I'd guess 0.2.0 
will probably be less stable than 0.1.3 being new but in others it will 
be more so (e.g. it has region balancing so no more will you start a 
cluster and see on node carrying 2 regions and then its partner 200).  
Its hard to say.  Best thing to do is do as your doing, testing the 
0.2.0 release candidate.  With enough folks banging on it, its possible 
that it will not only have more features but also be as stable if not 
more so than our current 0.1.3.

Thanks again for the feedback.

St.Ack

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Rick Hangartner <ha...@strands.com>.
See below.

On Jul 23, 2008, at 11:25 AM, Jean-Daniel Cryans wrote:

> Rick,
>
> When you deleted the hbase directory and it's content, you also  
> deleted the
> version file and this is why it told you to install 0.1.x first and  
> migrate
> it.

Ahh, that makes sense.  We thought something like this could be afoot,  
perhaps because version info was stored in one or more places that  
could end up "out of sync" after deleting directories and/or files.   
But we weren't sure why Hbase started up without saying we needed to  
do a migrate after we blew away the Hadoop disk, since in that case  
the disk also wouldn't have had an hbase directory or a version file  
in that directory at all the first time HBase is started after that.

I perhaps should mention we did see a case another issue with the  
version file.   As best I can recall, once when we blew away the  
Hadoop disk and rebuilt it HBase threw an exception because of an  
empty version file when we started up "bin/hbase shell" for the first  
time.  When we looked at the HBase FSUtils.java code and figured out  
what it was looking at and for, indeed the version file existed but  
was empty.  When we just deleted that file and re-started HBase it,  
and the shell, then worked just fine.

>
>
> So after to first step (2.a) it was still complaining about redo logs?

(2.a) was the one case we didn't repeat from start to finish carefully  
checking each step because of time limitations (I can't be sure we  
didn't accidentally blow away the version file on our first try).  At  
the time, I had a suspicion if we had done repeated that case  
carefully it would have succeeded.

But no, it wasn't complaining about redo logs after that even when we  
tried to re-run just the "bin/hbase migrate upgrade" operation.

Thanks,
Rick

>
>
> J-D
>
> On Wed, Jul 23, 2008 at 2:11 PM, Rick Hangartner <hangartner@strands.com 
> >
> wrote:
>
>> Hi Jean-Daniel,
>>
>> We just tried migrating test data and didn't get that to work.
>>
>> However, we're at a stage in this project where we are just  
>> finishing a new
>> application with new table(s) so we don't have "real" data to  
>> migrate.  In
>> the end, then, the "upgrade" for us worked because HBase-0.2.0 ran  
>> from a
>> clean install, and so far the pieces of the app I've already ported  
>> pass the
>> tests we have just fine on HBase-0.2.0.  In the next several days,  
>> we'll
>> know a little more as we re-work other pieces of the app and  
>> perhaps make
>> some other attempts at upgrading test data.
>>
>> (Another group here has an app in production using HBase-0.1.x.  I  
>> think
>> they may wait until we get our project up and have some experience  
>> with
>> HBase-0.2.x before upgrading.)
>>
>> Thanks,
>> Rick
>>
>>
>>
>> On Jul 23, 2008, at 10:50 AM, Jean-Daniel Cryans wrote:
>>
>> Rick,
>>>
>>> So you are saying that upgrading from 0.1.3 to 0.2.0 didn't work  
>>> for you?
>>>
>>> Regards your question, I strongly suggest that you develop for the  
>>> 0.2
>>> branch. If you think there is some features that are missing, feel  
>>> free to
>>> fill a Jira and this will solve your problem. Or you can even  
>>> contribute a
>>> patch.
>>>
>>> J-D
>>>
>>> On Wed, Jul 23, 2008 at 1:40 PM, Rick Hangartner <hangartner@strands.com 
>>> >
>>> wrote:
>>>
>>> Hi,
>>>>
>>>> All in all, we like what we've seen of HBase-0.2.0 so far and the  
>>>> API
>>>> changes.
>>>>
>>>> Our upgrade test with Release Candidate 1 was a little bumpy but
>>>> generally
>>>> positive.
>>>>
>>>> We set up a single virtualized test machine (in this case VMWare  
>>>> Fusion
>>>> because an OS X machine was close at hand, although we also run Xen
>>>> virtualized installs), with a virtual disk, running hadoop-0.16.4  
>>>> and
>>>> hbase-0.1.3 with a small test table in a Debian install so we could
>>>> rapidly
>>>> test alternative upgrades paths and data migration.
>>>>
>>>> We tried our own upgrade procedure once based on some previous  
>>>> experience
>>>> and we tried the "formal" upgrade procedure  found at:
>>>>
>>>> http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>> http://wiki.apache.org/hadoop/Hadoop%20Upgrade
>>>>
>>>> In each case, we of course started with a fresh copy of our  
>>>> virtualized
>>>> machine.
>>>>
>>>> Here's what we found:
>>>>
>>>> 1) In both approaches, the upgrade from hadoop-0.16.4 to  
>>>> hadoop-0.17.1
>>>> seem
>>>> to go OK.
>>>>
>>>> 2) In both approaches, when we tried to do the data migration from
>>>> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to
>>>> "unrecovered region server logs". Following the 'Redo Logs'  
>>>> comments in
>>>> the
>>>> "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc, starting  
>>>> afresh
>>>> with a  new copy of our virtualized system each time, we tried  
>>>> these
>>>> methods
>>>> of getting rid of the those logs and the fatal error:
>>>>
>>>> a) deleting just the log files in the "/hbase" directory
>>>> b) deleting the entire contents of the "/hbase" directory (which  
>>>> means
>>>> we
>>>> lost our data, but we are just investigating the upgrade path,  
>>>> after all)
>>>> c) deleting the "/hbase" directory entirely and creating a new "/ 
>>>> hbase"
>>>> directory.
>>>>
>>>> I should also note that we would need to repeat approach a) to be  
>>>> 100%
>>>> certain of our results for that case.  (We've already repeated  
>>>> approaches
>>>> b)
>>>> and c) and have just run out of time for these upgrade tests  
>>>> because we
>>>> need
>>>> to get to other things).
>>>>
>>>> In all cases, the migrate then failed as:
>>>>
>>>> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
>>>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
>>>> available...
>>>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not
>>>> running...
>>>> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
>>>> savory1/10.0.0.45:60000. Already tried 1 time(s).
>>>> ...
>>>> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
>>>> savory1/10.0.0.45:60000. Already tried 10 time(s).
>>>> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
>>>> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
>>>> java.io.IOException: Install 0.1.x of hbase and run its migration  
>>>> first
>>>>    at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>>>>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>    at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)
>>>>
>>>> As you can see, we were then in a bit of a Catch-22.  To re-install
>>>> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried
>>>> reinstalling
>>>> HBase-0.1.x without doing that!) so there was no way to proceed.
>>>> Attempting
>>>> to start up HBase-0.2.0 just resulted in an error message that we  
>>>> needed
>>>> to
>>>> do the migrate.
>>>>
>>>> 3) Since this was just a test, we then blew away the disk used by  
>>>> Hadoop
>>>> and re-built the namenode per a standard Hadoop new install.
>>>> Hadoop-0.17.1
>>>> and Hbase-0.2.0 then started up just fine.  We only ran a few  
>>>> tests with
>>>> the
>>>> new Hbase command line shell in some ways we used the old HQL  
>>>> shell for
>>>> sanity checks, and everything seems copacetic.
>>>>
>>>> A few other comments:
>>>>
>>>> - The new shell takes a bit of getting used to, but seems quite
>>>> functional
>>>> (we're not the biggest Ruby fans, but hey, someone took this on and
>>>> upgraded
>>>> the shell so we just say: Thanks!)
>>>>
>>>> - We really like how timestamps have become first-class objects  
>>>> in the
>>>> HBase-0.2.0 API .  Although, we were in the middle of developing  
>>>> some
>>>> code
>>>> under HBase-0.1.3 with workarounds for timestamps not being first- 
>>>> class
>>>> objects and we will have to decide whether we should backup and
>>>> re-develop
>>>> for HBase-0.2.0 (we know we should), or plunge ahead with what we  
>>>> were
>>>> doing
>>>> under HBase-0.1.3 just to discard it in the near future because  
>>>> of the
>>>> other
>>>> advantages of HBase-0.2.0.  Is there anything we should consider in
>>>> making
>>>> this decision, perhaps about timing of any bug fixes and an  
>>>> official
>>>> release
>>>> of HBase-0.2.0? (HBase-0.2.1?).
>>>>
>>>> Thanks for what continues to be a very useful and interesting  
>>>> product.
>>>>
>>>> Rick
>>>>
>>>>
>>>> On Jul 22, 2008, at 3:24 PM, stack wrote:
>>>>
>>>> The first 0.2.0 release candidate is available for download:
>>>>
>>>>>
>>>>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/ 
>>>>> >
>>>>> <http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>>>>
>>>>> Please take this release candidate for a spin. Check the  
>>>>> documentation,
>>>>> that unit tests all complete on your platform, etc.
>>>>>
>>>>> Should we release this candidate as hbase 0.2.0?  Vote yes or no  
>>>>> before
>>>>> Friday, July 25th.
>>>>>
>>>>> Release 0.2.0 has over 240 issues resolved [1] since the branch  
>>>>> for 0.1
>>>>> hbase was made.  Be warned that hbase 0.2.0 is not backward  
>>>>> compatible
>>>>> with
>>>>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high- 
>>>>> level API
>>>>> differences between 0.1 and 0.2.  For notes on how to migrate  
>>>>> your 0.1
>>>>> era
>>>>> hbase data to 0.2, see Izaak's migration guide [3].
>>>>>
>>>>> Yours,
>>>>> The HBase Team
>>>>>
>>>>> 1.
>>>>>
>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>>>>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>>>>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>>>
>>>>>
>>>>
>>>>
>>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Rick,

When you deleted the hbase directory and it's content, you also deleted the
version file and this is why it told you to install 0.1.x first and migrate
it.

So after to first step (2.a) it was still complaining about redo logs?

J-D

On Wed, Jul 23, 2008 at 2:11 PM, Rick Hangartner <ha...@strands.com>
wrote:

> Hi Jean-Daniel,
>
> We just tried migrating test data and didn't get that to work.
>
> However, we're at a stage in this project where we are just finishing a new
> application with new table(s) so we don't have "real" data to migrate.  In
> the end, then, the "upgrade" for us worked because HBase-0.2.0 ran from a
> clean install, and so far the pieces of the app I've already ported pass the
> tests we have just fine on HBase-0.2.0.  In the next several days, we'll
> know a little more as we re-work other pieces of the app and perhaps make
> some other attempts at upgrading test data.
>
> (Another group here has an app in production using HBase-0.1.x.  I think
> they may wait until we get our project up and have some experience with
> HBase-0.2.x before upgrading.)
>
> Thanks,
> Rick
>
>
>
> On Jul 23, 2008, at 10:50 AM, Jean-Daniel Cryans wrote:
>
>  Rick,
>>
>> So you are saying that upgrading from 0.1.3 to 0.2.0 didn't work for you?
>>
>> Regards your question, I strongly suggest that you develop for the 0.2
>> branch. If you think there is some features that are missing, feel free to
>> fill a Jira and this will solve your problem. Or you can even contribute a
>> patch.
>>
>> J-D
>>
>> On Wed, Jul 23, 2008 at 1:40 PM, Rick Hangartner <ha...@strands.com>
>> wrote:
>>
>>  Hi,
>>>
>>> All in all, we like what we've seen of HBase-0.2.0 so far and the API
>>> changes.
>>>
>>> Our upgrade test with Release Candidate 1 was a little bumpy but
>>> generally
>>> positive.
>>>
>>> We set up a single virtualized test machine (in this case VMWare Fusion
>>> because an OS X machine was close at hand, although we also run Xen
>>> virtualized installs), with a virtual disk, running hadoop-0.16.4 and
>>> hbase-0.1.3 with a small test table in a Debian install so we could
>>> rapidly
>>> test alternative upgrades paths and data migration.
>>>
>>> We tried our own upgrade procedure once based on some previous experience
>>> and we tried the "formal" upgrade procedure  found at:
>>>
>>> http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>> http://wiki.apache.org/hadoop/Hadoop%20Upgrade
>>>
>>> In each case, we of course started with a fresh copy of our virtualized
>>> machine.
>>>
>>> Here's what we found:
>>>
>>> 1) In both approaches, the upgrade from hadoop-0.16.4 to hadoop-0.17.1
>>> seem
>>> to go OK.
>>>
>>> 2) In both approaches, when we tried to do the data migration from
>>> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to
>>> "unrecovered region server logs". Following the 'Redo Logs' comments in
>>> the
>>> "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc, starting afresh
>>> with a  new copy of our virtualized system each time, we tried these
>>> methods
>>> of getting rid of the those logs and the fatal error:
>>>
>>>  a) deleting just the log files in the "/hbase" directory
>>>  b) deleting the entire contents of the "/hbase" directory (which means
>>> we
>>> lost our data, but we are just investigating the upgrade path, after all)
>>>  c) deleting the "/hbase" directory entirely and creating a new "/hbase"
>>> directory.
>>>
>>> I should also note that we would need to repeat approach a) to be 100%
>>> certain of our results for that case.  (We've already repeated approaches
>>> b)
>>> and c) and have just run out of time for these upgrade tests because we
>>> need
>>> to get to other things).
>>>
>>> In all cases, the migrate then failed as:
>>>
>>> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
>>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
>>> available...
>>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not
>>> running...
>>> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
>>> savory1/10.0.0.45:60000. Already tried 1 time(s).
>>> ...
>>> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
>>> savory1/10.0.0.45:60000. Already tried 10 time(s).
>>> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
>>> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
>>> java.io.IOException: Install 0.1.x of hbase and run its migration first
>>>     at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>>>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>     at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>     at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)
>>>
>>> As you can see, we were then in a bit of a Catch-22.  To re-install
>>> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried
>>> reinstalling
>>> HBase-0.1.x without doing that!) so there was no way to proceed.
>>>  Attempting
>>> to start up HBase-0.2.0 just resulted in an error message that we needed
>>> to
>>> do the migrate.
>>>
>>> 3) Since this was just a test, we then blew away the disk used by Hadoop
>>> and re-built the namenode per a standard Hadoop new install.
>>>  Hadoop-0.17.1
>>> and Hbase-0.2.0 then started up just fine.  We only ran a few tests with
>>> the
>>> new Hbase command line shell in some ways we used the old HQL shell for
>>> sanity checks, and everything seems copacetic.
>>>
>>> A few other comments:
>>>
>>> - The new shell takes a bit of getting used to, but seems quite
>>> functional
>>> (we're not the biggest Ruby fans, but hey, someone took this on and
>>> upgraded
>>> the shell so we just say: Thanks!)
>>>
>>> - We really like how timestamps have become first-class objects in the
>>> HBase-0.2.0 API .  Although, we were in the middle of developing some
>>> code
>>> under HBase-0.1.3 with workarounds for timestamps not being first-class
>>> objects and we will have to decide whether we should backup and
>>> re-develop
>>> for HBase-0.2.0 (we know we should), or plunge ahead with what we were
>>> doing
>>> under HBase-0.1.3 just to discard it in the near future because of the
>>> other
>>> advantages of HBase-0.2.0.  Is there anything we should consider in
>>> making
>>> this decision, perhaps about timing of any bug fixes and an official
>>> release
>>> of HBase-0.2.0? (HBase-0.2.1?).
>>>
>>> Thanks for what continues to be a very useful and interesting product.
>>>
>>> Rick
>>>
>>>
>>> On Jul 22, 2008, at 3:24 PM, stack wrote:
>>>
>>> The first 0.2.0 release candidate is available for download:
>>>
>>>>
>>>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>>> <http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>>>
>>>> Please take this release candidate for a spin. Check the documentation,
>>>> that unit tests all complete on your platform, etc.
>>>>
>>>> Should we release this candidate as hbase 0.2.0?  Vote yes or no before
>>>> Friday, July 25th.
>>>>
>>>> Release 0.2.0 has over 240 issues resolved [1] since the branch for 0.1
>>>> hbase was made.  Be warned that hbase 0.2.0 is not backward compatible
>>>> with
>>>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API
>>>> differences between 0.1 and 0.2.  For notes on how to migrate your 0.1
>>>> era
>>>> hbase data to 0.2, see Izaak's migration guide [3].
>>>>
>>>> Yours,
>>>> The HBase Team
>>>>
>>>> 1.
>>>>
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>>>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>>>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>>
>>>>
>>>
>>>
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Rick Hangartner <ha...@strands.com>.
Hi Jean-Daniel,

We just tried migrating test data and didn't get that to work.

However, we're at a stage in this project where we are just finishing  
a new application with new table(s) so we don't have "real" data to  
migrate.  In the end, then, the "upgrade" for us worked because  
HBase-0.2.0 ran from a clean install, and so far the pieces of the app  
I've already ported pass the tests we have just fine on HBase-0.2.0.   
In the next several days, we'll know a little more as we re-work other  
pieces of the app and perhaps make some other attempts at upgrading  
test data.

(Another group here has an app in production using HBase-0.1.x.  I  
think they may wait until we get our project up and have some  
experience with HBase-0.2.x before upgrading.)

Thanks,
Rick


On Jul 23, 2008, at 10:50 AM, Jean-Daniel Cryans wrote:

> Rick,
>
> So you are saying that upgrading from 0.1.3 to 0.2.0 didn't work for  
> you?
>
> Regards your question, I strongly suggest that you develop for the 0.2
> branch. If you think there is some features that are missing, feel  
> free to
> fill a Jira and this will solve your problem. Or you can even  
> contribute a
> patch.
>
> J-D
>
> On Wed, Jul 23, 2008 at 1:40 PM, Rick Hangartner <hangartner@strands.com 
> >
> wrote:
>
>> Hi,
>>
>> All in all, we like what we've seen of HBase-0.2.0 so far and the API
>> changes.
>>
>> Our upgrade test with Release Candidate 1 was a little bumpy but  
>> generally
>> positive.
>>
>> We set up a single virtualized test machine (in this case VMWare  
>> Fusion
>> because an OS X machine was close at hand, although we also run Xen
>> virtualized installs), with a virtual disk, running hadoop-0.16.4 and
>> hbase-0.1.3 with a small test table in a Debian install so we could  
>> rapidly
>> test alternative upgrades paths and data migration.
>>
>> We tried our own upgrade procedure once based on some previous  
>> experience
>> and we tried the "formal" upgrade procedure  found at:
>>
>> http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>> http://wiki.apache.org/hadoop/Hadoop%20Upgrade
>>
>> In each case, we of course started with a fresh copy of our  
>> virtualized
>> machine.
>>
>> Here's what we found:
>>
>> 1) In both approaches, the upgrade from hadoop-0.16.4 to  
>> hadoop-0.17.1 seem
>> to go OK.
>>
>> 2) In both approaches, when we tried to do the data migration from
>> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to
>> "unrecovered region server logs". Following the 'Redo Logs'  
>> comments in the
>> "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc, starting  
>> afresh
>> with a  new copy of our virtualized system each time, we tried  
>> these methods
>> of getting rid of the those logs and the fatal error:
>>
>>  a) deleting just the log files in the "/hbase" directory
>>  b) deleting the entire contents of the "/hbase" directory (which  
>> means we
>> lost our data, but we are just investigating the upgrade path,  
>> after all)
>>  c) deleting the "/hbase" directory entirely and creating a new "/ 
>> hbase"
>> directory.
>>
>> I should also note that we would need to repeat approach a) to be  
>> 100%
>> certain of our results for that case.  (We've already repeated  
>> approaches b)
>> and c) and have just run out of time for these upgrade tests  
>> because we need
>> to get to other things).
>>
>> In all cases, the migrate then failed as:
>>
>> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
>> available...
>> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not  
>> running...
>> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
>> savory1/10.0.0.45:60000. Already tried 1 time(s).
>> ...
>> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
>> savory1/10.0.0.45:60000. Already tried 10 time(s).
>> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
>> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
>> java.io.IOException: Install 0.1.x of hbase and run its migration  
>> first
>>      at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>>      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>      at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)
>>
>> As you can see, we were then in a bit of a Catch-22.  To re-install
>> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried  
>> reinstalling
>> HBase-0.1.x without doing that!) so there was no way to proceed.   
>> Attempting
>> to start up HBase-0.2.0 just resulted in an error message that we  
>> needed to
>> do the migrate.
>>
>> 3) Since this was just a test, we then blew away the disk used by  
>> Hadoop
>> and re-built the namenode per a standard Hadoop new install.   
>> Hadoop-0.17.1
>> and Hbase-0.2.0 then started up just fine.  We only ran a few tests  
>> with the
>> new Hbase command line shell in some ways we used the old HQL shell  
>> for
>> sanity checks, and everything seems copacetic.
>>
>> A few other comments:
>>
>> - The new shell takes a bit of getting used to, but seems quite  
>> functional
>> (we're not the biggest Ruby fans, but hey, someone took this on and  
>> upgraded
>> the shell so we just say: Thanks!)
>>
>> - We really like how timestamps have become first-class objects in  
>> the
>> HBase-0.2.0 API .  Although, we were in the middle of developing  
>> some code
>> under HBase-0.1.3 with workarounds for timestamps not being first- 
>> class
>> objects and we will have to decide whether we should backup and re- 
>> develop
>> for HBase-0.2.0 (we know we should), or plunge ahead with what we  
>> were doing
>> under HBase-0.1.3 just to discard it in the near future because of  
>> the other
>> advantages of HBase-0.2.0.  Is there anything we should consider in  
>> making
>> this decision, perhaps about timing of any bug fixes and an  
>> official release
>> of HBase-0.2.0? (HBase-0.2.1?).
>>
>> Thanks for what continues to be a very useful and interesting  
>> product.
>>
>> Rick
>>
>>
>> On Jul 22, 2008, at 3:24 PM, stack wrote:
>>
>> The first 0.2.0 release candidate is available for download:
>>>
>>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/ 
>>> >
>>>
>>> Please take this release candidate for a spin. Check the  
>>> documentation,
>>> that unit tests all complete on your platform, etc.
>>>
>>> Should we release this candidate as hbase 0.2.0?  Vote yes or no  
>>> before
>>> Friday, July 25th.
>>>
>>> Release 0.2.0 has over 240 issues resolved [1] since the branch  
>>> for 0.1
>>> hbase was made.  Be warned that hbase 0.2.0 is not backward  
>>> compatible with
>>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level  
>>> API
>>> differences between 0.1 and 0.2.  For notes on how to migrate your  
>>> 0.1 era
>>> hbase data to 0.2, see Izaak's migration guide [3].
>>>
>>> Yours,
>>> The HBase Team
>>>
>>> 1.
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>
>>
>>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Rick,

So you are saying that upgrading from 0.1.3 to 0.2.0 didn't work for you?

Regards your question, I strongly suggest that you develop for the 0.2
branch. If you think there is some features that are missing, feel free to
fill a Jira and this will solve your problem. Or you can even contribute a
patch.

J-D

On Wed, Jul 23, 2008 at 1:40 PM, Rick Hangartner <ha...@strands.com>
wrote:

> Hi,
>
> All in all, we like what we've seen of HBase-0.2.0 so far and the API
> changes.
>
> Our upgrade test with Release Candidate 1 was a little bumpy but generally
> positive.
>
> We set up a single virtualized test machine (in this case VMWare Fusion
> because an OS X machine was close at hand, although we also run Xen
> virtualized installs), with a virtual disk, running hadoop-0.16.4 and
> hbase-0.1.3 with a small test table in a Debian install so we could rapidly
> test alternative upgrades paths and data migration.
>
> We tried our own upgrade procedure once based on some previous experience
> and we tried the "formal" upgrade procedure  found at:
>
> http://wiki.apache.org/hadoop/Hbase/HowToMigrate
> http://wiki.apache.org/hadoop/Hadoop%20Upgrade
>
> In each case, we of course started with a fresh copy of our virtualized
> machine.
>
> Here's what we found:
>
> 1) In both approaches, the upgrade from hadoop-0.16.4 to hadoop-0.17.1 seem
> to go OK.
>
> 2) In both approaches, when we tried to do the data migration from
> hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to
> "unrecovered region server logs". Following the 'Redo Logs' comments in the
> "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc, starting afresh
> with a  new copy of our virtualized system each time, we tried these methods
> of getting rid of the those logs and the fatal error:
>
>   a) deleting just the log files in the "/hbase" directory
>   b) deleting the entire contents of the "/hbase" directory (which means we
> lost our data, but we are just investigating the upgrade path, after all)
>   c) deleting the "/hbase" directory entirely and creating a new "/hbase"
> directory.
>
> I should also note that we would need to repeat approach a) to be 100%
> certain of our results for that case.  (We've already repeated approaches b)
> and c) and have just run out of time for these upgrade tests because we need
> to get to other things).
>
> In all cases, the migrate then failed as:
>
> hbase@savory1:~/hbase$ bin/hbase migrate upgrade
> 08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
> available...
> 08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not running...
> 08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
> savory1/10.0.0.45:60000. Already tried 1 time(s).
>  ...
> 08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
> savory1/10.0.0.45:60000. Already tried 10 time(s).
> 08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
> 08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
> java.io.IOException: Install 0.1.x of hbase and run its migration first
>       at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>       at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>       at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)
>
> As you can see, we were then in a bit of a Catch-22.  To re-install
> HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried reinstalling
> HBase-0.1.x without doing that!) so there was no way to proceed.  Attempting
> to start up HBase-0.2.0 just resulted in an error message that we needed to
> do the migrate.
>
> 3) Since this was just a test, we then blew away the disk used by Hadoop
> and re-built the namenode per a standard Hadoop new install.  Hadoop-0.17.1
> and Hbase-0.2.0 then started up just fine.  We only ran a few tests with the
> new Hbase command line shell in some ways we used the old HQL shell for
> sanity checks, and everything seems copacetic.
>
> A few other comments:
>
> - The new shell takes a bit of getting used to, but seems quite functional
> (we're not the biggest Ruby fans, but hey, someone took this on and upgraded
> the shell so we just say: Thanks!)
>
> - We really like how timestamps have become first-class objects in the
> HBase-0.2.0 API .  Although, we were in the middle of developing some code
> under HBase-0.1.3 with workarounds for timestamps not being first-class
> objects and we will have to decide whether we should backup and re-develop
> for HBase-0.2.0 (we know we should), or plunge ahead with what we were doing
> under HBase-0.1.3 just to discard it in the near future because of the other
> advantages of HBase-0.2.0.  Is there anything we should consider in making
> this decision, perhaps about timing of any bug fixes and an official release
> of HBase-0.2.0? (HBase-0.2.1?).
>
> Thanks for what continues to be a very useful and interesting product.
>
> Rick
>
>
> On Jul 22, 2008, at 3:24 PM, stack wrote:
>
>  The first 0.2.0 release candidate is available for download:
>>
>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>
>> Please take this release candidate for a spin. Check the documentation,
>> that unit tests all complete on your platform, etc.
>>
>> Should we release this candidate as hbase 0.2.0?  Vote yes or no before
>> Friday, July 25th.
>>
>> Release 0.2.0 has over 240 issues resolved [1] since the branch for 0.1
>> hbase was made.  Be warned that hbase 0.2.0 is not backward compatible with
>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API
>> differences between 0.1 and 0.2.  For notes on how to migrate your 0.1 era
>> hbase data to 0.2, see Izaak's migration guide [3].
>>
>> Yours,
>> The HBase Team
>>
>> 1.
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>
>
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Rick Hangartner <ha...@strands.com>.
Hi,

All in all, we like what we've seen of HBase-0.2.0 so far and the API  
changes.

Our upgrade test with Release Candidate 1 was a little bumpy but  
generally positive.

We set up a single virtualized test machine (in this case VMWare  
Fusion because an OS X machine was close at hand, although we also run  
Xen virtualized installs), with a virtual disk, running hadoop-0.16.4  
and hbase-0.1.3 with a small test table in a Debian install so we  
could rapidly test alternative upgrades paths and data migration.

We tried our own upgrade procedure once based on some previous  
experience and we tried the "formal" upgrade procedure  found at:

http://wiki.apache.org/hadoop/Hbase/HowToMigrate
http://wiki.apache.org/hadoop/Hadoop%20Upgrade

In each case, we of course started with a fresh copy of our  
virtualized machine.

Here's what we found:

1) In both approaches, the upgrade from hadoop-0.16.4 to hadoop-0.17.1  
seem to go OK.

2) In both approaches, when we tried to do the data migration from  
hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to  
"unrecovered region server logs". Following the 'Redo Logs' comments  
in the "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc,  
starting afresh with a  new copy of our virtualized system each time,  
we tried these methods of getting rid of the those logs and the fatal  
error:

    a) deleting just the log files in the "/hbase" directory
    b) deleting the entire contents of the "/hbase" directory (which  
means we lost our data, but we are just investigating the upgrade  
path, after all)
    c) deleting the "/hbase" directory entirely and creating a new "/ 
hbase" directory.

I should also note that we would need to repeat approach a) to be 100%  
certain of our results for that case.  (We've already repeated  
approaches b) and c) and have just run out of time for these upgrade  
tests because we need to get to other things).

In all cases, the migrate then failed as:

hbase@savory1:~/hbase$ bin/hbase migrate upgrade
08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
available...
08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not  
running...
08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
savory1/10.0.0.45:60000. Already tried 1 time(s).
  ...
08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
savory1/10.0.0.45:60000. Already tried 10 time(s).
08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
java.io.IOException: Install 0.1.x of hbase and run its migration first
        at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)

As you can see, we were then in a bit of a Catch-22.  To re-install  
HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried  
reinstalling HBase-0.1.x without doing that!) so there was no way to  
proceed.  Attempting to start up HBase-0.2.0 just resulted in an error  
message that we needed to do the migrate.

3) Since this was just a test, we then blew away the disk used by  
Hadoop and re-built the namenode per a standard Hadoop new install.   
Hadoop-0.17.1 and Hbase-0.2.0 then started up just fine.  We only ran  
a few tests with the new Hbase command line shell in some ways we used  
the old HQL shell for sanity checks, and everything seems copacetic.

A few other comments:

- The new shell takes a bit of getting used to, but seems quite  
functional (we're not the biggest Ruby fans, but hey, someone took  
this on and upgraded the shell so we just say: Thanks!)

- We really like how timestamps have become first-class objects in the  
HBase-0.2.0 API .  Although, we were in the middle of developing some  
code under HBase-0.1.3 with workarounds for timestamps not being first- 
class objects and we will have to decide whether we should backup and  
re-develop for HBase-0.2.0 (we know we should), or plunge ahead with  
what we were doing under HBase-0.1.3 just to discard it in the near  
future because of the other advantages of HBase-0.2.0.  Is there  
anything we should consider in making this decision, perhaps about  
timing of any bug fixes and an official release of HBase-0.2.0?  
(HBase-0.2.1?).

Thanks for what continues to be a very useful and interesting product.

Rick

On Jul 22, 2008, at 3:24 PM, stack wrote:

> The first 0.2.0 release candidate is available for download:
>
> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
>
> Please take this release candidate for a spin. Check the  
> documentation, that unit tests all complete on your platform, etc.
>
> Should we release this candidate as hbase 0.2.0?  Vote yes or no  
> before Friday, July 25th.
>
> Release 0.2.0 has over 240 issues resolved [1] since the branch for  
> 0.1 hbase was made.  Be warned that hbase 0.2.0 is not backward  
> compatible with the hbase 0.1 API.  See [2] Izaak Rubins' notes on  
> the high-level API differences between 0.1 and 0.2.  For notes on  
> how to migrate your 0.1 era hbase data to 0.2, see Izaak's migration  
> guide [3].
>
> Yours,
> The HBase Team
>
> 1. https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Billy Pearson <sa...@pearsonwholesale.com>.
I got my MR job running again not sure what I did to fix it but I have it 
working in current trunk

Billy

"stack" <st...@duboce.net> wrote in message 
news:4888B880.4070905@duboce.net...
> Billy Pearson wrote:
>> ...
>>
>> Maybe this is a api triping me up and eclipse is not finding the change 
>> with the new jar for 0.2.0
> Incompatible classes whose serialization/deserializations don't matchup 
> can manifest as socket timeout.
>
>
>> I have a map class that has below code shortened
>>
>> public HBaseConfiguration c = new HBaseConfiguration();
>>
>> public static HTable getTable(HBaseConfiguration c) throws IOException {
>>  c.set("hbase.master", CompSpySet.hbasemaster);
>>  HTable table = new HTable(c, "webdata");
>>  return table;
>> }
>>
>> map{
>>
>> the map splits the record into row,column,timestamp,data
>> then it builds a backUpdate called update
>> then the map calls the above code
>>
>> HTable table = getTable(c);
>> table.commit(update);
>> }
> Your hadoop instance has the new hbase 0.2 jars in its CLASSPATH?  Your 
> not mistakenly pointing at an older hbase?
>
>>
>> not sure whats going on this import class it was working about 2 weeks 
>> ago and I have not had any luck in the last 4-5  days sense
>> I updated to latest trunk and tryed RC 1 but eclipse is not showing any 
>> errors or warring.
> Lets try and figure out whats going on Billy,
> St.Ack
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
Billy Pearson wrote:
> ...
>
> Maybe this is a api triping me up and eclipse is not finding the 
> change with the new jar for 0.2.0
Incompatible classes whose serialization/deserializations don't matchup 
can manifest as socket timeout.


> I have a map class that has below code shortened
>
> public HBaseConfiguration c = new HBaseConfiguration();
>
> public static HTable getTable(HBaseConfiguration c) throws IOException {
>  c.set("hbase.master", CompSpySet.hbasemaster);
>  HTable table = new HTable(c, "webdata");
>  return table;
> }
>
> map{
>
> the map splits the record into row,column,timestamp,data
> then it builds a backUpdate called update
> then the map calls the above code
>
> HTable table = getTable(c);
> table.commit(update);
> }
Your hadoop instance has the new hbase 0.2 jars in its CLASSPATH?  Your 
not mistakenly pointing at an older hbase?

>
> not sure whats going on this import class it was working about 2 weeks 
> ago and I have not had any luck in the last 4-5  days sense
> I updated to latest trunk and tryed RC 1 but eclipse is not showing 
> any errors or warring. 
Lets try and figure out whats going on Billy,
St.Ack

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Billy Pearson <sa...@pearsonwholesale.com>.
Master is online I can put one column and scan it form all servers
Restart did not help the MR job
I can not find anything in the logs to tell me there is an error except the 
MR error log for the tasks
The MR jobs fail on all server including the master I run a tasktracker on 
the master.
updated to trunk just to make sure same problem

Maybe this is a api triping me up and eclipse is not finding the change with 
the new jar for 0.2.0

I have a map class that has below code shortened

 public HBaseConfiguration c = new HBaseConfiguration();

 public static HTable getTable(HBaseConfiguration c) throws IOException {
  c.set("hbase.master", CompSpySet.hbasemaster);
  HTable table = new HTable(c, "webdata");
  return table;
 }

map{

the map splits the record into row,column,timestamp,data
then it builds a backUpdate called update
then the map calls the above code

HTable table = getTable(c);
table.commit(update);
}

not sure whats going on this import class it was working about 2 weeks ago 
and I have not had any luck in the last 4-5  days sense
I updated to latest trunk and tryed RC 1 but eclipse is not showing any 
errors or warring.

Billy

"stack" <st...@duboce.net> wrote in message 
news:4887BA79.1080407@duboce.net...
> The below happens for every task?  None can see the master?  You verify 
> the master is running via shell or something?  There could be something 
> going on here.  I heard a second-report of such a phenomeon where there is 
> a timeout though master seems to be listening fine.  Does a restart of MR 
> and hbase clusters change the story?  Any more clues that you can figure 
> Billy?
>
> I ain't sure how HBASE-770 could have fixed your problem.  It did touch 
> the area that was throwing the exception you reported earlier.
>
> St.Ack
>
>
> Billy Pearson wrote:
>> I got the errors after untaring the RC1 for 0.2.0 and running a MR job on 
>> it
>>
>> I download trunk now and it seams the errors went away
>> I thank that HBASE-770 fixed my first errors
>> but now I am getting a different error
>>
>> 2008-07-23 17:30:40,300 INFO 
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 0 
>> of 5 failed with <java.net.SocketTimeoutException: timed out waiting for 
>> rpc response>. Retrying after sleep of 10000
>> 2008-07-23 17:31:50,307 INFO 
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 1 
>> of 5 failed with <java.net.SocketTimeoutException: timed out waiting for 
>> rpc response>. Retrying after sleep of 10000
>> 2008-07-23 17:33:00,312 INFO 
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 2 
>> of 5 failed with <java.net.SocketTimeoutException: timed out waiting for 
>> rpc response>. Retrying after sleep of 10000
>> 2008-07-23 17:34:10,318 INFO 
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 3 
>> of 5 failed with <java.net.SocketTimeoutException: timed out waiting for 
>> rpc response>. Retrying after sleep of 10000
>> 2008-07-23 17:35:20,391 WARN org.apache.hadoop.mapred.TaskTracker: Error 
>> running child
>> org.apache.hadoop.hbase.MasterNotRunningException: xx.xx.xx.xx:60000
>> at 
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:219)
>> at 
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:431)
>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
>> at 
>> com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
>> at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>>
>> I commented out the ip above it was correct and the port is correct I 
>> check and the master is a live and well when I get these errors when 
>> running a MR job to import records.
>>
>> Billy
>>
>>
>>
>> "Jean-Daniel Cryans" <jd...@gmail.com> 
>> wrote in message 
>> news:31a243e70807231413n21cfda44y2728705566d84c31@mail.gmail.com...
>>> Billy,
>>>
>>> I looked at the code where the exceptions were thrown and something is
>>> weird. For the NPE, the line 291 in HbaseObjectWritable looks like:
>>> public static Object readObject(DataInput in,
>>>      HbaseObjectWritable objectWritable, Configuration conf)
>>>  throws IOException {
>>>    Class<?> declaredClass = CODE_TO_CLASS.get(in.readByte());
>>>    Object instance;
>>>    *if (declaredClass.isPrimitive()) {            // primitive types*
>>>      if (declaredClass == Boolean.TYPE) {             // boolean
>>>        instance = Boolean.valueOf(in.readBoolean());
>>>
>>> The chance that declaredClass would be null is low. Also, at line 821 of 
>>> HCM
>>> regards the other exception it reads:
>>>
>>> server.getRegionInfo(HRegionInfo.ROOT_REGIONINFO.getRegionName());
>>>          *if (LOG.isDebugEnabled()) {*
>>>            LOG.debug("Found ROOT " + HRegionInfo.ROOT_REGIONINFO);
>>>          }
>>>
>>> So I clearly see that the " at $Proxy3.getRegionInfo(Unknown Source)" 
>>> the
>>> came after that is not called here. So I'm wondering what version of 
>>> HBase
>>> exactly  are you running?
>>>
>>> Thx for looking at this and thx for testing!
>>>
>>> J-D
>>>
>>> On Tue, Jul 22, 2008 at 10:10 PM, Billy Pearson 
>>> <sa...@pearsonwholesale.com>
>>> wrote:
>>>
>>>> I get this when trying to run a mr job on the Release Candidate
>>>>
>>>> 2008-07-22 21:05:00,237 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>>>> Initializing JVM Metrics with processName=MAP, sessionId=
>>>> 2008-07-22 21:05:00,254 WARN org.apache.hadoop.fs.FileSystem: "
>>>> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
>>>> 64.69.33.145:9000/" instead.
>>>> 2008-07-22 21:05:00,378 INFO org.apache.hadoop.mapred.MapTask:
>>>> numReduceTasks: 0
>>>> 2008-07-22 21:05:00,379 WARN org.apache.hadoop.fs.FileSystem: "
>>>> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
>>>> 64.69.33.145:9000/" instead.
>>>> 2008-07-22 21:05:00,797 INFO org.apache.hadoop.ipc.Client:
>>>> java.lang.NullPointerException
>>>> at
>>>> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:291)
>>>> at
>>>> org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:166)
>>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:306)
>>>>
>>>> 2008-07-22 21:06:00,804 WARN org.apache.hadoop.mapred.TaskTracker: 
>>>> Error
>>>> running child
>>>> java.lang.reflect.UndeclaredThrowableException
>>>> at $Proxy3.getRegionInfo(Unknown Source)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:821)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:458)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:440)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:575)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:468)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:511)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:472)
>>>> at
>>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
>>>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
>>>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
>>>> at 
>>>> com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
>>>> at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
>>>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>>>> at 
>>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>>> response
>>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>>> at 
>>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:213)
>>>> ... 17 more
>>>>
>>>>
>>>> not sure what the problem is here same job I been running for over a 
>>>> month
>>>> and now today I get this from the EC1 and Trunk on clean installs of 
>>>> hadoop
>>>> 0.17.1 and hbase
>>>>
>>>> Billy
>>>>
>>>> "stack" <st...@duboce.net> wrote in 
>>>> message
>>>> news:48865E15.3010507@duboce.net...
>>>>
>>>>  The first 0.2.0 release candidate is available for download:
>>>>>
>>>>>
>>>>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>>>>
>>>>> Please take this release candidate for a spin. Check the 
>>>>> documentation,
>>>>> that unit tests all complete on your platform, etc.
>>>>>
>>>>> Should we release this candidate as hbase 0.2.0?  Vote yes or no 
>>>>> before
>>>>> Friday, July 25th.
>>>>>
>>>>> Release 0.2.0 has over 240 issues resolved [1] since the branch for 
>>>>> 0.1
>>>>> hbase was made.  Be warned that hbase 0.2.0 is not backward compatible 
>>>>> with
>>>>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API
>>>>> differences between 0.1 and 0.2.  For notes on how to migrate your 0.1 
>>>>> era
>>>>> hbase data to 0.2, see Izaak's migration guide [3].
>>>>>
>>>>> Yours,
>>>>> The HBase Team
>>>>>
>>>>> 1.
>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>>>>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>>>>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
Andrew Purtell wrote:
>> Should we up the default lease period from 60 to 90 or 120
>> seconds?
>>     
>
> Maybe. Increasing the lease period might just move the problem around. On the other hand this is the prescription already for a heavily loaded cluster. 
>
>   
Unless any objection, I'll change the default to 120 for RC2.  In 
hadoop, the roughly equivalent interval, 
mapred.tasktracker.expiry.interval, is ten minutes.  Will make hbase 
more sluggish recognizing downed regionservers but more robust against 
the occasional hogging process running on same host.

We can revisit after we integrate zookeeper (hbase-546).

St.Ack

>> St.Ack
>>
>>     
>>>> 1) In master log, lease timeout notices.
>>>>
>>>> 2) In regionserver logs, quiesce/restart.
>>>>     
>>>> 3) In master log, errors related to META going away.
>>>>
>>>> 4) In master log, reassignment of META to a regionserver
>>>> still up.
>>>>
>>>> 5) In master log, new start messages from the regionservers
>>>> that have finished restarting.
>>>>
>>>> 6) Import continues (eventually).
>>>>
>>>> Before #1 there are no errors or anything out of the
>>>> ordinary in either the master or regionserver logs. 
>>>>
>>>> I upped the regionserver lease period from 60 to 120
>>>> seconds, reinitialized, and ran the import again. No
>>>> problems.
>>>>
>>>>    - Andy
>>>>         
>
>
>
>       
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Andrew Purtell <ap...@yahoo.com>.
Comments inline.

> From: stack <st...@duboce.net>
> Subject: Re: [ANN] hbase-0.2.0 Release Candidate 1
> To: hbase-user@hadoop.apache.org
> Date: Thursday, July 24, 2008, 10:39 AM
>
> Andrew Purtell wrote:
> > This was before you committed the HRPC update stuff.
> >
> > I'm running that code now, no problems. 
> >   
> With or without the upped lease period?

With.
 
> Should we up the default lease period from 60 to 90 or 120
> seconds?

Maybe. Increasing the lease period might just move the problem around. On the other hand this is the prescription already for a heavily loaded cluster. 

> Regards the regionserver crash you saw below, any instances
> of HBASE-616 messages in the regionserver logs?

No, which was why the lease timeout/expiration messages appearing out of the blue in the master log was so odd. 

> St.Ack
> 
> >>
> >> 1) In master log, lease timeout notices.
> >>
> >> 2) In regionserver logs, quiesce/restart.
> >>     
> >> 3) In master log, errors related to META going away.
> >>
> >> 4) In master log, reassignment of META to a regionserver
> >> still up.
> >>
> >> 5) In master log, new start messages from the regionservers
> >> that have finished restarting.
> >>
> >> 6) Import continues (eventually).
> >>
> >> Before #1 there are no errors or anything out of the
> >> ordinary in either the master or regionserver logs. 
> >>
> >> I upped the regionserver lease period from 60 to 120
> >> seconds, reinitialized, and ran the import again. No
> >> problems.
> >>
> >>    - Andy

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
Andrew Purtell wrote:
> This was before you committed the HRPC update stuff.
>
> I'm running that code now, no problems. 
>   
With or without the upped lease period?

Should we up the default lease period from 60 to 90 or 120 seconds?

Regards the regionserver crash you saw below, any instances of HBASE-616 
messages in the regionserver logs?

St.Ack

>>
>> 1) In master log, lease timeout notices.
>>
>> 2) In regionserver logs, quiesce/restart.
>>     


>> 3) In master log, errors related to META going away.
>>
>> 4) In master log, reassignment of META to a regionserver
>> still up.
>>
>> 5) In master log, new start messages from the regionservers
>> that have finished restarting.
>>
>> 6) Import continues (eventually).
>>
>> Before #1 there are no errors or anything out of the
>> ordinary in either the master or regionserver logs. 
>>
>> I upped the regionserver lease period from 60 to 120
>> seconds, reinitialized, and ran the import again. No
>> problems.
>>
>>    - Andy
>>
>>
>> --- On Wed, 7/23/08, stack <st...@duboce.net> wrote:
>>
>>     
>>> From: stack <st...@duboce.net>
>>> Subject: Re: [ANN] hbase-0.2.0 Release Candidate 1
>>> To: hbase-user@hadoop.apache.org
>>> Date: Wednesday, July 23, 2008, 4:10 PM
>>> The below happens for every task?  None can see the
>>>       
>> master? 
>>     
>>> You verify the master is running via shell or
>>>       
>> something?
>>     
>>> There could be something going on here.  I heard a
>>>       
>> second-
>>     
>>> report of such a phenomeon where there is a timeout
>>>       
>> though
>>     
>>> master seems to be listening fine.
>>>       
>
>
>       
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Andrew Purtell <ap...@yahoo.com>.
This was before you committed the HRPC update stuff.

I'm running that code now, no problems. 

   - Andy


--- On Wed, 7/23/08, Andrew Purtell <ap...@yahoo.com> wrote:

> From: Andrew Purtell <ap...@yahoo.com>
> Subject: Re: [ANN] hbase-0.2.0 Release Candidate 1
> To: hbase-user@hadoop.apache.org
> Date: Wednesday, July 23, 2008, 4:59 PM
> I have a third datapoint. A database import here failed
> today when the master timed out the leases of some
> regionservers (5 out of 16), including the one hosting
> META. The regionservers restarted and eventually things
> came back. There was little in the logs to go on. The
> progression was:
> 
> 1) In master log, lease timeout notices.
> 
> 2) In regionserver logs, quiesce/restart.
> 
> 3) In master log, errors related to META going away.
> 
> 4) In master log, reassignment of META to a regionserver
> still up.
> 
> 5) In master log, new start messages from the regionservers
> that have finished restarting.
> 
> 6) Import continues (eventually).
> 
> Before #1 there are no errors or anything out of the
> ordinary in either the master or regionserver logs. 
> 
> I upped the regionserver lease period from 60 to 120
> seconds, reinitialized, and ran the import again. No
> problems.
> 
>    - Andy
> 
> 
> --- On Wed, 7/23/08, stack <st...@duboce.net> wrote:
> 
> > From: stack <st...@duboce.net>
> > Subject: Re: [ANN] hbase-0.2.0 Release Candidate 1
> > To: hbase-user@hadoop.apache.org
> > Date: Wednesday, July 23, 2008, 4:10 PM
> > The below happens for every task?  None can see the
> master? 
> > You verify the master is running via shell or
> something?
> > There could be something going on here.  I heard a
> second-
> > report of such a phenomeon where there is a timeout
> though
> > master seems to be listening fine.

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Andrew Purtell <ap...@yahoo.com>.
I have a third datapoint. A database import here failed today when the master timed out the leases of some regionservers (5 out of 16), including the one hosting META. The regionservers restarted and eventually things came back. There was little in the logs to go on. The progression was:

1) In master log, lease timeout notices.

2) In regionserver logs, quiesce/restart.

3) In master log, errors related to META going away.

4) In master log, reassignment of META to a regionserver still up.

5) In master log, new start messages from the regionservers that have finished restarting.

6) Import continues (eventually).

Before #1 there are no errors or anything out of the ordinary in either the master or regionserver logs. 

I upped the regionserver lease period from 60 to 120 seconds, reinitialized, and ran the import again. No problems.

   - Andy


--- On Wed, 7/23/08, stack <st...@duboce.net> wrote:

> From: stack <st...@duboce.net>
> Subject: Re: [ANN] hbase-0.2.0 Release Candidate 1
> To: hbase-user@hadoop.apache.org
> Date: Wednesday, July 23, 2008, 4:10 PM
> The below happens for every task?  None can see the master? 
> You verify the master is running via shell or something?
> There could be something going on here.  I heard a second-
> report of such a phenomeon where there is a timeout though
> master seems to be listening fine.

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
The below happens for every task?  None can see the master?  You verify 
the master is running via shell or something?  There could be something 
going on here.  I heard a second-report of such a phenomeon where there 
is a timeout though master seems to be listening fine.  Does a restart 
of MR and hbase clusters change the story?  Any more clues that you can 
figure Billy?

I ain't sure how HBASE-770 could have fixed your problem.  It did touch 
the area that was throwing the exception you reported earlier.

St.Ack


Billy Pearson wrote:
> I got the errors after untaring the RC1 for 0.2.0 and running a MR job 
> on it
>
> I download trunk now and it seams the errors went away
> I thank that HBASE-770 fixed my first errors
> but now I am getting a different error
>
> 2008-07-23 17:30:40,300 INFO 
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: 
> Attempt 0 of 5 failed with <java.net.SocketTimeoutException: timed out 
> waiting for rpc response>. Retrying after sleep of 10000
> 2008-07-23 17:31:50,307 INFO 
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: 
> Attempt 1 of 5 failed with <java.net.SocketTimeoutException: timed out 
> waiting for rpc response>. Retrying after sleep of 10000
> 2008-07-23 17:33:00,312 INFO 
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: 
> Attempt 2 of 5 failed with <java.net.SocketTimeoutException: timed out 
> waiting for rpc response>. Retrying after sleep of 10000
> 2008-07-23 17:34:10,318 INFO 
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: 
> Attempt 3 of 5 failed with <java.net.SocketTimeoutException: timed out 
> waiting for rpc response>. Retrying after sleep of 10000
> 2008-07-23 17:35:20,391 WARN org.apache.hadoop.mapred.TaskTracker: 
> Error running child
> org.apache.hadoop.hbase.MasterNotRunningException: xx.xx.xx.xx:60000
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:219) 
>
> at 
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:431) 
>
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
> at 
> com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
> at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>
> I commented out the ip above it was correct and the port is correct I 
> check and the master is a live and well when I get these errors when 
> running a MR job to import records.
>
> Billy
>
>
>
> "Jean-Daniel Cryans" <jd...@gmail.com> wrote in message 
> news:31a243e70807231413n21cfda44y2728705566d84c31@mail.gmail.com...
>> Billy,
>>
>> I looked at the code where the exceptions were thrown and something is
>> weird. For the NPE, the line 291 in HbaseObjectWritable looks like:
>> public static Object readObject(DataInput in,
>>      HbaseObjectWritable objectWritable, Configuration conf)
>>  throws IOException {
>>    Class<?> declaredClass = CODE_TO_CLASS.get(in.readByte());
>>    Object instance;
>>    *if (declaredClass.isPrimitive()) {            // primitive types*
>>      if (declaredClass == Boolean.TYPE) {             // boolean
>>        instance = Boolean.valueOf(in.readBoolean());
>>
>> The chance that declaredClass would be null is low. Also, at line 821 
>> of HCM
>> regards the other exception it reads:
>>
>> server.getRegionInfo(HRegionInfo.ROOT_REGIONINFO.getRegionName());
>>          *if (LOG.isDebugEnabled()) {*
>>            LOG.debug("Found ROOT " + HRegionInfo.ROOT_REGIONINFO);
>>          }
>>
>> So I clearly see that the " at $Proxy3.getRegionInfo(Unknown Source)" 
>> the
>> came after that is not called here. So I'm wondering what version of 
>> HBase
>> exactly  are you running?
>>
>> Thx for looking at this and thx for testing!
>>
>> J-D
>>
>> On Tue, Jul 22, 2008 at 10:10 PM, Billy Pearson 
>> <sa...@pearsonwholesale.com>
>> wrote:
>>
>>> I get this when trying to run a mr job on the Release Candidate
>>>
>>> 2008-07-22 21:05:00,237 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>>> Initializing JVM Metrics with processName=MAP, sessionId=
>>> 2008-07-22 21:05:00,254 WARN org.apache.hadoop.fs.FileSystem: "
>>> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
>>> 64.69.33.145:9000/" instead.
>>> 2008-07-22 21:05:00,378 INFO org.apache.hadoop.mapred.MapTask:
>>> numReduceTasks: 0
>>> 2008-07-22 21:05:00,379 WARN org.apache.hadoop.fs.FileSystem: "
>>> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
>>> 64.69.33.145:9000/" instead.
>>> 2008-07-22 21:05:00,797 INFO org.apache.hadoop.ipc.Client:
>>> java.lang.NullPointerException
>>> at
>>> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:291) 
>>>
>>> at
>>> org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:166) 
>>>
>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:306)
>>>
>>> 2008-07-22 21:06:00,804 WARN org.apache.hadoop.mapred.TaskTracker: 
>>> Error
>>> running child
>>> java.lang.reflect.UndeclaredThrowableException
>>> at $Proxy3.getRegionInfo(Unknown Source)
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:821) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:458) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:440) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:575) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:468) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:511) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:472) 
>>>
>>> at
>>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432) 
>>>
>>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
>>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
>>> at 
>>> com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
>>> at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
>>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>>> at 
>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>>> response
>>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>>> at 
>>> org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:213)
>>> ... 17 more
>>>
>>>
>>> not sure what the problem is here same job I been running for over a 
>>> month
>>> and now today I get this from the EC1 and Trunk on clean installs of 
>>> hadoop
>>> 0.17.1 and hbase
>>>
>>> Billy
>>>
>>> "stack" <st...@duboce.net> wrote in message
>>> news:48865E15.3010507@duboce.net...
>>>
>>>  The first 0.2.0 release candidate is available for download:
>>>>
>>>>
>>>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/> 
>>>>
>>>>
>>>> Please take this release candidate for a spin. Check the 
>>>> documentation,
>>>> that unit tests all complete on your platform, etc.
>>>>
>>>> Should we release this candidate as hbase 0.2.0?  Vote yes or no 
>>>> before
>>>> Friday, July 25th.
>>>>
>>>> Release 0.2.0 has over 240 issues resolved [1] since the branch for 
>>>> 0.1
>>>> hbase was made.  Be warned that hbase 0.2.0 is not backward 
>>>> compatible with
>>>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API
>>>> differences between 0.1 and 0.2.  For notes on how to migrate your 
>>>> 0.1 era
>>>> hbase data to 0.2, see Izaak's migration guide [3].
>>>>
>>>> Yours,
>>>> The HBase Team
>>>>
>>>> 1.
>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create 
>>>>
>>>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>>>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>>
>>>>
>>>
>>>
>>
>
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Billy Pearson <sa...@pearsonwholesale.com>.
I got the errors after untaring the RC1 for 0.2.0 and running a MR job on it

I download trunk now and it seams the errors went away
I thank that HBASE-770 fixed my first errors
but now I am getting a different error

2008-07-23 17:30:40,300 INFO 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 0 of 
5 failed with <java.net.SocketTimeoutException: timed out waiting for rpc 
response>. Retrying after sleep of 10000
2008-07-23 17:31:50,307 INFO 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 1 of 
5 failed with <java.net.SocketTimeoutException: timed out waiting for rpc 
response>. Retrying after sleep of 10000
2008-07-23 17:33:00,312 INFO 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 2 of 
5 failed with <java.net.SocketTimeoutException: timed out waiting for rpc 
response>. Retrying after sleep of 10000
2008-07-23 17:34:10,318 INFO 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Attempt 3 of 
5 failed with <java.net.SocketTimeoutException: timed out waiting for rpc 
response>. Retrying after sleep of 10000
2008-07-23 17:35:20,391 WARN org.apache.hadoop.mapred.TaskTracker: Error 
running child
org.apache.hadoop.hbase.MasterNotRunningException: xx.xx.xx.xx:60000
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getMaster(HConnectionManager.java:219)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:431)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
 at com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
 at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
 at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)

I commented out the ip above it was correct and the port is correct I check 
and the master is a live and well when I get these errors when running a MR 
job to import records.

Billy



"Jean-Daniel Cryans" <jd...@gmail.com> 
wrote in message 
news:31a243e70807231413n21cfda44y2728705566d84c31@mail.gmail.com...
> Billy,
>
> I looked at the code where the exceptions were thrown and something is
> weird. For the NPE, the line 291 in HbaseObjectWritable looks like:
> public static Object readObject(DataInput in,
>      HbaseObjectWritable objectWritable, Configuration conf)
>  throws IOException {
>    Class<?> declaredClass = CODE_TO_CLASS.get(in.readByte());
>    Object instance;
>    *if (declaredClass.isPrimitive()) {            // primitive types*
>      if (declaredClass == Boolean.TYPE) {             // boolean
>        instance = Boolean.valueOf(in.readBoolean());
>
> The chance that declaredClass would be null is low. Also, at line 821 of 
> HCM
> regards the other exception it reads:
> 
> server.getRegionInfo(HRegionInfo.ROOT_REGIONINFO.getRegionName());
>          *if (LOG.isDebugEnabled()) {*
>            LOG.debug("Found ROOT " + HRegionInfo.ROOT_REGIONINFO);
>          }
>
> So I clearly see that the " at $Proxy3.getRegionInfo(Unknown Source)" the
> came after that is not called here. So I'm wondering what version of HBase
> exactly  are you running?
>
> Thx for looking at this and thx for testing!
>
> J-D
>
> On Tue, Jul 22, 2008 at 10:10 PM, Billy Pearson 
> <sa...@pearsonwholesale.com>
> wrote:
>
>> I get this when trying to run a mr job on the Release Candidate
>>
>> 2008-07-22 21:05:00,237 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>> Initializing JVM Metrics with processName=MAP, sessionId=
>> 2008-07-22 21:05:00,254 WARN org.apache.hadoop.fs.FileSystem: "
>> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
>> 64.69.33.145:9000/" instead.
>> 2008-07-22 21:05:00,378 INFO org.apache.hadoop.mapred.MapTask:
>> numReduceTasks: 0
>> 2008-07-22 21:05:00,379 WARN org.apache.hadoop.fs.FileSystem: "
>> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
>> 64.69.33.145:9000/" instead.
>> 2008-07-22 21:05:00,797 INFO org.apache.hadoop.ipc.Client:
>> java.lang.NullPointerException
>> at
>> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:291)
>> at
>> org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:166)
>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:306)
>>
>> 2008-07-22 21:06:00,804 WARN org.apache.hadoop.mapred.TaskTracker: Error
>> running child
>> java.lang.reflect.UndeclaredThrowableException
>> at $Proxy3.getRegionInfo(Unknown Source)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:821)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:458)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:440)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:575)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:468)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:511)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:472)
>> at
>> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
>> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
>> at 
>> com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
>> at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
>> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
>> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
>> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
>> response
>> at org.apache.hadoop.ipc.Client.call(Client.java:559)
>> at org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:213)
>> ... 17 more
>>
>>
>> not sure what the problem is here same job I been running for over a 
>> month
>> and now today I get this from the EC1 and Trunk on clean installs of 
>> hadoop
>> 0.17.1 and hbase
>>
>> Billy
>>
>> "stack" <st...@duboce.net> wrote in message
>> news:48865E15.3010507@duboce.net...
>>
>>  The first 0.2.0 release candidate is available for download:
>>>
>>> 
>>> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>>
>>> Please take this release candidate for a spin. Check the documentation,
>>> that unit tests all complete on your platform, etc.
>>>
>>> Should we release this candidate as hbase 0.2.0?  Vote yes or no before
>>> Friday, July 25th.
>>>
>>> Release 0.2.0 has over 240 issues resolved [1] since the branch for 0.1
>>> hbase was made.  Be warned that hbase 0.2.0 is not backward compatible 
>>> with
>>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API
>>> differences between 0.1 and 0.2.  For notes on how to migrate your 0.1 
>>> era
>>> hbase data to 0.2, see Izaak's migration guide [3].
>>>
>>> Yours,
>>> The HBase Team
>>>
>>> 1.
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>>
>>>
>>
>>
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Jean-Daniel Cryans <jd...@gmail.com>.
Billy,

I looked at the code where the exceptions were thrown and something is
weird. For the NPE, the line 291 in HbaseObjectWritable looks like:
 public static Object readObject(DataInput in,
      HbaseObjectWritable objectWritable, Configuration conf)
  throws IOException {
    Class<?> declaredClass = CODE_TO_CLASS.get(in.readByte());
    Object instance;
    *if (declaredClass.isPrimitive()) {            // primitive types*
      if (declaredClass == Boolean.TYPE) {             // boolean
        instance = Boolean.valueOf(in.readBoolean());

The chance that declaredClass would be null is low. Also, at line 821 of HCM
regards the other exception it reads:
          server.getRegionInfo(HRegionInfo.ROOT_REGIONINFO.getRegionName());
          *if (LOG.isDebugEnabled()) {*
            LOG.debug("Found ROOT " + HRegionInfo.ROOT_REGIONINFO);
          }

So I clearly see that the " at $Proxy3.getRegionInfo(Unknown Source)" the
came after that is not called here. So I'm wondering what version of HBase
exactly  are you running?

Thx for looking at this and thx for testing!

J-D

On Tue, Jul 22, 2008 at 10:10 PM, Billy Pearson <sa...@pearsonwholesale.com>
wrote:

> I get this when trying to run a mr job on the Release Candidate
>
> 2008-07-22 21:05:00,237 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
> Initializing JVM Metrics with processName=MAP, sessionId=
> 2008-07-22 21:05:00,254 WARN org.apache.hadoop.fs.FileSystem: "
> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
> 64.69.33.145:9000/" instead.
> 2008-07-22 21:05:00,378 INFO org.apache.hadoop.mapred.MapTask:
> numReduceTasks: 0
> 2008-07-22 21:05:00,379 WARN org.apache.hadoop.fs.FileSystem: "
> 64.69.33.145:9000" is a deprecated filesystem name. Use "hdfs://
> 64.69.33.145:9000/" instead.
> 2008-07-22 21:05:00,797 INFO org.apache.hadoop.ipc.Client:
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:291)
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:166)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:306)
>
> 2008-07-22 21:06:00,804 WARN org.apache.hadoop.mapred.TaskTracker: Error
> running child
> java.lang.reflect.UndeclaredThrowableException
> at $Proxy3.getRegionInfo(Unknown Source)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:821)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:458)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:440)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:575)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:468)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:511)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:472)
> at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
> at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
> at com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
> at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
> Caused by: java.net.SocketTimeoutException: timed out waiting for rpc
> response
> at org.apache.hadoop.ipc.Client.call(Client.java:559)
> at org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:213)
> ... 17 more
>
>
> not sure what the problem is here same job I been running for over a month
> and now today I get this from the EC1 and Trunk on clean installs of hadoop
> 0.17.1 and hbase
>
> Billy
>
> "stack" <st...@duboce.net> wrote in message
> news:48865E15.3010507@duboce.net...
>
>  The first 0.2.0 release candidate is available for download:
>>
>>  http://people.apache.org/~stack/hbase-0.2.0-candidate-1/<http://people.apache.org/%7Estack/hbase-0.2.0-candidate-1/>
>>
>> Please take this release candidate for a spin. Check the documentation,
>> that unit tests all complete on your platform, etc.
>>
>> Should we release this candidate as hbase 0.2.0?  Vote yes or no before
>> Friday, July 25th.
>>
>> Release 0.2.0 has over 240 issues resolved [1] since the branch for 0.1
>> hbase was made.  Be warned that hbase 0.2.0 is not backward compatible with
>> the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API
>> differences between 0.1 and 0.2.  For notes on how to migrate your 0.1 era
>> hbase data to 0.2, see Izaak's migration guide [3].
>>
>> Yours,
>> The HBase Team
>>
>> 1.
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
>> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
>> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>>
>>
>
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Billy Pearson <sa...@pearsonwholesale.com>.
I get this when trying to run a mr job on the Release Candidate

2008-07-22 21:05:00,237 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: 
Initializing JVM Metrics with processName=MAP, sessionId=
2008-07-22 21:05:00,254 WARN org.apache.hadoop.fs.FileSystem: 
"64.69.33.145:9000" is a deprecated filesystem name. Use 
"hdfs://64.69.33.145:9000/" instead.
2008-07-22 21:05:00,378 INFO org.apache.hadoop.mapred.MapTask: 
numReduceTasks: 0
2008-07-22 21:05:00,379 WARN org.apache.hadoop.fs.FileSystem: 
"64.69.33.145:9000" is a deprecated filesystem name. Use 
"hdfs://64.69.33.145:9000/" instead.
2008-07-22 21:05:00,797 INFO org.apache.hadoop.ipc.Client: 
java.lang.NullPointerException
 at 
org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:291)
 at 
org.apache.hadoop.hbase.io.HbaseObjectWritable.readFields(HbaseObjectWritable.java:166)
 at org.apache.hadoop.ipc.Client$Connection.run(Client.java:306)

2008-07-22 21:06:00,804 WARN org.apache.hadoop.mapred.TaskTracker: Error 
running child
java.lang.reflect.UndeclaredThrowableException
 at $Proxy3.getRegionInfo(Unknown Source)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:821)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:458)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:440)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:575)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:468)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:511)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:472)
 at 
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:432)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:124)
 at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:109)
 at com.compspy.mapred.RecordImport$MapClass.getTable(RecordImport.java:50)
 at com.compspy.mapred.RecordImport$MapClass.map(RecordImport.java:76)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
 at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2124)
Caused by: java.net.SocketTimeoutException: timed out waiting for rpc 
response
 at org.apache.hadoop.ipc.Client.call(Client.java:559)
 at org.apache.hadoop.hbase.ipc.HbaseRPC$Invoker.invoke(HbaseRPC.java:213)
 ... 17 more


not sure what the problem is here same job I been running for over a month 
and now today I get this from the EC1 and Trunk on clean installs of hadoop 
0.17.1 and hbase

Billy

"stack" <st...@duboce.net> wrote in message 
news:48865E15.3010507@duboce.net...
> The first 0.2.0 release candidate is available for download:
>
>  http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
>
> Please take this release candidate for a spin. Check the documentation, 
> that unit tests all complete on your platform, etc.
>
> Should we release this candidate as hbase 0.2.0?  Vote yes or no before 
> Friday, July 25th.
>
> Release 0.2.0 has over 240 issues resolved [1] since the branch for 0.1 
> hbase was made.  Be warned that hbase 0.2.0 is not backward compatible 
> with the hbase 0.1 API.  See [2] Izaak Rubins' notes on the high-level API 
> differences between 0.1 and 0.2.  For notes on how to migrate your 0.1 era 
> hbase data to 0.2, see Izaak's migration guide [3].
>
> Yours,
> The HBase Team
>
> 1. 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Rick Hangartner <ha...@strands.com>.
Hi,

All in all, we like what we've seen of HBase-0.2.0 so far and the API  
changes.

Our upgrade test with Release Candidate 1 was a little bumpy but  
generally positive.

We set up a single virtualized test machine (in this case VMWare  
Fusion because an OS X machine was close at hand, although we also run  
Xen virtualized installs), with a virtual disk, running hadoop-0.16.4  
and hbase-0.1.3 with a small test table in a Debian install so we  
could rapidly test alternative upgrades paths and data migration.

We tried our own upgrade procedure once based on some previous  
experience and we tried the "formal" upgrade procedure  found at:

http://wiki.apache.org/hadoop/Hbase/HowToMigrate
http://wiki.apache.org/hadoop/Hadoop%20Upgrade

In each case, we of course started with a fresh copy of our  
virtualized machine.

Here's what we found:

1) In both approaches, the upgrade from hadoop-0.16.4 to hadoop-0.17.1  
seem to go OK.

2) In both approaches, when we tried to do the data migration from  
hbase-0.1.3 to hbase-0.2.0 we first got migration failures due to  
"unrecovered region server logs". Following the 'Redo Logs' comments  
in the "http://wiki.apache.org/hadoop/Hbase/HowToMigrate" doc,  
starting afresh with a  new copy of our virtualized system each time,  
we tried these methods of getting rid of the those logs and the fatal  
error:

    a) deleting just the log files in the "/hbase" directory
    b) deleting the entire contents of the "/hbase" directory (which  
means we lost our data, but we are just investigating the upgrade  
path, after all)
    c) deleting the "/hbase" directory entirely and creating a new "/ 
hbase" directory.

I should also note that we would need to repeat approach a) to be 100%  
certain of our results for that case.  (We've already repeated  
approaches b) and c) and have just run out of time for these upgrade  
tests because we need to get to other things).

In all cases, the migrate then failed as:

hbase@savory1:~/hbase$ bin/hbase migrate upgrade
08/07/22 18:03:16 INFO util.Migrate: Verifying that file system is
available...
08/07/22 18:03:16 INFO util.Migrate: Verifying that HBase is not  
running...
08/07/22 18:03:17 INFO ipc.Client: Retrying connect to server:
savory1/10.0.0.45:60000. Already tried 1 time(s).
  ...
08/07/22 18:03:26 INFO ipc.Client: Retrying connect to server:
savory1/10.0.0.45:60000. Already tried 10 time(s).
08/07/22 18:03:27 INFO util.Migrate: Starting upgrade
08/07/22 18:03:27 FATAL util.Migrate: Upgrade failed
java.io.IOException: Install 0.1.x of hbase and run its migration first
        at org.apache.hadoop.hbase.util.Migrate.run(Migrate.java:181)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at org.apache.hadoop.hbase.util.Migrate.main(Migrate.java:446)

As you can see, we were then in a bit of a Catch-22.  To re-install  
HBase-0.1.x also required re-installing Hadoop-0.16.4 (we tried  
reinstalling HBase-0.1.x without doing that!) so there was no way to  
proceed.  Attempting to start up HBase-0.2.0 just resulted in an error  
message that we needed to do the migrate.

3) Since this was just a test, we then blew away the disk used by  
Hadoop and re-built the namenode per a standard Hadoop new install.   
Hadoop-0.17.1 and Hbase-0.2.0 then started up just fine.  We only ran  
a few tests with the new Hbase command line shell in some ways we used  
the old HQL shell for sanity checks, and everything seems copacetic.

A few other comments:

- The new shell takes a bit of getting used to, but seems quite  
functional (we're not the biggest Ruby fans, but hey, someone took  
this on and upgraded the shell so we just say: Thanks!)

- We really like how timestamps have become first-class objects in the  
HBase-0.2.0 API .  Although, we were in the middle of developing some  
code under HBase-0.1.3 with workarounds for timestamps not being first- 
class objects and we will have to decide whether we should backup and  
re-develop for HBase-0.2.0 (we know we should), or plunge ahead with  
what we were doing under HBase-0.1.3 just to discard it in the near  
future because of the other advantages of HBase-0.2.0.  Is there  
anything we should consider in making this decision, perhaps about  
timing of any bug fixes and an official release of HBase-0.2.0?  
(HBase-0.2.1?).

Thanks for what continues to be a very useful and interesting product.

Rick

On Jul 22, 2008, at 3:24 PM, stack wrote:

> The first 0.2.0 release candidate is available for download:
>
> http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
>
> Please take this release candidate for a spin. Check the  
> documentation, that unit tests all complete on your platform, etc.
>
> Should we release this candidate as hbase 0.2.0?  Vote yes or no  
> before Friday, July 25th.
>
> Release 0.2.0 has over 240 issues resolved [1] since the branch for  
> 0.1 hbase was made.  Be warned that hbase 0.2.0 is not backward  
> compatible with the hbase 0.1 API.  See [2] Izaak Rubins' notes on  
> the high-level API differences between 0.1 and 0.2.  For notes on  
> how to migrate your 0.1 era hbase data to 0.2, see Izaak's migration  
> guide [3].
>
> Yours,
> The HBase Team
>
> 1. https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12312955&styleName=Html&projectId=12310753&Create=Create
> 2. http://wiki.apache.org/hadoop/Hbase/Plan-0.2/APIChanges
> 3. http://wiki.apache.org/hadoop/Hbase/HowToMigrate

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by stack <st...@duboce.net>.
OK.  There's a few other issues coming in too that together make a 
sufficient argument for squashing RC1.  I'll put up a new RC tomorrow.  
Meantime, keep the complaints -- and fixes -- coming so we can get them 
into RC2.
Thanks,
St.Ack

Andrew Purtell wrote:
> -1
>
> Please see HBASE-763
>
>
> --- On Tue, 7/22/08, stack <st...@duboce.net> wrote:
>
>   
>> From: stack <st...@duboce.net>
>> Subject: [ANN] hbase-0.2.0 Release Candidate 1
>> To: hbase-user@hadoop.apache.org, hbase-dev@hadoop.apache.org
>> Date: Tuesday, July 22, 2008, 3:24 PM
>> The first 0.2.0 release candidate is available for download:
>>
>>   http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
>>
>> Please take this release candidate for a spin. Check the
>> documentation, that unit tests all complete on your platform, etc.
>>
>> Should we release this candidate as hbase 0.2.0?  Vote yes
>> or no before Friday, July 25th.
>>     
>
>
>
>       
>

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Andrew Purtell <ap...@yahoo.com>.
-1

Please see HBASE-763


--- On Tue, 7/22/08, stack <st...@duboce.net> wrote:

> From: stack <st...@duboce.net>
> Subject: [ANN] hbase-0.2.0 Release Candidate 1
> To: hbase-user@hadoop.apache.org, hbase-dev@hadoop.apache.org
> Date: Tuesday, July 22, 2008, 3:24 PM
> The first 0.2.0 release candidate is available for download:
> 
>   http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
> 
> Please take this release candidate for a spin. Check the
> documentation, that unit tests all complete on your platform, etc.
> 
> Should we release this candidate as hbase 0.2.0?  Vote yes
> or no before Friday, July 25th.

Re: [ANN] hbase-0.2.0 Release Candidate 1

Posted by Andrew Purtell <ap...@yahoo.com>.
-1

Please see HBASE-763


--- On Tue, 7/22/08, stack <st...@duboce.net> wrote:

> From: stack <st...@duboce.net>
> Subject: [ANN] hbase-0.2.0 Release Candidate 1
> To: hbase-user@hadoop.apache.org, hbase-dev@hadoop.apache.org
> Date: Tuesday, July 22, 2008, 3:24 PM
> The first 0.2.0 release candidate is available for download:
> 
>   http://people.apache.org/~stack/hbase-0.2.0-candidate-1/
> 
> Please take this release candidate for a spin. Check the
> documentation, that unit tests all complete on your platform, etc.
> 
> Should we release this candidate as hbase 0.2.0?  Vote yes
> or no before Friday, July 25th.