You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Jonathan Hsieh (JIRA)" <ji...@apache.org> on 2013/01/25 01:09:14 UTC

[jira] [Comment Edited] (HBASE-7660) Remove HFileV1 code

    [ https://issues.apache.org/jira/browse/HBASE-7660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13562169#comment-13562169 ] 

Jonathan Hsieh edited comment on HBASE-7660 at 1/25/13 12:08 AM:
-----------------------------------------------------------------

When HDFS upgrades incompatibly (hadoop1 -> hadoop2), it happens in is in "upgrade mode" and "finalize" command that I assume forces everything to the newer format.  Once finalized, you can't go back but you should be safe to go forward.  Before finalized, I believe it continues to only write in the older format.  This feels like #3 (except front loaded instead of lazy before the next upgrade).

We've actually run into some HFile format problems between 0.92 and 0.94 recently -- [~eclark]'s HBASE-7647, and I believe he's found another.
                
      was (Author: jmhsieh):
    When HDFS upgrades incompatibly (hadoop1 -> hadoop2), it happens in is in "upgrade mode" and "finalize" command that I assume forces everything to the newer format.  Once finalized, you can't go back but you should be safe to go forward.  Before finalized, I believe it continues to only write in the new format.  This feels like #3 (except front loaded instead of lazy before the next upgrade).

We've actually run into some HFile format problems between 0.92 and 0.94 recently -- [~eclark]'s HBASE-7647, and I believe he's found another.
                  
> Remove HFileV1 code
> -------------------
>
>                 Key: HBASE-7660
>                 URL: https://issues.apache.org/jira/browse/HBASE-7660
>             Project: HBase
>          Issue Type: Improvement
>          Components: hbck, HFile, migration
>            Reporter: Matt Corgan
>             Fix For: 0.96.0
>
>
> HFileV1 should be removed from the regionserver because it is somewhat of a drag on development for working on the lower level read paths.  It's an impediment to cleaning up the Store code.
> V1 HFiles ceased to be written in 0.92, but the V1 reader was left in place so users could upgrade from 0.90 to 0.92.  Once all HFiles are compacted in 0.92, then the V1 code is no longer needed.  We then decided to leave the V1 code in place in 0.94 so users could upgrade directly from 0.90 to 0.94.  The code is still there in trunk but should probably be shown the door.  I see a few options:
> 1) just delete the code and tell people to make sure they compact everything using 0.92 or 0.94
> 2) create a standalone script that people can run on their 0.92 or 0.94 cluster that iterates the filesystem and prints out any v1 files with a message that the user should run a major compaction
> 3) add functionality to 0.96.0 (first release, maybe in hbck) that proactively kills v1 files, so that we can be sure there are none when upgrading from 0.96 to 0.98
> 4) punt to 0.98 and probably do one of the above options in a year
> I would vote for #1 or #2 which will allow us to have a v1-free 0.96.0.  HFileV1 has already survived 2 major release upgrades which i think many would agree is more than enough for a pre-1.0, free product.  If we can remove it in 0.96.0 it will be out of the way to introduce some nice performance improvements in subsequent 0.96.x releases.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira