[jira] [Comment Edited] (HBASE-20649) Validate HFiles do not have PREFIX_TREE DataBlockEncoding

["Sean Busbey (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-20649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511716#comment-16511716 ] 

Sean Busbey edited comment on HBASE-20649 at 6/13/18 10:03 PM:
---------------------------------------------------------------

The current patch misses any hfiles in a snapshot that aren't referenced by an active table. I think we could have this done in a follow-on, though it'd be nice to have here.

Unfortunately, it also misses any hfiles that are for an active table that was made by cloning a snapshot.

e.g.:

* create a table that uses PREFIX_TREE on a family, write some data, flush
* take a snapshot
* update the table to use FAST_DIFF
* major compact
* clone_snapshot to a new table
* use pre-upgrade validate-hfile

that last bit will claim that all the hfiles are good. If you list the contents of the cloned table in HDFS you'll see the hlink entries pointing to files in the archive dir. If you use the hfile tool to print the meta from these hfiles they'll confirm that they are still using PREFIX_TREE.


was (Author: busbey):
The current patch misses any hfiles in a snapshot that aren't referenced by an active table. I think we could have this done in a follow-on, though it'd be nice to have here.

Unfortunately, it also misses any hfiles that are for an active table that was made by cloning a snapshot.

e.g.:

* create a table that uses PREFIX_TREE on a family, write some data, flush
* take a snapshot
* update the table to use FAST_DIFF
* major compact
* use pre-upgrade validate-hfile

that last bit will claim that all the hfiles are good. If you list the contents of the cloned table in HDFS you'll see the hlink entries pointing to files in the archive dir. If you use the hfile tool to print the meta from these hfiles they'll confirm that they are still using PREFIX_TREE.

> Validate HFiles do not have PREFIX_TREE DataBlockEncoding
> ---------------------------------------------------------
>
>                 Key: HBASE-20649
>                 URL: https://issues.apache.org/jira/browse/HBASE-20649
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Peter Somogyi
>            Assignee: Peter Somogyi
>            Priority: Minor
>         Attachments: HBASE-20649.master.001.patch, HBASE-20649.master.002.patch, HBASE-20649.master.003.patch
>
>
> HBASE-20592 adds a tool to check column families on the cluster do not have PREFIX_TREE encoding.
> Since it is possible that DataBlockEncoding was already changed but HFiles are not rewritten yet we would need a tool that can verify the content of hfiles in the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)