[jira] [Comment Edited] (HBASE-10095) Selective WALEdit encryption

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-10095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13854422#comment-13854422 ] 

Andrew Purtell edited comment on HBASE-10095 at 12/20/13 7:02 PM:
------------------------------------------------------------------

I've spent some time looking at how to accomplish this. We have implemented WALEdit encryption using a WALCellCodec, which is necessary because WALEdits are stratified by rows, not columns, so some cells in a WALEdit will be encrypted and some not if we are selectively doing this. In the WALCellCodec context, we only have information about the cell, we can't get a reference to anything that will lead to family information.

Replication provides an existing example of how to do family-specific WALEdit modification. Replication modifies WALEdits by adding a WALActionsListener at a high level where it has access to the server. The WALEdit type already has fields for carrying scope information. We could do something similar here: We could add a field to WALEdit indicating which cells for which famililes within it should be encrypted, and register a listener (up in HStore?) that sets it accordingly, but this is not enough because WALCellCodecs only see Cells, not the WALEdit that contains them.

I have experimented with a few interface changes and am not happy with any of the results so far. So I am going to move this out.


was (Author: apurtell):
I've spent some time looking at how to accomplish this. We have implemented WALEdit encryption using a WALCellCodec, which is necessary because WALEdits are stratified by rows, not columns, so some cells in a WALEdit will be encrypted and some not if we are selectively doing this. In the WALCellCodec context, we only have information about the cell, we can't get a reference to anything that will lead to family information.

Replication provides an existing example of how to do family-specific WALEdit modification. Replication modifies WALEdits by adding a WALActionsListener at a high level where it has access to the server. The WALEdit type already has fields for carrying scope information. We could do something similar here: We could add a field to WALEdit indicating if it should be encrypted or not and register a listener (up in HStore?) that sets it accordingly, but this is not enough because WALCellCodecs only see Cells, not the WALEdit that contains them.

I have experimented with a few interface changes and am not happy with any of the results so far. So I am going to move this out.

> Selective WALEdit encryption
> ----------------------------
>
>                 Key: HBASE-10095
>                 URL: https://issues.apache.org/jira/browse/HBASE-10095
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 0.99.0
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>
> The SecureWALProtobufWriter currently will encrypt every WAL entry if WAL encryption is enabled. However, SecureWALProtobufReader can distinguish between encrypted and unencrypted entries, and we encrypt every entry individually in part because the reader can skip and seek around during split and recovery, but also in part to enable selective encryption of WALedits. We should consider encrypting only the WALedits of column families for which HBASE-7544 features are configured. If few column families are encrypted relative to all CFs on the cluster, the performance difference will be significant.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)