[jira] [Comment Edited] (HBASE-11827) Encryption support for bulkloading data into table with encryption configured for hfile format 3

["Andrew Purtell (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-11827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111660#comment-14111660 ] 

Andrew Purtell edited comment on HBASE-11827 at 8/27/14 12:49 AM:
------------------------------------------------------------------

The master key should not be available to any process or principal except the HBase service daemons and account. Therefore I think this issue is invalid. This patch would require the cluster master key be available to the potentially (i.e. probably) untrustworthy mapreduce execution environment.

It's fine to bulk load unencrypted HFiles into an encrypted table. The regionservers determine on a per file basis if something is encrypted or not. The bulk loaded files, even though they are unencrypted in the beginning, can be read right alongside existing encrypted HFiles. To have the regionserver encrypt the newly loaded HFiles, trigger a major compaction. Understood that this requires a rewrite of the data that was just loaded in. It's necessary when only the regionservers are trusted with sensitive key material.


was (Author: apurtell):
The master key should not be available to any process or principal except the HBase service daemons and account. Therefore I think this issue is invalid. This patch would require the cluster master key be available to the potentially (i.e. probably) untrustworthy mapreduce execution environment.

It's fine to bulk load unencrypted HFiles into an encrypted table. The region servers determine on a per file basis if something is encrypted or not. To have the region server encrypt the bulk loaded data, trigger a major compaction.

> Encryption support for bulkloading data into table with encryption configured for hfile format 3
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-11827
>                 URL: https://issues.apache.org/jira/browse/HBASE-11827
>             Project: HBase
>          Issue Type: Improvement
>          Components: mapreduce
>    Affects Versions: 0.98.5
>            Reporter: Kashif J S
>            Assignee: Kashif J S
>             Fix For: 2.0.0, 0.98.7
>
>         Attachments: HBASE-11827-98-v1.patch, HBASE-11827-trunk-v1.patch
>
>
> The solution would be to add support to auto detect encryption parameters similar to other parameters like compression, datablockencoding, etc when encryption is enabled for hfile format 3. 
> The current patch does the following:
> 1. Automatically detects encryption type and key in HFileOutputFormat & HFileOutputFormat2.
> 2. Uses Base64encoder/decoder for url passing of Encryption key which is in bytes format



--
This message was sent by Atlassian JIRA
(v6.2#6252)