[kudu-CR] [security] KUDU-3316 Add encrypted file keys

["Attila Bukor (Code Review)" <ge...@cloudera.org>]

Hello Tidy Bot, Zoltan Chovan, Alexey Serbin, Kudu Jenkins, Andrew Wong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18025

to look at the new patch set (#8).

Change subject: [security] KUDU-3316 Add encrypted file keys
......................................................................

[security] KUDU-3316 Add encrypted file keys

The previous patches in the data at rest encryption saga add the ability
to encrypt data at rest, but files were encrypted using the same hard-coded
key.

This patch adds an extra header to encrypted files to store the
encryption algorithm used and the encrypted file key. For now, the file
keys are encrypted with the same dummy encryption key which was
previously used to encrypt the files.

The header is a bit different from the one described in the design doc:
The encryption algorithm and key length was changed to be stored in
1 byte instead of 2 for easier handling and the magic string is
"kuduenc" instead of "kuduen".

This patch also introduces a new flag which is hidden for now:
--encryption_key_length. This can be set to any valid AES key length as
per its specification (128, 192, or 256 bits), as only AES encryption is
supported for now, and there are no plans to support anything else in
the foreseeable future.

As we add a 64-byte header to encrypted files, some changes had to be
made to code that handles files and relies on sizes and offsets,
including in tests.

This commit also changes the PBC tool to check if a file is encrypted
based on the encryption header instead of the file name.

I ran the full test suite manually with encryption enabled to make sure
turning on encryption doesn't break anything.

Change-Id: Idb1282c117271fda63a8cc54c00add7cc96dcffd
---
M src/kudu/consensus/log-test.cc
M src/kudu/consensus/log_index.cc
M src/kudu/consensus/log_util.cc
M src/kudu/fs/dir_util.cc
M src/kudu/fs/file_block_manager.cc
M src/kudu/fs/log_block_manager-test-util.cc
M src/kudu/fs/log_block_manager-test.cc
M src/kudu/fs/log_block_manager.cc
M src/kudu/mini-cluster/external_mini_cluster.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_pbc.cc
M src/kudu/tserver/tablet_copy_client-test.cc
M src/kudu/tserver/tablet_copy_service-test.cc
M src/kudu/tserver/tablet_copy_source_session-test.cc
M src/kudu/tserver/tablet_copy_source_session.cc
M src/kudu/util/env-test.cc
M src/kudu/util/env.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M src/kudu/util/env_util.cc
M src/kudu/util/env_util.h
M src/kudu/util/file_cache-stress-test.cc
M src/kudu/util/file_cache-test.cc
M src/kudu/util/file_cache.cc
M src/kudu/util/pb_util-test.cc
M src/kudu/util/pb_util.cc
26 files changed, 687 insertions(+), 246 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/25/18025/8
-- 
To view, visit http://gerrit.cloudera.org:8080/18025
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Idb1282c117271fda63a8cc54c00add7cc96dcffd
Gerrit-Change-Number: 18025
Gerrit-PatchSet: 8
Gerrit-Owner: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Zoltan Chovan <zc...@cloudera.com>