[GitHub] mccheah opened a new issue #20: Encryption in Data Files

[GitBox <gi...@apache.org>]

mccheah opened a new issue #20: Encryption in Data Files
URL: https://github.com/apache/incubator-iceberg/issues/20
 
 
   We want to support encrypting and decrypting data that is recorded in Iceberg tables. There are several API extensions that we can consider to make this work:
   
   - Define a `KeyReference` field, which is a byte blob in the `DataFile` object. A `KeyReference` is a pointer to a key.
   - Define an `EncryptionKey` which is a composition of the key bytes, the iv, and the key algorithm (see e.g. [here](https://github.com/palantir/hadoop-crypto/blob/develop/crypto-keys/src/main/java/com/palantir/crypto2/keys/KeyMaterial.java#L28) and [here](https://github.com/palantir/hadoop-crypto/blob/b6b1680a003c23bdd0ce0b01a394eacd95eb0e9d/crypto-keys/src/main/java/com/palantir/crypto2/keys/serialization/KeyMaterials.java#L53))
   
   ```
   struct EncryptionKey {
       byte[] encodedKey();
       String keyAlgorithm();
       byte[] iv();
   }
   ```
   
   - Define a `KeyManager` which manages creating new keys and retrieving keys based on key references. The `TableOperations` API should support returning an `Optional<KeyManager>`; return `Optional.empty()` if the table operations doesn't support encryption.
   
   ```
   struct CreatedKey {
       EncryptionKey key();
       byte[] keyReference();
   }
   
   interface KeyManager {
       CreatedKey createKey(String pathToEncrypt);
       EncryptionKey getKey(KeyReference reference);
   }
   ```

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services