You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@orc.apache.org by do...@apache.org on 2021/08/08 19:53:08 UTC

[orc] branch branch-1.7 updated: ORC-921: Add an encrypted example file (#833)

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/orc.git


The following commit(s) were added to refs/heads/branch-1.7 by this push:
     new caa8641  ORC-921: Add an encrypted example file (#833)
caa8641 is described below

commit caa86418f4f01df4d968b5e5cace34f8cb967b0d
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Aug 8 12:46:03 2021 -0700

    ORC-921: Add an encrypted example file (#833)
    
    ### What changes were proposed in this pull request?
    
    This PR aims to add an encrypted example file to the `example` directory.
    
    ### Why are the changes needed?
    
    `example` directory is a good place to show Apache ORC feature.
    
    ### How was this patch tested?
    
    This is an example file which can be read by the following.
    
    **From Java Tools**
    ```
    $ cd java
    $ mvn package -DskipTests
    $ java -jar tools/target/*-uber.jar data ../examples/encrypted/*.orc 2> /dev/null
    {"ssn":null,"email":"841626795E7D351555B835A002E3BF10669DE9B81C95A3D59E10865AC37EA7C3","name":"Dongjoon Hyun"}
    ________________________________________________________________________________________________________________________
    
    {"ssn":null,"email":"B948E77D79105EAA53E54502CB39DA20C6A86263FA301D5B7EDBACC66D15B642","name":"William Hyun"}
    ________________________________________________________________________________________________________________________
    ```
    
    **From Spark 3.2 + Hadoop KMS with the provided `kms.keystore` file**
    ```
    export HADOOP_KEYSTORE_PASSWORD=ORC_KEY
    bin/hadoop --daemon start kms
    ```
    ```
    scala> spark.read.option("orc.key.provider", "hadoop").option("hadoop.security.key.provider.path", "kms://http@localhost:9600/kms").option("orc.encrypt", "pii:ssn,email").orc("sample.orc").show
    +---------+-------------------+-------------+
    |      ssn|              email|         name|
    +---------+-------------------+-------------+
    |123456789|dongjoon@apache.org|Dongjoon Hyun|
    +---------+-------------------+-------------+
    ```
    
    (cherry picked from commit ed1cda90710bd0c7bf406761337af4b36a9b931e)
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 examples/encrypted/kms.keystore | Bin 0 -> 1158 bytes
 examples/encrypted/sample1.orc  | Bin 0 -> 1355 bytes
 examples/encrypted/sample2.orc  | Bin 0 -> 1341 bytes
 3 files changed, 0 insertions(+), 0 deletions(-)

diff --git a/examples/encrypted/kms.keystore b/examples/encrypted/kms.keystore
new file mode 100644
index 0000000..cd9152b
Binary files /dev/null and b/examples/encrypted/kms.keystore differ
diff --git a/examples/encrypted/sample1.orc b/examples/encrypted/sample1.orc
new file mode 100644
index 0000000..aa0e502
Binary files /dev/null and b/examples/encrypted/sample1.orc differ
diff --git a/examples/encrypted/sample2.orc b/examples/encrypted/sample2.orc
new file mode 100644
index 0000000..7f9b214
Binary files /dev/null and b/examples/encrypted/sample2.orc differ