You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@orc.apache.org by do...@apache.org on 2021/08/08 19:53:08 UTC
[orc] branch branch-1.7 updated: ORC-921: Add an encrypted example
file (#833)
This is an automated email from the ASF dual-hosted git repository.
dongjoon pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/orc.git
The following commit(s) were added to refs/heads/branch-1.7 by this push:
new caa8641 ORC-921: Add an encrypted example file (#833)
caa8641 is described below
commit caa86418f4f01df4d968b5e5cace34f8cb967b0d
Author: Dongjoon Hyun <do...@apache.org>
AuthorDate: Sun Aug 8 12:46:03 2021 -0700
ORC-921: Add an encrypted example file (#833)
### What changes were proposed in this pull request?
This PR aims to add an encrypted example file to the `example` directory.
### Why are the changes needed?
`example` directory is a good place to show Apache ORC feature.
### How was this patch tested?
This is an example file which can be read by the following.
**From Java Tools**
```
$ cd java
$ mvn package -DskipTests
$ java -jar tools/target/*-uber.jar data ../examples/encrypted/*.orc 2> /dev/null
{"ssn":null,"email":"841626795E7D351555B835A002E3BF10669DE9B81C95A3D59E10865AC37EA7C3","name":"Dongjoon Hyun"}
________________________________________________________________________________________________________________________
{"ssn":null,"email":"B948E77D79105EAA53E54502CB39DA20C6A86263FA301D5B7EDBACC66D15B642","name":"William Hyun"}
________________________________________________________________________________________________________________________
```
**From Spark 3.2 + Hadoop KMS with the provided `kms.keystore` file**
```
export HADOOP_KEYSTORE_PASSWORD=ORC_KEY
bin/hadoop --daemon start kms
```
```
scala> spark.read.option("orc.key.provider", "hadoop").option("hadoop.security.key.provider.path", "kms://http@localhost:9600/kms").option("orc.encrypt", "pii:ssn,email").orc("sample.orc").show
+---------+-------------------+-------------+
| ssn| email| name|
+---------+-------------------+-------------+
|123456789|dongjoon@apache.org|Dongjoon Hyun|
+---------+-------------------+-------------+
```
(cherry picked from commit ed1cda90710bd0c7bf406761337af4b36a9b931e)
Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
examples/encrypted/kms.keystore | Bin 0 -> 1158 bytes
examples/encrypted/sample1.orc | Bin 0 -> 1355 bytes
examples/encrypted/sample2.orc | Bin 0 -> 1341 bytes
3 files changed, 0 insertions(+), 0 deletions(-)
diff --git a/examples/encrypted/kms.keystore b/examples/encrypted/kms.keystore
new file mode 100644
index 0000000..cd9152b
Binary files /dev/null and b/examples/encrypted/kms.keystore differ
diff --git a/examples/encrypted/sample1.orc b/examples/encrypted/sample1.orc
new file mode 100644
index 0000000..aa0e502
Binary files /dev/null and b/examples/encrypted/sample1.orc differ
diff --git a/examples/encrypted/sample2.orc b/examples/encrypted/sample2.orc
new file mode 100644
index 0000000..7f9b214
Binary files /dev/null and b/examples/encrypted/sample2.orc differ