Zeppelin Pulling Files from S3 That Are KMS Encrypted Denied

["Tseytlin, Keren" <Ke...@capitalone.com>]

Hi All,

I have a bucket that I’m working with and I want to pull orc files from there and use it in my Spark/Scala magic. The only thing is that these files are KMS encrypted. When I try to get a KMS file however, it shows me an AWS Access Denied error, although there is no possible way that could be true.

First, I can successfully pull a file from that directory (this file below is SSE encrypted):

val orctest8 = sqlContext.read.format("orc").load("s3a://kafkaTest/connect-prefix/BLUECOATCIM/2016/10/06/FILE123.orc")
orctest8.count()

res149: Long = 17

Then, I try to pull a file from that directory that is KMS encrypted:

val orctest8 = sqlContext.read.format("orc").load("s3a://kafkaTest/connect-prefix/BLUECOATCIM/2016/10/06/FILEXYZ.orc")
orctest8.count()

java.nio.file.AccessDeniedException: s3a://kafkaTest/connect-prefix/BLUECOATCIM/2016/10/06/FILEXYZ.orc: Reopen at position 1011102 on s3a://kafkaTest/connect-prefix/BLUECOATCIM/2016/10/06/FILEXYZ.orc: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: XXXXXXXXXXXX)

As you can see above, both of the files are originating from the same directory. They are also the same exact file, except one is KMS encrypted and the other is SSE. Does Zeppelin currently have the capability to read in KMS encrypted files and use them? If so, what parameters need to be set?

Best,
Keren

________________________________________________________

The information contained in this e-mail is confidential and/or proprietary to Capital One and/or its affiliates and may only be used solely in performance of work or services for Capital One. The information transmitted herewith is intended only for use by the individual or entity to which it is addressed. If the reader of this message is not the intended recipient, you are hereby notified that any review, retransmission, dissemination, distribution, copying or other use of, or taking of any action in reliance upon this information is strictly prohibited. If you have received this communication in error, please contact the sender and delete the material from your computer.