Encryption on RDDs or in-memory on Apache Spark

[IASIB1 <mo...@qub.ac.uk>]

I am currently working on the latest version of Apache Spark (1.4.1),
pre-built package for Hadoop 2.6+.

Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory (similarly
to Altibase's HDB: 
http://altibase.com/in-memory-database-computing-solutions/security/
<http://altibase.com/in-memory-database-computing-solutions/security/>  )
when running applications in Spark? Or is there an external
library/framework which could be used to encrypt RDDs or in-memory in Spark?

Any help would be appreciated. Many thanks.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Encryption-on-RDDs-or-in-memory-on-Apache-Spark-tp23982.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Encryption on RDDs or in-memory on Apache Spark

[Akhil Das <ak...@sigmoidanalytics.com>]

Have a look at the current security support
<https://spark.apache.org/docs/latest/security.html>, Spark does not have
any encryption support for objects in memory out of the box. But if your
concern is to protect the data being cached in memory, then you can easily
encrypt your objects in memory with SealedObject
<http://docs.oracle.com/javase/7/docs/api/javax/crypto/SealedObject.html>,
Here's one simple example
<http://stackoverflow.com/questions/16950833/is-there-an-easy-way-to-encrypt-a-java-object#answers-header>
which
you can make use of.

Thanks
Best Regards

On Fri, Jul 24, 2015 at 2:12 PM, IASIB1 <mo...@qub.ac.uk> wrote:

> I am currently working on the latest version of Apache Spark (1.4.1),
> pre-built package for Hadoop 2.6+.
>
> Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory
> (similarly
> to Altibase's HDB:
> http://altibase.com/in-memory-database-computing-solutions/security/
> <http://altibase.com/in-memory-database-computing-solutions/security/>  )
> when running applications in Spark? Or is there an external
> library/framework which could be used to encrypt RDDs or in-memory in
> Spark?
>
> Any help would be appreciated. Many thanks.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Encryption-on-RDDs-or-in-memory-on-Apache-Spark-tp23982.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>