You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by rx...@apache.org on 2017/02/10 16:11:08 UTC
spark git commit: Encryption of shuffle files
Repository: spark
Updated Branches:
refs/heads/master 8640dc082 -> c5a66356d
Encryption of shuffle files
Hello
According to my understanding of commits 4b4e329e49f8af28fa6301bd06c48d7097eaf9e6 & 8b325b17ecdf013b7a6edcb7ee3773546bd914df, one may now encrypt shuffle files regardless of the cluster manager in use.
However I have limited understanding of the code, I'm not able to find out whether theses changes also comprise all "temporary local storage, such as shuffle files, cached data, and other application files".
Please feel free to amend or reject my PR if I'm wrong.
dud
Author: Herv� <du...@users.noreply.github.com>
Closes #16885 from dud225/patch-1.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c5a66356
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c5a66356
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c5a66356
Branch: refs/heads/master
Commit: c5a66356d431dc07dbd44540a495264fb19bd5d9
Parents: 8640dc0
Author: Herv� <du...@users.noreply.github.com>
Authored: Fri Feb 10 17:11:03 2017 +0100
Committer: Reynold Xin <rx...@databricks.com>
Committed: Fri Feb 10 17:11:03 2017 +0100
----------------------------------------------------------------------
docs/security.md | 6 +-----
1 file changed, 1 insertion(+), 5 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/c5a66356/docs/security.md
----------------------------------------------------------------------
diff --git a/docs/security.md b/docs/security.md
index 42a09a9..a479676 100644
--- a/docs/security.md
+++ b/docs/security.md
@@ -28,11 +28,7 @@ If your applications are using event logging, the directory where the event logs
## Encryption
Spark supports SSL for HTTP protocols. SASL encryption is supported for the block transfer service
-and the RPC endpoints.
-
-Encryption is not yet supported for data stored by Spark in temporary local storage, such as shuffle
-files, cached data, and other application files. If encrypting this data is desired, a workaround is
-to configure your cluster manager to store application data on encrypted disks.
+and the RPC endpoints. Shuffle files can also be encrypted if desired.
### SSL Configuration
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org