You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2019/01/18 13:05:00 UTC
[jira] [Commented] (HADOOP-16050) Support setting cipher suites for
s3a file system
[ https://issues.apache.org/jira/browse/HADOOP-16050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16746263#comment-16746263 ]
Steve Loughran commented on HADOOP-16050:
-----------------------------------------
Is this actually in session setup (slow, random IO with ORC/Parquet triggered unless you set fs.s3a.experimental.fadvise = random), or in actual wire communications?
If its the wire, then we have some fun, because that's all happening in the AWS SDK layer, which is beyond our direct control.
Ideally, they'd copy the Azure teams and use the wildfly JARs if on the CP, and, if bundled in their shaded SDK, come for ~free. Otherwise it looks like some configuration problem, which will need to be pushed out to every single process across a cluster. Which is always a pain to do
> Support setting cipher suites for s3a file system
> -------------------------------------------------
>
> Key: HADOOP-16050
> URL: https://issues.apache.org/jira/browse/HADOOP-16050
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.9.1
> Reporter: Justin Uang
> Priority: Major
> Attachments: Screen Shot 2019-01-17 at 2.57.06 PM.png
>
>
> We have found that when running the S3AFileSystem, it picks GCM as the ssl cipher suite. Unfortunately this is well known to be slow on java 8: [https://stackoverflow.com/questions/25992131/slow-aes-gcm-encryption-and-decryption-with-java-8u20.]
>
> In practice we have seen that it can take well over 50% of our CPU time in spark workflows. We should add an option to set the list of cipher suites we would like to use. !Screen Shot 2019-01-17 at 2.57.06 PM.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org