You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2024/01/10 17:19:00 UTC
[jira] [Created] (HADOOP-19033) S3A: disable checksum validation
Steve Loughran created HADOOP-19033:
---------------------------------------
Summary: S3A: disable checksum validation
Key: HADOOP-19033
URL: https://issues.apache.org/jira/browse/HADOOP-19033
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Reporter: Steve Loughran
Assignee: Steve Loughran
AWS v2 sdk turns on client-side checksum validation; this kills performance
Given we are using TLS to download from AWS s3, there's implicit channel checksumming going on on, that's along with the IPv4 TCP checksumming.
We don't need it, all it does is slow us down.
proposed: disable in DefaultS3ClientFactory
I don't want to add an option to enable it as it only complicates life (yet another config option), but I am open to persuasion
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org