You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/10/07 14:08:00 UTC
[jira] [Commented] (IMPALA-10205) Avoid MD5 hash for data file path
of IcebergTable
[ https://issues.apache.org/jira/browse/IMPALA-10205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17209570#comment-17209570 ]
ASF subversion and git services commented on IMPALA-10205:
----------------------------------------------------------
Commit b907a1498d798ccf4ac89089b947d08d21b84814 in impala's branch refs/heads/master from wzhou-code
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b907a14 ]
IMPALA-10205: Replace MD5 with Murmur3 for generating datafile path hash
Current code generates data path hash in MD5 for Iceberg Table. But
MD5 is one of forbidden algorithms for FIPS. Even for non-security
purposes, like hash map, we still cannot use MD5 algorithm.
This patch replaces MD5 with non-cryptographic hash function
murmur3_128, which generates hash value with same length as MD5.
Testing:
- Passed core tests.
Change-Id: If7c805f2fdf0cf5a69738579c7e55f4bd047ed59
Reviewed-on: http://gerrit.cloudera.org:8080/16534
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Avoid MD5 hash for data file path of IcebergTable
> -------------------------------------------------
>
> Key: IMPALA-10205
> URL: https://issues.apache.org/jira/browse/IMPALA-10205
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Wenzhe Zhou
> Assignee: Wenzhe Zhou
> Priority: Major
> Labels: FIPS
> Fix For: Impala 4.0
>
>
> To support FIPS, all new code have to use FIPS-approved algorithms. Current code generate data path hash in MD5 hash for Iceberg Table. But MD5 is one of forbidden algorithms for FIPS. Even for non-security purposes, like hash map, we still cannot use MD5 since CryptoComply libraries have no way to distinguish between hash function being used for non-security purposes versus security purposes. We have to get rid of the md5 in Iceberg table related code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org