You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Fredy Wijaya (JIRA)" <ji...@apache.org> on 2019/02/01 07:20:00 UTC
[jira] [Resolved] (IMPALA-7934) Switch to using Java 8's Base64
impl for incremental stats encoding
[ https://issues.apache.org/jira/browse/IMPALA-7934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fredy Wijaya resolved IMPALA-7934.
----------------------------------
Resolution: Fixed
Fix Version/s: Impala 3.2.0
> Switch to using Java 8's Base64 impl for incremental stats encoding
> -------------------------------------------------------------------
>
> Key: IMPALA-7934
> URL: https://issues.apache.org/jira/browse/IMPALA-7934
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.1.0
> Reporter: bharath v
> Assignee: Fredy Wijaya
> Priority: Major
> Labels: ramp-up
> Fix For: Impala 3.2.0
>
> Attachments: base64.png
>
>
> Incremental stats are compressed and Base64 encoded before they are chunked and written to the HMS' partition parameters map. When they are read back, we need to Base64 decode and decompress.
> For certain incremental stats heavy tables, we noticed that a significant amount of time is spent in these base64 classes (see the attached image for the stack. Unfortunately, I don't have the text version of it).
> Java 8 comes with its own Base64 implementation and that has shown much better perf results [1] compared to apache codec's impl. So consider switching to Java 8's base64 impl.
> [1] http://java-performance.info/base64-encoding-and-decoding-performance/
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)