You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "bharath v (JIRA)" <ji...@apache.org> on 2018/12/06 06:02:00 UTC
[jira] [Updated] (IMPALA-7934) Switch to using Java 8's Base64 impl
for incremental stats encoding
[ https://issues.apache.org/jira/browse/IMPALA-7934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
bharath v updated IMPALA-7934:
------------------------------
Labels: ramp-up (was: )
> Switch to using Java 8's Base64 impl for incremental stats encoding
> -------------------------------------------------------------------
>
> Key: IMPALA-7934
> URL: https://issues.apache.org/jira/browse/IMPALA-7934
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.1.0
> Reporter: bharath v
> Priority: Major
> Labels: ramp-up
> Attachments: base64.png
>
>
> Incremental stats are compressed and Base64 encoded before they are chunked and written to the HMS' partition parameters map. When they are read back, we need to Base64 decode and decompress.
> For certain incremental stats heavy tables, we noticed that a significant amount of time is spent in these base64 classes (see the attached image for the stack. Unfortunately, I don't have the text version of it).
> Java 8 comes with its own Base64 implementation and that has shown much better perf results [1] compared to apache codec's impl. So consider switching to Java 8's base64 impl.
> [1] http://java-performance.info/base64-encoding-and-decoding-performance/
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org