[jira] [Updated] (PARQUET-2202) Redundant String allocation on the hot path in CapacityByteArrayOutputStream.setByte

["Gang Wu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PARQUET-2202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gang Wu updated PARQUET-2202:
-----------------------------
    Fix Version/s: 1.13.0

> Redundant String allocation on the hot path in CapacityByteArrayOutputStream.setByte
> ------------------------------------------------------------------------------------
>
>                 Key: PARQUET-2202
>                 URL: https://issues.apache.org/jira/browse/PARQUET-2202
>             Project: Parquet
>          Issue Type: Bug
>          Components: parquet-mr
>    Affects Versions: 1.12.3
>            Reporter: Andrei Pangin
>            Priority: Major
>              Labels: performance
>             Fix For: 1.13.0
>
>         Attachments: profile-alloc.png, profile-cpu.png
>
>
> Profiling of a Spark application revealed a performance issue in production:
> {{CapacityByteArrayOutputStream.setByte}} consumed 2.2% of total CPU time and made up 4.6% of total allocations. However, in normal case, this method should allocate nothing at all.
> Here is an excerpt from async-profiler report.
> CPU profile:
> !profile-cpu.png|width=560!
> Allocation profile:
> !profile-alloc.png|width=560!
> The reason is a {{checkArgument()}} call with an unconditionally constructed dynamic String:
> [https://github.com/apache/parquet-mr/blob/62b774cd0f0c60cfbe540bbfa60bee15929af5d4/parquet-common/src/main/java/org/apache/parquet/bytes/CapacityByteArrayOutputStream.java#L303]
> The suggested fix is to move String construction under the condition:
> {code:java}
> if (index >= bytesUsed) {
>   throw new IllegalArgumentException("Index: " + index +
>       " is >= the current size of: " + bytesUsed);
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)