You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@parquet.apache.org by "Alexey Kudinkin (Jira)" <ji...@apache.org> on 2021/12/03 23:29:00 UTC

[jira] [Created] (PARQUET-2106) BinaryComparator should avoid doing ByteBuffer.wrap

Alexey Kudinkin created PARQUET-2106:
----------------------------------------

             Summary: BinaryComparator should avoid doing ByteBuffer.wrap
                 Key: PARQUET-2106
                 URL: https://issues.apache.org/jira/browse/PARQUET-2106
             Project: Parquet
          Issue Type: Task
          Components: parquet-mr
    Affects Versions: 1.12.2
            Reporter: Alexey Kudinkin
         Attachments: Screen Shot 2021-12-03 at 3.26.31 PM.png, profile_48449_alloc_1638494450_sort_by.html

While writing out large Parquet tables using Spark, we've noticed that BinaryComparator is the source of substantial churn of extremely short-lived `HeapByteBuffer` objects: 

It's taking up to *16%* of total amount of allocations in our benchmarks

!Screen Shot 2021-12-03 at 3.26.31 PM.png!

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)