You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "yiming.xu (Jira)" <ji...@apache.org> on 2020/05/27 12:30:00 UTC
[jira] [Comment Edited] (ARROW-8963) Parquet cpp optimize allocate
memory
[ https://issues.apache.org/jira/browse/ARROW-8963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17117571#comment-17117571 ]
yiming.xu edited comment on ARROW-8963 at 5/27/20, 12:29 PM:
-------------------------------------------------------------
this is github pr link : https://github.com/apache/arrow/pull/7288
fix design: switch the resevre and set to void resevre to 2 * size
this is 6_000_000 rows parquet,about 250M,read 3 long type columns:
before 748 742 703 740 726 708
after 656 667 678 671 687 649
improve about 10%
was (Author: hn5092):
this is github pr link : https://github.com/apache/arrow/pull/7288
fix design: switch the resevre and set to void resevre to 2 * size
> Parquet cpp optimize allocate memory
> ------------------------------------
>
> Key: ARROW-8963
> URL: https://issues.apache.org/jira/browse/ARROW-8963
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Format
> Affects Versions: 0.17.1
> Reporter: yiming.xu
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> LeafReader::NextBatch should Reset memory first, otherwise Reserve will allocate memory twice
--
This message was sent by Atlassian Jira
(v8.3.4#803005)