You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "Micah Kornfield (Jira)" <ji...@apache.org> on 2020/05/15 04:44:00 UTC
[jira] [Resolved] (ARROW-8169) [Java] Improve the performance of
JDBC adapter by allocating memory proactively
[ https://issues.apache.org/jira/browse/ARROW-8169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Micah Kornfield resolved ARROW-8169.
------------------------------------
Fix Version/s: 1.0.0
Resolution: Fixed
Issue resolved by pull request 6672
[https://github.com/apache/arrow/pull/6672]
> [Java] Improve the performance of JDBC adapter by allocating memory proactively
> -------------------------------------------------------------------------------
>
> Key: ARROW-8169
> URL: https://issues.apache.org/jira/browse/ARROW-8169
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
> Reporter: Liya Fan
> Assignee: Liya Fan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The current implementation use {{setSafe}} methods to dynamically allocate memory if necessary. For fixed width vectors (which are frequently used in JDBC), however, we can allocate memory proactively, since the vector size is known as a configuration parameter. So for fixed width vectors, we can use {{set}} methods instead.
> This change leads to two benefits:
> 1. When processing each value, we no longer have to check vector capacity and reallocate memroy if needed. This leads to better performance.
> 2. If we allow the memory to expand automatically (each time by 2x), the amount of memory usually ends up being more than necessary. By allocating memory by the configuration parameter, we allocate no more, or no less.
> Benchmark results show notable performance improvements:
> Before:
> Benchmark Mode Cnt Score Error Units
> JdbcAdapterBenchmarks.consumeBenchmark avgt 5 521.700 ± 4.837 us/op
> After:
> Benchmark Mode Cnt Score Error Units
> JdbcAdapterBenchmarks.consumeBenchmark avgt 5 430.523 ± 9.932 us/op
--
This message was sent by Atlassian Jira
(v8.3.4#803005)