You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@arrow.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2020/05/27 03:39:01 UTC

[jira] [Updated] (ARROW-8940) [Java] Fix the performance degradation of integration tests

     [ https://issues.apache.org/jira/browse/ARROW-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated ARROW-8940:
----------------------------------
    Labels: pull-request-available  (was: )

> [Java] Fix the performance degradation of integration tests
> -----------------------------------------------------------
>
>                 Key: ARROW-8940
>                 URL: https://issues.apache.org/jira/browse/ARROW-8940
>             Project: Apache Arrow
>          Issue Type: Improvement
>          Components: Java
>            Reporter: Liya Fan
>            Assignee: Liya Fan
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> In the past, we run integration tests from main methods, and recently, we have changed this to run them by the failsafe plugin. 
> This is a good change, but it also leads to significant performance degradation. In the past, it took about 10s to run {{ITTestLargeVector#testLargeDecimalVector}}, now it takes more than half an hour. 
> Our investigation shows that the problem was caused by calling {{HistoricalLog#recordEvent}} repeatedly. This method is called only when {{BaseAllocator#DEBUG}} is enabled. In a unit/integration test, the flag is enabled by default. 
> We solve the problem with the following steps:
> 1. We set system property to disable the {{BaseAllocator#DEBUG}} flag.
> 2. We change the logic so that the system property takes precedence over the {{AssertionUtil#isAssertionsEnabled}} method. 
> This makes the integration tests as fast as before. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)