You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@drill.apache.org by sachouche <gi...@git.apache.org> on 2018/04/18 03:42:59 UTC

[GitHub] drill issue #1060: DRILL-5846: Improve parquet performance for Flat Data Typ...

Github user sachouche commented on the issue:

    https://github.com/apache/drill/pull/1060
  
    @parthchandra , @vrozov 
    
    I have done the following modifications:
    - Renamed newly added files with the prefix "VL" with "VarLen" as suggested by @parthchandra 
    - After talking offline with @vrozov about his objections to the MemoryUtils and also from previous feedbacks from @parthchandra , I have a) deleted this utility, b) exposed the needed functionality under the class DrillBuf (using Netty API's), and c) used the same configuration to control the checks.
    
    @parthchandra , @vrozov, please review this [document](https://docs.google.com/document/d/1BSNem_ItP-Vxlr6auSP_iwwOLM9rwWZYxGwCsXi-IE8/edit?usp=sharing) and the associated JMH code [here](https://github.com/sachouche/drill-jmh). This should provide you with performance benchmarks regarding the bulk approach that I have used in this PR request (please focus on test-3 which is Parquet specific); I will be adding more fine-grained tests with regard to the Memory Access tests (requested by @vrozov).


---