[jira] [Resolved] (DRILL-2080) Add IO wait time stats for Parquet and Json input files

["Venki Korukanti (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DRILL-2080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Venki Korukanti resolved DRILL-2080.
------------------------------------
          Resolution: Fixed
       Fix Version/s: 0.8.0
    Target Version/s: 0.8.0

Fixed in c54bd6a.

ReviewBoard link: https://reviews.apache.org/r/30305/


> Add IO wait time stats for Parquet and Json input files
> -------------------------------------------------------
>
>                 Key: DRILL-2080
>                 URL: https://issues.apache.org/jira/browse/DRILL-2080
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Relational Operators
>            Reporter: Venki Korukanti
>            Assignee: Venki Korukanti
>             Fix For: 0.8.0
>
>         Attachments: 0001-Add-IO-Stats-for-JSON-Parquet-type-input-files.patch
>
>
> Currently time spent in IO read time is included as part of the total processing time. This JIRA is to measure the IO read time separately and add it to OperatorStats. 
> Implementation details:
> Add a FileSystem implementation called DrillFileSystem which takes an existing FileSystem instance and OperatorStats. Whenever a file is opened using DrillFileSystem, it returns an instance of DrillFSDataInputStream which is a facade to actual FSDataInputStream. DrillFSDataInputStream adds the IO read time stats whenever a read request is issued. 
> IO Stats work only when DrillFileSystem is used. This patch modified JSON and Parquet readers to use DrillFileSystem. Text reader is not included.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)