[jira] [Commented] (IMPALA-7556) Clean up ScanRange

["Amogh Margoor (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IMPALA-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343160#comment-17343160 ] 

Amogh Margoor commented on IMPALA-7556:
---------------------------------------

PR to decouple Buffer management from ScanRange is ready for review and existing tests on it are passing: https://gerrit.cloudera.org/#/c/17413/ cc [~boroknagyz]

> Clean up ScanRange
> ------------------
>
>                 Key: IMPALA-7556
>                 URL: https://issues.apache.org/jira/browse/IMPALA-7556
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>            Reporter: Zoltán Borók-Nagy
>            Assignee: Amogh Margoor
>            Priority: Major
>              Labels: ramp-up
>
> For IMPALA-7543 I want to add some additional functionality to scan ranges.
> However, the code of the ScanRange class is already quite messy. It handles different types of files, does some buffer management, updates all kinds of counters.
> So, instead of complicating the code further, let's refactor the ScanRange class a bit.
>  * Do the file operations in separate classes
>  ** A new, abstract class could be invented to provide an API for file operations, i.e. Open(), ReadFromPos(), Close(), etc.
>  *** Keep in mind that the interface must be a good fit for IMPALA-7543, i.e. we need positional reads from files
>  ** Operations for local files and HDFS files could be implemented in child classes
>  * Buffer management
>  ** A new BufferStore class could be created
>  ** This new class would be responsible for managing the unused buffers
>  *** if possible, it would also handle the client and cached buffers as well
>  * Counters and metrics would be updated by the corresponding new classes
>  ** E.g. ImpaladMetrics::IO_MGR_NUM_OPEN_FILES would be updated by the file handling classes



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org