You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Zoltán Borók-Nagy (Jira)" <ji...@apache.org> on 2021/06/10 16:56:00 UTC
[jira] [Resolved] (IMPALA-7556) Clean up ScanRange
[ https://issues.apache.org/jira/browse/IMPALA-7556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zoltán Borók-Nagy resolved IMPALA-7556.
---------------------------------------
Fix Version/s: Impala 4.1
Resolution: Fixed
> Clean up ScanRange
> ------------------
>
> Key: IMPALA-7556
> URL: https://issues.apache.org/jira/browse/IMPALA-7556
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Amogh Margoor
> Priority: Major
> Labels: ramp-up
> Fix For: Impala 4.1
>
>
> For IMPALA-7543 I want to add some additional functionality to scan ranges.
> However, the code of the ScanRange class is already quite messy. It handles different types of files, does some buffer management, updates all kinds of counters.
> So, instead of complicating the code further, let's refactor the ScanRange class a bit.
> * Do the file operations in separate classes
> ** A new, abstract class could be invented to provide an API for file operations, i.e. Open(), ReadFromPos(), Close(), etc.
> *** Keep in mind that the interface must be a good fit for IMPALA-7543, i.e. we need positional reads from files
> ** Operations for local files and HDFS files could be implemented in child classes
> * Buffer management
> ** A new BufferStore class could be created
> ** This new class would be responsible for managing the unused buffers
> *** if possible, it would also handle the client and cached buffers as well
> * Counters and metrics would be updated by the corresponding new classes
> ** E.g. ImpaladMetrics::IO_MGR_NUM_OPEN_FILES would be updated by the file handling classes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)