[jira] [Commented] (DRILL-7089) Implement caching of BaseMetadata classes

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/DRILL-7089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807710#comment-16807710 ] 

ASF GitHub Bot commented on DRILL-7089:
---------------------------------------

vvysotskyi commented on pull request #1728: DRILL-7089: Implement caching for TableMetadataProvider at query level and adapt statistics to use Drill metastore API
URL: https://github.com/apache/drill/pull/1728
 
 
   In the scope of this PR introduced caching of table metadata (schema and statistics) at the query level.
   Introduced `MetadataProviderManager` which holds both `SchemaProvider` and `DrillStatsTable` and `TableMetadataProvider` if it was already created.
   `MetadataProviderManager` instance will be cached and used for every `DrillTable` which corresponds to the same table.
   Such an approach was used to preserve lazy initialization of group scan and `TableMetadataProvider` instances, so once the first instance of `TableMetadataProvider` is created, it will be stored in the `MetadataProviderManager` and its metadata will be reused for all further `TableMetadataProvider` instances.
   
   Another part of this PR is connected with the adoption of statistics to use Drill Metastore API. Enhanced logic to distinguish exact and estimated metadata, and used `TableMetadata` for obtaining statistics.
   
   Will create and attach a class diagram later.
   
   Also, tests should be run for this PR, so for now, I'll leave it in draft state.
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Implement caching of BaseMetadata classes
> -----------------------------------------
>
>                 Key: DRILL-7089
>                 URL: https://issues.apache.org/jira/browse/DRILL-7089
>             Project: Apache Drill
>          Issue Type: Sub-task
>    Affects Versions: 1.16.0
>            Reporter: Volodymyr Vysotskyi
>            Assignee: Volodymyr Vysotskyi
>            Priority: Major
>             Fix For: 1.16.0
>
>
> In the scope of DRILL-6852 were introduced new classes for metadata usage. These classes may be reused in other GroupScan instances to preserve heap usage for the case when metadata is large.
> The idea is to store {{BaseMetadata}} inheritors in {{DrillTable}} and pass them to the {{GroupScan}}, so in the scope of the single query, it will be possible to reuse them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)