[jira] [Commented] (SENTRY-2304) Optimize the time taken for HMS snapshot creation.

["Na Li (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SENTRY-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538913#comment-16538913 ] 

Na Li commented on SENTRY-2304:
-------------------------------

Instead of asking table info per db, it is much faster for HMS to provide all table info for all DB. It is fetching with filter versus bulk fetching. If we are serious about improving performance, we should ask HMS to provide API to return result via bulk fetching. And HMS client can organize the result using DB names. 

The same approach applies to partition. The HMS client can pass filter for partition to HMS server to reduce the amount of data sending back to HMS client. Right now, the filtering is done at HMS client, and could result in large network traffic.

> Optimize the time taken for HMS snapshot creation.
> --------------------------------------------------
>
>                 Key: SENTRY-2304
>                 URL: https://issues.apache.org/jira/browse/SENTRY-2304
>             Project: Sentry
>          Issue Type: Sub-task
>          Components: Sentry
>    Affects Versions: 2.1.0
>            Reporter: kalyan kumar kalvagadda
>            Priority: Major
>
> First get all the database names and their locations.
>  # Create Table Task for each database by providing the db name
>  # DB Task will get the names and locations for all the tables in that database.
>  # Create Partition task for each table in that database by providing the database and table names.
>  ## Table task will get the locations of all the partitions in that table.
> This approach needs new API’s implemented in HMS to get the names and locations of the databases/tables/partitions.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)