[jira] [Closed] (LENS-796) Slow response times for /metastore/nativetables API

["Raju Bairishetti (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/LENS-796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raju Bairishetti closed LENS-796.
---------------------------------

> Slow response times for /metastore/nativetables API
> ---------------------------------------------------
>
>                 Key: LENS-796
>                 URL: https://issues.apache.org/jira/browse/LENS-796
>             Project: Apache Lens
>          Issue Type: Improvement
>            Reporter: Deepak Barr
>            Assignee: Deepak Barr
>             Fix For: 2.4
>
>         Attachments: LENS-796.patch
>
>
> To obtain list of native tables, CubeMetastoreService does the following -
> 1. Fetches the list of tables ( one MetastoreClient call)
> 2. Filters out the cube tables from the list. The filtering happens by looking at the table properties from the Table object. This table object is obtained with another Metastore call. So, If there are 'n' tables, there will be 'n' metastore calls.
> Here is the code snippet :
>   private List<String> getTablesFromDB(LensSessionHandle sessionid,
>     String dbName, boolean prependDbName)
>     throws MetaException, UnknownDBException, HiveSQLException, TException, LensException {
>     List<String> tables = getSession(sessionid).getMetaStoreClient().getAllTables(
>       dbName);
>     List<String> result = new ArrayList<String>();
>     if (tables != null && !tables.isEmpty()) {
>       Iterator<String> it = tables.iterator();
>       while (it.hasNext()) {
>         String tblName = it.next();
>         org.apache.hadoop.hive.metastore.api.Table tbl =
>           getSession(sessionid).getMetaStoreClient().getTable(dbName, tblName);
>         if (tbl.getParameters().get(MetastoreConstants.TABLE_TYPE_KEY) == null) {
>           if (prependDbName) {
>             result.add(dbName + "." + tblName);
>           } else {
>             result.add(tblName);
>           }
>         }
>       }
>     }
>     return result;
>   }
> Instead of this, we can directly fetch the list of table objects for our list of table names in a single API call using getMetaStoreClient().getTableObjectsByName() method.
> Currently, one of our databases contain 8000+ tables which leads to very very long response times.
>    



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)