[jira] [Created] (CALCITE-1959) Reduce the amount of Metadata and table name calls in Druid

["Zain Humayun (JIRA)" <ji...@apache.org>]

Zain Humayun created CALCITE-1959:
-------------------------------------

             Summary: Reduce the amount of Metadata and table name calls in Druid
                 Key: CALCITE-1959
                 URL: https://issues.apache.org/jira/browse/CALCITE-1959
             Project: Calcite
          Issue Type: Improvement
          Components: druid
            Reporter: Zain Humayun


Currently in the Druid adapter, when a model definition file is not provided, query times can be quite slow due to excessive metadata and table names calls. Initial investigation reveals that a simple query like

{code:sql}
SELECT * from table
{code}

produces at least 30 http calls for fetching the table names, and another 30 calls for meta data. This slows down the query considerably, even when the individual calls themselves are relatively quick. 

The source of the issue seems to be {{DruidSchema#getTableMap}}, as this method dispatches calls to both {{DruidConnectionImpl#metadata}} and {{DruidConnectionImpl#tableNames}}. In addition, {{DruidTable#create}} makes another call to {{DruidConnectionImpl#metadata}} even if it's given the populated fields as arguments.

{{getTableMap}} is called fairly often indirectly from many places, including the validator, so as the query becomes larger, the problems gets a lot worse. Ideally, one would provide a model definition to prevent this, but doing so is not always possible/preferred for users.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)