Slow Response Time for Webhcat DDLs

[Juan Delard de Rigoulières <ju...@datarepublic.io>]

Hi,
We're planning to use WebHCat REST API to render JavaScript based Web
console. As a result we'd like reasonable response times from WebHCat API.

The responses we measure in our cluster are slow to deliver a user friendly
UI. We are looking for ways to improve these response times without caching.

At the moment, our end to end response for a simple DDL describe database
is around 10 seconds. Logs are attached for all components along the way:
curl, knox, wehbcat and hive-metastore.

curl  : 7.390s
knox: 7.133s
hcat : 6.933s
hive : 5.192s

Do these numbers make sense? Or we have a misconfiguration in our
environment. FYI our cluster is secure. accessing second time with Knox
cookie does not help much.

WebHCat is invoking hcat command for each DDL call but I don't think it
contributes to much of slowness as most of time is spent in Hive Metastore
initialising and shutting down ObjectStore for every call. Also DDL is
pretty fast accessing Hive directly with beeline JDBC.

Appreciate your help,

Juan