You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org> on 2016/07/15 09:55:20 UTC
[jira] [Updated] (LENS-1165) Separate client for query submission
from other thrift operations
[ https://issues.apache.org/jira/browse/LENS-1165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated LENS-1165:
------------------------------------------
Fix Version/s: (was: 2.6)
2.7
> Separate client for query submission from other thrift operations
> -----------------------------------------------------------------
>
> Key: LENS-1165
> URL: https://issues.apache.org/jira/browse/LENS-1165
> Project: Apache Lens
> Issue Type: Improvement
> Components: driver-hive
> Reporter: Amareshwari Sriramadasu
> Assignee: Amareshwari Sriramadasu
> Fix For: 2.7
>
>
> Proposing to add separate thrift client for query submissions in HiveDriver with ability to give different thrift read timeouts and number of retries.
> This would help in the following cases :
> We have seen instances of query submission which can take more time. For ex :
> ----
> 2016-06-02T20:20:21,436 INFO [ef431743-b9f8-4fe0-8e3d-2bab154438eb HiveServer2-Handler-Pool: Thread-66837]: ql.Driver (Driver.java:compile(372)) - Compiling command(queryId=hive_20160602195701_2a3bd795-0919-4e2e-ad22-812881b084fd):
> 2016-06-02T21:02:43,380 INFO [ef431743-b9f8-4fe0-8e3d-2bab154438eb HiveServer2-Handler-Pool: Thread-66255]: ql.Driver (Driver.java:compile(524)) - Completed compiling command(queryId=hive_20160602200002_e58e07f9-61d0-423d-86ef-a626eb250114); Time taken: 1830.214 seconds
> ---
> In such cases lens submission will timeout with default client and resubmit, which would take more time again.
> And eventually, there will be number of submissions equal to the number of retries configured and lens would have marked the query as failed. But submission would actually launch multiple queries.
> If we have separate client, it can be configured with different timeout value and number retries to be different.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)