You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@drill.apache.org by "Keys Botzum (JIRA)" <ji...@apache.org> on 2017/02/21 21:47:44 UTC

[jira] [Commented] (DRILL-5287) Provide option to skip updates of ephemeral state changes in Zookeeper

    [ https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15876784#comment-15876784 ] 

Keys Botzum commented on DRILL-5287:
------------------------------------

Just curious if it would make sense for all queries (short or long) for the status update to be done in an async thread. That way it doesn't slow down query processing.

> Provide option to skip updates of ephemeral state changes in Zookeeper
> ----------------------------------------------------------------------
>
>                 Key: DRILL-5287
>                 URL: https://issues.apache.org/jira/browse/DRILL-5287
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.9.0
>            Reporter: Padma Penumarthy
>            Assignee: Padma Penumarthy
>             Fix For: 1.10
>
>
> We put transient profiles in zookeeper and update state as query progresses and changes states. It is observed that this adds latency of ~45msec for each update in the query execution path. This gets even worse when high number of concurrent queries are in progress. For concurrency=100, the average query response time even for short queries  is 8 sec vs 0.2 sec with these updates disabled. For short lived queries in a high-throughput scenario, it is of no value to update state changes in zookeeper. We need an option to disable these updates for short running operational queries.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)