You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@quickstep.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/06/14 19:43:39 UTC
[jira] [Commented] (QUICKSTEP-21) Measure WorkOrder execution time

    [ https://issues.apache.org/jira/browse/QUICKSTEP-21?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330264#comment-15330264 ] 

ASF GitHub Bot commented on QUICKSTEP-21:
-----------------------------------------

GitHub user hbdeshmukh opened a pull request:

    https://github.com/apache/incubator-quickstep/pull/32

    QUICKSTEP-21 Measure execution time of normal WorkOrders.

    This PR includes the following changes: 
    
    - Measure normal work order execution time
    - Split the WorkOrderCompletion proto message in two: Normal and Rebuild work orders.
    - Add execution time field in the NormalWorkOrderComplete proto message.
    - Include the recorded time in NormalWorkOrderComplete proto message that is sent back to Foreman.
    
    Note that I didn't add the execution time field in RebuildWorkOrder completion message because rebuilding storage block is pretty cheap for the storage layouts used by default. Therefore the time taken by rebuild work order is quite low. If in the future, we need this time, we can easily add it.
    
    Also note that, the CPU overhead for measuring this time is pretty low. In terms of memory, we pay a cost of an additional 8 bytes in each completion message (only for normal work orders) sent from each worker to the Foreman. 
    
    This feature can be quite helpful for performance monitoring. We can take a look at individual work order execution time of different operators. I am planning to create another PR which can produce reports consisting of average/max/min/variance of execution times grouped by different operators. 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/apache/incubator-quickstep record-wo-execution-time

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-quickstep/pull/32.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #32
    
----
commit ba57e68333527e2a9c4376ab784beb42630887e2
Author: Harshad Deshmukh <hb...@apache.org>
Date:   2016-06-14T19:02:45Z

    Measure execution time of normal WorkOrders.
    
    - Measure normal work order execution time
    - Split the WorkOrderCompletion proto message in two: Normal and Rebuild
      work orders.
    - Add execution time field in the NormalWorkOrderComplete proto message.
    - Include the recorded time in NormalWorkOrderComplete proto message
      that is sent back to Foreman.

----


> Measure WorkOrder execution time
> --------------------------------
>
>                 Key: QUICKSTEP-21
>                 URL: https://issues.apache.org/jira/browse/QUICKSTEP-21
>             Project: Apache Quickstep
>          Issue Type: Improvement
>          Components: Query Execution
>            Reporter: Harshad Deshmukh
>            Assignee: Harshad Deshmukh
>            Priority: Minor
>
> Right now we don't have a way to measure how long does each work order execution take. We can simply add timers around each execute() call that happens in a Worker thread and record the time. The recorded execution time can be sent back to the Foreman in the completion message.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)