You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Kevin Wilfong (Created) (JIRA)" <ji...@apache.org> on 2012/02/25 01:53:52 UTC
[jira] [Created] (HIVE-2823) Add logging of total run time of
Driver
Add logging of total run time of Driver
---------------------------------------
Key: HIVE-2823
URL: https://issues.apache.org/jira/browse/HIVE-2823
Project: Hive
Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Namit Jain (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-2823:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Kevin
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2823.D1977.1.patch, HIVE-2823.D1977.2.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2823:
------------------------------
Attachment: HIVE-2823.D1977.1.patch
kevinwilfong requested code review of "HIVE-2823 [jira] Add logging of total run time of Driver".
Reviewers: JIRA
https://issues.apache.org/jira/browse/HIVE-2823
Updated the PerfLogger to now log the total runtime of Driver.run now. I also added methods to the PerfLogger to get the start and finish times, so that these can be logged if desired.
We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
TEST PLAN
EMPTY
REVISION DETAIL
https://reviews.facebook.net/D1977
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
MANAGE HERALD DIFFERENTIAL RULES
https://reviews.facebook.net/herald/view/differential/
WHY DID I GET THIS EMAIL?
https://reviews.facebook.net/herald/transcript/4203/
Tip: use the X-Herald-Rules header to filter Herald messages in your client.
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2823.D1977.1.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Kevin Wilfong (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevin Wilfong updated HIVE-2823:
--------------------------------
Status: Patch Available (was: Open)
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2823.D1977.1.patch, HIVE-2823.D1977.2.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Phabricator (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Phabricator updated HIVE-2823:
------------------------------
Attachment: HIVE-2823.D1977.2.patch
kevinwilfong updated the revision "HIVE-2823 [jira] Add logging of total run time of Driver".
Reviewers: JIRA, njain
After discussing this with Namit Jain, we decided that since I ultimately intend to access the Driver.run start time from a post execution hook, it would make the most sense in terms of dependencies, if the start time was added to the QueryPlan, from which it can be accessed by the hook.
REVISION DETAIL
https://reviews.facebook.net/D1977
AFFECTED FILES
ql/src/java/org/apache/hadoop/hive/ql/log/PerfLogger.java
ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java
ql/src/java/org/apache/hadoop/hive/ql/Driver.java
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2823.D1977.1.patch, HIVE-2823.D1977.2.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Phabricator (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494427#comment-13494427 ]
Phabricator commented on HIVE-2823:
-----------------------------------
kevinwilfong has closed the revision "HIVE-2823 [jira] Add logging of total run time of Driver".
REVISION DETAIL
https://reviews.facebook.net/D1977
To: JIRA, njain, kevinwilfong
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Fix For: 0.9.0
>
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2823.D1977.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2823.D1977.2.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Phabricator (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13217681#comment-13217681 ]
Phabricator commented on HIVE-2823:
-----------------------------------
njain has accepted the revision "HIVE-2823 [jira] Add logging of total run time of Driver".
running tests
REVISION DETAIL
https://reviews.facebook.net/D1977
BRANCH
svn
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Attachments: HIVE-2823.D1977.1.patch, HIVE-2823.D1977.2.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2823) Add logging of total run time of
Driver
Posted by "Ashutosh Chauhan (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashutosh Chauhan updated HIVE-2823:
-----------------------------------
Fix Version/s: 0.9.0
> Add logging of total run time of Driver
> ---------------------------------------
>
> Key: HIVE-2823
> URL: https://issues.apache.org/jira/browse/HIVE-2823
> Project: Hive
> Issue Type: Improvement
> Reporter: Kevin Wilfong
> Assignee: Kevin Wilfong
> Fix For: 0.9.0
>
> Attachments: HIVE-2823.D1977.1.patch, HIVE-2823.D1977.2.patch
>
>
> We are currently logging the run time of several components of the Driver, e.g. compilation and map reduce execution time. It would be helpful to log the total run time of the Driver's run method as well.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira