You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org> on 2010/08/21 13:40:16 UTC
[jira] Created: (HIVE-1580) cleanup ExecDriver.progress
cleanup ExecDriver.progress
---------------------------
Key: HIVE-1580
URL: https://issues.apache.org/jira/browse/HIVE-1580
Project: Hadoop Hive
Issue Type: Bug
Components: Query Processor
Reporter: Joydeep Sen Sarma
Assignee: Joydeep Sen Sarma
a few problems:
- if a job is retired - then counters cannot be obtained and a stack trace is printed out (from history code). this confuses users
- too many calls to getCounters. after a job has been detected to be finished - there are quite a few more calls to get the job status and the counters. we need to figure out a way to curtail this - in busy clusters the gap between the job getting finished and the hive client noticing is very perceptible and impacts user experience.
calls to getCounters are very expensive in 0.20 as they grab a jobtracker global lock (something we have fixed internally at FB)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-1580) cleanup ExecDriver.progress
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joydeep Sen Sarma updated HIVE-1580:
------------------------------------
Attachment: hive-1580.1.patch
cleanup multiple calls to getCounters (which turns out to be really expensive call in JT) and don't print non-fatal stack traces to console.
> cleanup ExecDriver.progress
> ---------------------------
>
> Key: HIVE-1580
> URL: https://issues.apache.org/jira/browse/HIVE-1580
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Attachments: hive-1580.1.patch
>
>
> a few problems:
> - if a job is retired - then counters cannot be obtained and a stack trace is printed out (from history code). this confuses users
> - too many calls to getCounters. after a job has been detected to be finished - there are quite a few more calls to get the job status and the counters. we need to figure out a way to curtail this - in busy clusters the gap between the job getting finished and the hive client noticing is very perceptible and impacts user experience.
> calls to getCounters are very expensive in 0.20 as they grab a jobtracker global lock (something we have fixed internally at FB)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HIVE-1580) cleanup ExecDriver.progress
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain resolved HIVE-1580.
------------------------------
Hadoop Flags: [Reviewed]
Fix Version/s: 0.7.0
Resolution: Fixed
Committed. Thanks Joy
> cleanup ExecDriver.progress
> ---------------------------
>
> Key: HIVE-1580
> URL: https://issues.apache.org/jira/browse/HIVE-1580
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Fix For: 0.7.0
>
> Attachments: hive-1580.1.patch
>
>
> a few problems:
> - if a job is retired - then counters cannot be obtained and a stack trace is printed out (from history code). this confuses users
> - too many calls to getCounters. after a job has been detected to be finished - there are quite a few more calls to get the job status and the counters. we need to figure out a way to curtail this - in busy clusters the gap between the job getting finished and the hive client noticing is very perceptible and impacts user experience.
> calls to getCounters are very expensive in 0.20 as they grab a jobtracker global lock (something we have fixed internally at FB)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-1580) cleanup ExecDriver.progress
Posted by "Namit Jain (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12905766#action_12905766 ]
Namit Jain commented on HIVE-1580:
----------------------------------
+1
> cleanup ExecDriver.progress
> ---------------------------
>
> Key: HIVE-1580
> URL: https://issues.apache.org/jira/browse/HIVE-1580
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Query Processor
> Reporter: Joydeep Sen Sarma
> Assignee: Joydeep Sen Sarma
> Attachments: hive-1580.1.patch
>
>
> a few problems:
> - if a job is retired - then counters cannot be obtained and a stack trace is printed out (from history code). this confuses users
> - too many calls to getCounters. after a job has been detected to be finished - there are quite a few more calls to get the job status and the counters. we need to figure out a way to curtail this - in busy clusters the gap between the job getting finished and the hive client noticing is very perceptible and impacts user experience.
> calls to getCounters are very expensive in 0.20 as they grab a jobtracker global lock (something we have fixed internally at FB)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.