[jira] Updated: (HIVE-1349) Do not pull counters of non initialized jobs

["Dmytro Molkov (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dmytro Molkov updated HIVE-1349:
--------------------------------

    Attachment: HIVE-1349.patch

Hey guys, take a look at the patch please.
When the job is created the status is PREP, it is switched to RUNNING when the setup is finished, so while the job status is PREP we are skipping the polling of counters.
The poll interval is also made configurable, so hive can hit jobtrackers getCounter which is not perfect less frequently.

> Do not pull counters of non initialized jobs
> --------------------------------------------
>
>                 Key: HIVE-1349
>                 URL: https://issues.apache.org/jira/browse/HIVE-1349
>             Project: Hadoop Hive
>          Issue Type: Improvement
>            Reporter: Dmytro Molkov
>         Attachments: HIVE-1349.patch
>
>
> Right now hive client puts a lot of pressure on the jobtracker by pulling the counters of the job that has just been submitted and is in the process of initializing tasks.
> While the job initialized tasks and reads the splits file it is holding the JobInProgress lock. The getCounters method acquires a JobTracker lock and waits for the JobInProgress lock. Since the read of the file can  take a couple (a couple of tens of seconds) the job tracker is effectively blocked for that period of time.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.