You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Daniel Dai (JIRA)" <ji...@apache.org> on 2014/09/11 00:23:34 UTC
[jira] [Updated] (PIG-4164) After Pig job finish, Pig client spend
too much time retry to connect to AM
[ https://issues.apache.org/jira/browse/PIG-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-4164:
----------------------------
Attachment: PIG-4164-0.patch
According to [~vinodkv], Pig shall not reuse the Job object to pull the taskReport/counters since it will keep trying AM. We shall construct a new Job object with designated Job ID to do that.
Note this issue only happen in Hadoop 2 since Hadoop 1 does not have AM/JobHistoryServer (will always query JobTracker).
Attach initial patch. In patch, I created a Cluster object every time which can be further optimized.
> After Pig job finish, Pig client spend too much time retry to connect to AM
> ---------------------------------------------------------------------------
>
> Key: PIG-4164
> URL: https://issues.apache.org/jira/browse/PIG-4164
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4164-0.patch
>
>
> For some script, after job finish, Pig spend a lot time try to connect AM before get redirect to JobHistoryServer. Here is the message we saw:
> {code}
> 2014-09-10 15:13:55,370 [main] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:56,371 [main] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:57,372 [main] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:57,476 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)