You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Rohini Palaniswamy (JIRA)" <ji...@apache.org> on 2014/09/17 06:16:34 UTC
[jira] [Commented] (PIG-4164) After Pig job finish, Pig client
spend too much time retry to connect to AM
[ https://issues.apache.org/jira/browse/PIG-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14136750#comment-14136750 ]
Rohini Palaniswamy commented on PIG-4164:
-----------------------------------------
+1.
bq. I created a Cluster object every time which can be further optimized.
How do you intend to cache and reuse the Cluster object?
Listing places called.
getTaskReports - Called once for every job at the end of the job.
getCounters - Called once for every job and also called once for computingWarningAggregate at the end of the job. We can further optimize by clubbing computingWarningAggregate in MRJobStats when collecting stats. Also used in RANK jobs, but that is only for that one job.
> After Pig job finish, Pig client spend too much time retry to connect to AM
> ---------------------------------------------------------------------------
>
> Key: PIG-4164
> URL: https://issues.apache.org/jira/browse/PIG-4164
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Fix For: 0.14.0
>
> Attachments: PIG-4164-0.patch
>
>
> For some script, after job finish, Pig spend a lot time try to connect AM before get redirect to JobHistoryServer. Here is the message we saw:
> {code}
> 2014-09-10 15:13:55,370 [main] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:56,371 [main] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:57,372 [main] INFO org.apache.hadoop.ipc.Client - Retrying connect to server: daijymacpro-2.local/10.11.2.30:55223. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
> 2014-09-10 15:13:57,476 [main] INFO org.apache.hadoop.mapred.ClientServiceDelegate - Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)