[jira] [Commented] (PIG-4135) Fetch optimization should be disabled if plan contains no limit

["Daniel Dai (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PIG-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105067#comment-14105067 ] 

Daniel Dai commented on PIG-4135:
---------------------------------

+1

> Fetch optimization should be disabled if plan contains no limit
> ---------------------------------------------------------------
>
>                 Key: PIG-4135
>                 URL: https://issues.apache.org/jira/browse/PIG-4135
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>             Fix For: 0.14.0
>
>         Attachments: PIG-4135-1.patch
>
>
> After deploying fetch optimization in production, a couple of users ran into this situation. They had fairly large input data, but after filtering it by a regular expression, it becomes small. So they didn't add limit to the query. 
> The problem is that even though the output is small, processing the input must be done in the cluster not in the client. However, fetch optimization blindly fetches the entire input into the client since the plan is map-only job and finishes with dump.



--
This message was sent by Atlassian JIRA
(v6.2#6252)