You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alan Gates (JIRA)" <ji...@apache.org> on 2008/08/28 21:59:44 UTC
[jira] Created: (PIG-403) Porting of the slicer into types branch
caused all scripts to run just one map.
Porting of the slicer into types branch caused all scripts to run just one map.
-------------------------------------------------------------------------------
Key: PIG-403
URL: https://issues.apache.org/jira/browse/PIG-403
Project: Pig
Issue Type: Bug
Affects Versions: types_branch
Reporter: Alan Gates
Assignee: Alan Gates
Priority: Critical
Fix For: types_branch
In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-403) Porting of the slicer into types branch
caused all scripts to run just one map.
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-403:
---------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch checked in.
> Porting of the slicer into types branch caused all scripts to run just one map.
> -------------------------------------------------------------------------------
>
> Key: PIG-403
> URL: https://issues.apache.org/jira/browse/PIG-403
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-403.patch
>
>
> In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-403) Porting of the slicer into types branch
caused all scripts to run just one map.
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-403:
---------------------------
Status: Patch Available (was: Open)
> Porting of the slicer into types branch caused all scripts to run just one map.
> -------------------------------------------------------------------------------
>
> Key: PIG-403
> URL: https://issues.apache.org/jira/browse/PIG-403
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-403.patch
>
>
> In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-403) Porting of the slicer into types branch
caused all scripts to run just one map.
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-403:
---------------------------
Attachment: PIG-403.patch
Changed PigSlicer.splittable default to true from false.
> Porting of the slicer into types branch caused all scripts to run just one map.
> -------------------------------------------------------------------------------
>
> Key: PIG-403
> URL: https://issues.apache.org/jira/browse/PIG-403
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-403.patch
>
>
> In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-403) Porting of the slicer into types branch
caused all scripts to run just one map.
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12626735#action_12626735 ]
Pradeep Kamath commented on PIG-403:
------------------------------------
Review comments:
I think this one line change should be good - "splittable" field is used in the slice() method in PigSlicer and that looks at filename (to see if it ends with .gz) or if splittable is set to false. The only way we allow splittable to set is through the setSplittable() which is not being called currently. I will be changing PigInputFormat to call setSplittable(false) when the script has "SPLIT by file" as part of the patch for streaming
> Porting of the slicer into types branch caused all scripts to run just one map.
> -------------------------------------------------------------------------------
>
> Key: PIG-403
> URL: https://issues.apache.org/jira/browse/PIG-403
> Project: Pig
> Issue Type: Bug
> Affects Versions: types_branch
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Fix For: types_branch
>
> Attachments: PIG-403.patch
>
>
> In PigSlicer.java, splittable is set to false by default. This causes the system to scan an entire file in a single map. It should be set to true by default, so that files are only scanned in a single map if the user explicitly requests it.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.