You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Pradeep Kamath (JIRA)" <ji...@apache.org> on 2009/10/12 20:12:31 UTC
[jira] Commented: (PIG-958) Splitting output data on key field
[ https://issues.apache.org/jira/browse/PIG-958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12764773#action_12764773 ]
Pradeep Kamath commented on PIG-958:
------------------------------------
+1 - changes looks good!
For the test, I observed you were using the mapreduce mode pigserver object even in local mode - I made some changes but was unable to run the tests due to some config issue in setting up the test run - did not explore more - nevertheless here is what I changed:
{noformat}
127 private void testMultiStorage(PigServer pigServer, Mode mode,
128 String... queries) throws IOException {
129 PigServer ps = (mode == Mode.cluster) ? pigServer: pigServerLocal;
130 ps.setBatchOn();
131 for (String query : queries) {
132 ps.registerQuery(query);
133 }
134 ps.executeBatch();
135 verifyResults(mode);
136 }
{nofrmat}
Check if making the above changes solves the issue you are seeing.
> Splitting output data on key field
> ----------------------------------
>
> Key: PIG-958
> URL: https://issues.apache.org/jira/browse/PIG-958
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.4.0
> Reporter: Ankur
> Attachments: 958.v3.patch
>
>
> Pig users often face the need to split the output records into a bunch of files and directories depending on the type of record. Pig's SPLIT operator is useful when record types are few and known in advance. In cases where type is not directly known but is derived dynamically from values of a key field in the output tuple, a custom store function is a better solution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.