You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Olga Natkovich (JIRA)" <ji...@apache.org> on 2009/11/18 02:44:39 UTC
[jira] Created: (PIG-1096) Make all unit tests run in MR or local
mode based on a property
Make all unit tests run in MR or local mode based on a property
---------------------------------------------------------------
Key: PIG-1096
URL: https://issues.apache.org/jira/browse/PIG-1096
Project: Pig
Issue Type: Improvement
Reporter: Olga Natkovich
No that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (PIG-1096) Make all unit tests run in MR or local
mode based on a property
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Ding reassigned PIG-1096:
---------------------------------
Assignee: Richard Ding
> Make all unit tests run in MR or local mode based on a property
> ---------------------------------------------------------------
>
> Key: PIG-1096
> URL: https://issues.apache.org/jira/browse/PIG-1096
> Project: Pig
> Issue Type: Improvement
> Reporter: Olga Natkovich
> Assignee: Richard Ding
>
> Now that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1096) Make all unit tests run in MR or local
mode based on a property
Posted by "Pradeep Kamath (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779575#action_12779575 ]
Pradeep Kamath commented on PIG-1096:
-------------------------------------
The load-store redesign to work with the proposal in http://wiki.apache.org/pig/LoadStoreRedesignProposal will also change unit tests a bit. We should delay this till the load-store-redesign branch is merged back to trunk.
In general there are a couple of issues which will need to be addressed to make this work.
* There is already a PigExecTestCase class which was introduced with the intent of making it easy to switch a test case to run in either mode - we would have model all the concerned tests to be subclass of this and work in that model
* In local mode, the input to a pig script would be a local file while in hadoop mode, it has to be a file on MiniCluster (there is a utility method Util.createInputFile() to create this file on the Minicluster). Tests which would need to work in both modes would need to abstract the setting up of the input so that the same test case can work in either mode.
> Make all unit tests run in MR or local mode based on a property
> ---------------------------------------------------------------
>
> Key: PIG-1096
> URL: https://issues.apache.org/jira/browse/PIG-1096
> Project: Pig
> Issue Type: Improvement
> Reporter: Olga Natkovich
>
> Now that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (PIG-1096) Make all unit tests run in MR or local
mode based on a property
Posted by "Richard Ding (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782070#action_12782070 ]
Richard Ding commented on PIG-1096:
-----------------------------------
In fixing unit tests due to the change introduced by PIG-879, a new helper method is added to test Util class:
{code}
static String generateURI(String filepath, PigContext context) throws IOException {
{code}
This method, based on the execution mode (MapReduce or Local), returns either a hadoopified hdfs file name or a local file name, and therefore provides a abstraction for the input so that the same time case can work in either mode.
> Make all unit tests run in MR or local mode based on a property
> ---------------------------------------------------------------
>
> Key: PIG-1096
> URL: https://issues.apache.org/jira/browse/PIG-1096
> Project: Pig
> Issue Type: Improvement
> Reporter: Olga Natkovich
>
> Now that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (PIG-1096) Make all unit tests run in MR or local
mode based on a property
Posted by "Olga Natkovich (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-1096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Olga Natkovich updated PIG-1096:
--------------------------------
Description: Now that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode. (was: No that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode.)
> Make all unit tests run in MR or local mode based on a property
> ---------------------------------------------------------------
>
> Key: PIG-1096
> URL: https://issues.apache.org/jira/browse/PIG-1096
> Project: Pig
> Issue Type: Improvement
> Reporter: Olga Natkovich
>
> Now that we have both local and MR mode based on Hadoop, it should be pretty easy to make all tests run in either mode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.