You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Alan Gates (JIRA)" <ji...@apache.org> on 2011/07/23 16:27:09 UTC
[jira] [Created] (PIG-2189) e2e test harness needs to use Pig as a
source of truth
e2e test harness needs to use Pig as a source of truth
------------------------------------------------------
Key: PIG-2189
URL: https://issues.apache.org/jira/browse/PIG-2189
Project: Pig
Issue Type: Improvement
Components: tools
Affects Versions: 0.10
Reporter: Alan Gates
Assignee: Alan Gates
Currently the end-to-end test harness requires a Postgresql database to generate expected results for many tests. This makes it hard to install and use in automated builds and by users who don't already have postgresql installed. As per the proposal put forth at https://cwiki.apache.org/confluence/display/PIG/PigTestProposal , we need to change the test harness to test either against old versions of Pig or against alternate Pig Latin scripts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2189) e2e test harness needs to use Pig as a
source of truth
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2189:
----------------------------
Fix Version/s: 0.10
Release Note:
Usage:
To use this feature cd to test/e2e/pig/ and do:
ant -Dpig.harness.old.pig=_path_to_old_pig_ -Dpig.harness.cluster=_path_to_top_cluster_dir_
where _path_to_old_pig_ is the path to where you have installed an older version of Pig. For example if you want to use 0.8.1 as the older version (recommended for current testing of 0.9 and trunk) and you have it installed in your home directory under tools, you would set this to "/homes/you/tools/pig-0.8.1".
and _path_top_top_cluster_dir is the path to where you cluster is installed. This must be the top directory (it should have bin, conf, etc. in it) not the conf directory where the *-site.xml files are. If your hadoop program is installed in "/grids/snoopy/bin/hadoop" then you would pass "/grid/snoopy" for this value.
Running the above will run all of the currently specified end-to-end tests. It takes around 8 hours currently.
If you wish to run just some of the tests you can define -Dtests.to.run=_tests_ where _tests_ is of the form "-t _test_or_group_". _test_or_group_ can be a group name (like Checkin) or a particular test (like Checkin_1). You can put multiple of these in one definition. So to run all the checkin tests and the 3rd multiquery test you would do -Dtests.to.run-"-t Checkin -t MultiQuery_3".
Before you run the tests the first time against a new cluster the harness can generate and deploy into the cluster all of the data you need. To do this do:
ant -Dpig.harness.old.pig=_path_to_old_pig_ -Dpig.harness.cluster=_path_to_top_cluster_dir_ deploy
This will not run the tests, just deploy the data. You can then run the tests.
Status: Patch Available (was: Open)
The attached patch removes use of the database as a source of truth and instead uses an old version of Pig by default, or an alternate Pig script if provided.
A number of tests do not yet work with this (mostly the Jython and Macro tests, for which I need to write alternate Pig Latin scripts, but a few others as well). These have all been marked ignore for now.
Local mode testing is not yet supported.
> e2e test harness needs to use Pig as a source of truth
> ------------------------------------------------------
>
> Key: PIG-2189
> URL: https://issues.apache.org/jira/browse/PIG-2189
> Project: Pig
> Issue Type: Improvement
> Components: tools
> Affects Versions: 0.10
> Reporter: Alan Gates
> Assignee: Alan Gates
> Fix For: 0.10
>
> Attachments: PIG-2189.patch
>
>
> Currently the end-to-end test harness requires a Postgresql database to generate expected results for many tests. This makes it hard to install and use in automated builds and by users who don't already have postgresql installed. As per the proposal put forth at https://cwiki.apache.org/confluence/display/PIG/PigTestProposal , we need to change the test harness to test either against old versions of Pig or against alternate Pig Latin scripts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2189) e2e test harness needs to use Pig as a
source of truth
Posted by "Daniel Dai (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daniel Dai updated PIG-2189:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Patch committed to trunk.
> e2e test harness needs to use Pig as a source of truth
> ------------------------------------------------------
>
> Key: PIG-2189
> URL: https://issues.apache.org/jira/browse/PIG-2189
> Project: Pig
> Issue Type: Improvement
> Components: tools
> Affects Versions: 0.10
> Reporter: Alan Gates
> Assignee: Alan Gates
> Fix For: 0.10
>
> Attachments: PIG-2189.patch
>
>
> Currently the end-to-end test harness requires a Postgresql database to generate expected results for many tests. This makes it hard to install and use in automated builds and by users who don't already have postgresql installed. As per the proposal put forth at https://cwiki.apache.org/confluence/display/PIG/PigTestProposal , we need to change the test harness to test either against old versions of Pig or against alternate Pig Latin scripts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2189) e2e test harness needs to use Pig as a
source of truth
Posted by "Alan Gates (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alan Gates updated PIG-2189:
----------------------------
Attachment: PIG-2189.patch
> e2e test harness needs to use Pig as a source of truth
> ------------------------------------------------------
>
> Key: PIG-2189
> URL: https://issues.apache.org/jira/browse/PIG-2189
> Project: Pig
> Issue Type: Improvement
> Components: tools
> Affects Versions: 0.10
> Reporter: Alan Gates
> Assignee: Alan Gates
> Attachments: PIG-2189.patch
>
>
> Currently the end-to-end test harness requires a Postgresql database to generate expected results for many tests. This makes it hard to install and use in automated builds and by users who don't already have postgresql installed. As per the proposal put forth at https://cwiki.apache.org/confluence/display/PIG/PigTestProposal , we need to change the test harness to test either against old versions of Pig or against alternate Pig Latin scripts.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira