You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pig.apache.org by jc...@apache.org on 2012/03/27 03:11:37 UTC
svn commit: r1305701 - in /pig/branches/branch-0.10: CHANGES.txt
test/e2e/pig/deployers/LocalDeployer.pm
Author: jcoveney
Date: Tue Mar 27 01:11:36 2012
New Revision: 1305701
URL: http://svn.apache.org/viewvc?rev=1305701&view=rev
Log:
[PIG-2618] e2e fails to build
Modified:
pig/branches/branch-0.10/CHANGES.txt
pig/branches/branch-0.10/test/e2e/pig/deployers/LocalDeployer.pm
Modified: pig/branches/branch-0.10/CHANGES.txt
URL: http://svn.apache.org/viewvc/pig/branches/branch-0.10/CHANGES.txt?rev=1305701&r1=1305700&r2=1305701&view=diff
==============================================================================
--- pig/branches/branch-0.10/CHANGES.txt (original)
+++ pig/branches/branch-0.10/CHANGES.txt Tue Mar 27 01:11:36 2012
@@ -70,10 +70,10 @@ PIG-1387: Syntactical Sugar for PIG-1385
PIG-2305: Pig should log the split locations in task logs (vivekp via thejas)
-PIG-2293: Pig should support a more efficient merge join against data sources that natively support point
+PIG-2293: Pig should support a more efficient merge join against data sources that natively support point
lookups or where the join is against large, sparse tables (aklish via daijy)
-PIG-2287: add test cases for limit and sample that use expressions with
+PIG-2287: add test cases for limit and sample that use expressions with
constants only (no scalar variables) (thejas via gates)
PIG-2092: Missing sh command from Grant shell (olgan)
@@ -119,12 +119,12 @@ PIG-2189: e2e test harness needs to use
PIG-1904: Default split destination (azaroth via thejas)
-PIG-2143: Make PigStorage optionally store schema; improve docs. (dvryaboy)
+PIG-2143: Make PigStorage optionally store schema; improve docs. (dvryaboy)
-PIG-1973: UDFContext.getUDFContext usage of ThreadLocal pattern
+PIG-1973: UDFContext.getUDFContext usage of ThreadLocal pattern
is not typical (woody via thejas)
-PIG-2053: PigInputFormat uses class.isAssignableFrom() where
+PIG-2053: PigInputFormat uses class.isAssignableFrom() where
instanceof is more appropriate (woody via thejas)
PIG-2161: TOTUPLE should use no-copy tuple creation (dvryaboy)
@@ -132,8 +132,8 @@ PIG-2161: TOTUPLE should use no-copy tup
PIG-1946: HBaseStorage constructor syntax is error prone (billgraham via dvryaboy)
PIG-2001: DefaultTuple(List) constructor is inefficient, causes List.size()
- System.arraycopy() calls (though they are 0 byte copies),
- DefaultTuple(int) constructor is a bit misleading wrt time
+ System.arraycopy() calls (though they are 0 byte copies),
+ DefaultTuple(int) constructor is a bit misleading wrt time
complexity (woody via thejas)
PIG-1916: Nested cross (zjshen via daijy)
@@ -170,7 +170,7 @@ PIG-2036: [piggybank] Set header delimit
PIG-1949: e2e test harness should use bin/pig rather than calling java
directly (gates)
-PIG-2026: e2e tests in eclipse classpath (azaroth via hashutosh)
+PIG-2026: e2e tests in eclipse classpath (azaroth via hashutosh)
PIG-2024: Incorrect jar paths in .classpath template for eclipse (azaroth via hashutosh)
@@ -182,6 +182,8 @@ PIG-2228: support partial aggregation in
BUG FIXES
+PIG-2618: e2e local fails to build
+
PIG-2612: e2e harness: should not require PH_OLD_CLUSTER_CONF (thw via daijy)
PIG-2505: AvroStorage won't read any file not ending in .avro (russell.jurney via daijy)
@@ -214,7 +216,7 @@ PIG-2400: Document has based aggregation
PIG-2444: Remove the Zebra *.xml documentation files from the TRUNK and Branch-10 (chandec via daijy)
-PIG-2430: An EvalFunc which overrides getArgToFuncMapping with FuncSpec
+PIG-2430: An EvalFunc which overrides getArgToFuncMapping with FuncSpec
with constructor arguments is not properly instantiated with said arguments (jcoveney via thejas)
PIG-2457: JsonLoaderStorage tests is broken for e2e (daijy)
@@ -255,12 +257,12 @@ PIG-2119: DuplicateForEachColumnRewrite
PIG-2290: TOBAG wraps tuple parameters in another tuple (ryan.hoegg via thejas)
-PIG-2288: Pig 0.9 error message not useful as compared to 0.8 in case
+PIG-2288: Pig 0.9 error message not useful as compared to 0.8 in case
of group by (vivekp via thejas)
PIG-2309: Keyword 'NOT' is wrongly treated as a UDF in split statement (vivekp via thejas)
-PIG-2307: Jetty version should be updated in .eclipse.templates/.classpath,
+PIG-2307: Jetty version should be updated in .eclipse.templates/.classpath,
pig-template.xml and pig.pom as well (zjshen via daijy)
PIG-2273: Pig.compileFromFile in embedded python fails when pig script starts with a comment (ddaniels888 via gates)
@@ -293,12 +295,12 @@ PIG-2124: Script never ending when joini
PIG-2170: NPE thrown during illustrate (thejas)
-PIG-2186: PigStorage new warnings about missing schema file
+PIG-2186: PigStorage new warnings about missing schema file
can be confusing (thejas)
PIG-2179: tests in TestLoad are failing (thejas)
-PIG-2146: POStore.getSchema() returns null because of which PigOutputCommitter
+PIG-2146: POStore.getSchema() returns null because of which PigOutputCommitter
is not storing schema while cleanup (thejas)
PIG-2027: NPE if Pig don't have permission for log file (daijy)
@@ -311,7 +313,7 @@ PIG-2162: bin/pig should not modify user
PIG-2060: Fix errors in pig grammars reported by ANTLRWorks (azaroth via thejas)
-PIG-2156: Limit/Sample with variable does not work if the expression starts
+PIG-2156: Limit/Sample with variable does not work if the expression starts
with an integer/double (azaroth via thejas)
PIG-2130: Piggybank:MultiStorage is not compressing output files (vivekp via daijy)
@@ -329,7 +331,7 @@ PIG-2139: LogicalExpressionSimplifier op
PIG-2137: SAMPLE should not be pushed above DISTINCT (dvryaboy and thejas)
-PIG-2136: Implementation of Sample should use LessThanExpression
+PIG-2136: Implementation of Sample should use LessThanExpression
instead of LessThanEqualExpression (azaroth via thejas)
PIG-2140: Usage printed from Main.java gives wrong option for disabling
@@ -347,7 +349,7 @@ PIG-1702: Streaming debug output outputs
PIG-2109: Ant build continues even if the parser classes fail to be generated. (zjshen via daijy)
-PIG-2071: casting numeric type to chararray during schema merge for union
+PIG-2071: casting numeric type to chararray during schema merge for union
is inconsistent with other schema merge cases (thejas)
PIG-2044: Patten match bug in org.apache.pig.newplan.optimizer.Rule (knoguchi via daijy)
@@ -558,7 +560,7 @@ PIG-1921: Improve error messages in new
PIG-1996: Pig new parser fails to recognize PARALLEL keywords in a case (xuefu)
-PIG-1612: error reporting: PigException needs to have a way to indicate that
+PIG-1612: error reporting: PigException needs to have a way to indicate that
its message is appropriate for user (laukik via thejas)
PIG-1782: Add ability to load data by column family in HBaseStorage (billgraham via dvryaboy)
@@ -590,8 +592,8 @@ PIG-1932: GFCross should allow the user
PIG-1913: Use a file for excluding tests (tomwhite via gates)
-PIG-1693: support project-range expression. (was: There
- needs to be a way in foreach to indicate "and all the
+PIG-1693: support project-range expression. (was: There
+ needs to be a way in foreach to indicate "and all the
rest of the fields" ) (thejas)
PIG-1772: Pig 090 Documentation (chandec via daijy)
@@ -645,7 +647,7 @@ PIG-1755: Clean up duplicated code in Ph
PIG-750: Use combiner when algebraic UDFs are used in expressions (thejas)
-PIG-490: Combiner not used when group elements referred to in
+PIG-490: Combiner not used when group elements referred to in
tuple notation instead of flatten. (thejas)
PIG-1768: 09 docs: illustrate (changec via olgan)
@@ -668,7 +670,7 @@ against newer hadoop versions (pradeepkt
PIG-1618: Switch to new parser generator technology (xuefuz via thejas)
-PIG-1531: Pig gobbles up error messages (nrai via hashutosh)
+PIG-1531: Pig gobbles up error messages (nrai via hashutosh)
PIG-1508: Make 'docs' target (forrest) work with Java 1.6 (cwsteinbach via gates)
@@ -688,7 +690,7 @@ PIG-2083: bincond ERROR 1025: Invalid fi
PIG-2089: Javadoc for ResourceFieldSchema.getSchema() is wrong (daijy)
-PIG-2084: pig is running validation for a statement at a time batch mode,
+PIG-2084: pig is running validation for a statement at a time batch mode,
instead of running it for whole script (thejas)
PIG-2088: Return alias validation failed when there is single line comment in the macro (rding)
@@ -703,7 +705,7 @@ PIG-2070: "Unknown" appears in error mes
PIG-2069: LoadFunc jar does not ship to backend in MultiQuery case (rding)
-PIG-2076: update documentation, help command with correct default value
+PIG-2076: update documentation, help command with correct default value
of pig.cachedbag.memusage (thejas)
PIG-2072: NPE when udf has project-star argument and input schema is null (thejas)
@@ -746,7 +748,7 @@ PIG-1990: support casting of complex typ
PIG-2016: -dot option does not work with explain and new logical plan (daijy)
-PIG-2018: NPE for co-group with group-by column having complex schema and
+PIG-2018: NPE for co-group with group-by column having complex schema and
different load functions for each input (thejas)
PIG-2015: Explain writes out logical plan twice (alangates)
@@ -769,11 +771,11 @@ PIG-1910: incorrect schema shown when pr
PIG-2005: Discrepancy in the way dry run handles semicolon in macro definition (rding)
-PIG-1281: Detect org.apache.pig.data.DataByteArray cannot be cast to
- org.apache.pig.data.Tuple type of errors at Compile Type during
+PIG-1281: Detect org.apache.pig.data.DataByteArray cannot be cast to
+ org.apache.pig.data.Tuple type of errors at Compile Type during
creation of logical plan (thejas)
-PIG-1939: order-by statement should support project-range to-end in
+PIG-1939: order-by statement should support project-range to-end in
any position among the sort columns if input schema is known (thejas)
PIG-1978: Secondary sort fail when dereferencing two fields inside foreach (daijy)
@@ -788,7 +790,7 @@ PIG-1871: Dont throw exception if partit
PIG-1870: HBaseStorage doesn't project correctly (dvryaboy)
-PIG-1788: relation-as-scalar error messages should indicate the field
+PIG-1788: relation-as-scalar error messages should indicate the field
being used as scalar (laukik via thejas)
PIG-1697: NullPointerException if log4j.properties is Used (laukik via daijy)
@@ -799,10 +801,10 @@ PIG-1928: Type Checking, incorrect error
PIG-1979: New logical plan failing with ERROR 2229: Couldn't find matching uid -1 (daijy)
-PIG-1897: multiple star projection in a statement does not produce
+PIG-1897: multiple star projection in a statement does not produce
the right plan (thejas)
-PIG-1917: NativeMapReduce does not Allow Configuration Parameters
+PIG-1917: NativeMapReduce does not Allow Configuration Parameters
containing Spaces (thejas)
PIG-1974: Lineage need to set for every cast (thejas)
@@ -839,7 +841,7 @@ PIG-1934: Fix zebra test TestCheckin1, T
PIG-1931: Integrate Macro Expansion with New Parser (rding)
-PIG-1933: Hints such as 'collected' and 'skewed' for "group by" or "join by"
+PIG-1933: Hints such as 'collected' and 'skewed' for "group by" or "join by"
should not be treated as tokens. (xuefuz via thejas)
PIG-1925: Parser error message doesn't show location of the error or show it
@@ -1065,10 +1067,10 @@ PIG-1317: LOLoad should cache results of
subsequent calls to LOLoad.getSchema() or LOLoad.determineSchema()
(pradeepkth)
-PIG-1413: Remove svn:externals reference for test-patch.sh and
+PIG-1413: Remove svn:externals reference for test-patch.sh and
create a local copy of test-patch.sh (gkesavan)
-PIG-1302: Include zebra's "pigtest" ant target as a part of pig's
+PIG-1302: Include zebra's "pigtest" ant target as a part of pig's
ant test target. (gkesavan)
PIG-1582: To upgrade commons-logging
@@ -1114,7 +1116,7 @@ PIG-1812: Problem with DID_NOT_FIND_LOAD
PIG-1813: Pig 0.8 throws ERROR 1075 while trying to refer a map in the result
of eval udf.Works with 0.7 (daijy)
-PIG-1776: changing statement corresponding to alias after explain , then
+PIG-1776: changing statement corresponding to alias after explain , then
doing dump gives incorrect result (thejas)
PIG-1800: Missing Signature for maven staging release (rding)
@@ -1221,22 +1223,22 @@ PIG-1664: leading '_' in directory/file
PIG-1662: Need better error message for MalFormedProbVecException (rding)
-PIG-1656: TOBAG udfs ignores columns with null value; it does not use input type
+PIG-1656: TOBAG udfs ignores columns with null value; it does not use input type
to determine output schema (thejas)
PIG-1658: ORDER BY does not work properly on integer/short keys that are -1 (yanz)
PIG-1638: sh output gets mixed up with the grunt prompt (nrai via daijy)
-PIG-1607: pig should have separate javadoc.jar in the maven
+PIG-1607: pig should have separate javadoc.jar in the maven
repository (nrai via thejas)
PIG-1651: PIG class loading mishandled (rding)
-PIG-1650: pig grunt shell breaks for many commands like perl , awk ,
+PIG-1650: pig grunt shell breaks for many commands like perl , awk ,
pipe , 'ls -l' etc (nrai via thejas)
-PIG-1649: FRJoin fails to compute number of input files for replicated
+PIG-1649: FRJoin fails to compute number of input files for replicated
input (thejas)
PIG-1637: Combiner not use because optimizor inserts a foreach between group
@@ -1261,10 +1263,10 @@ PIG-1645: Using both small split combina
PIG-1635: Logical simplifier does not simplify away constants under AND and OR; after simplificaion the ordering of operands of
AND and OR may get changed (yanz)
-PIG-1639: New logical plan: PushUpFilter should not push before group/cogroup
+PIG-1639: New logical plan: PushUpFilter should not push before group/cogroup
if filter condition contains UDF (xuefuz via daijy)
-PIG-1643: join fails for a query with input having 'load using pigstorage
+PIG-1643: join fails for a query with input having 'load using pigstorage
without schema' + 'foreach' (thejas)
PIG-1628: log this message at debug level : 'Pig Internal storage in use' (thejas)
@@ -1276,7 +1278,7 @@ PIG-1605: PIG-1605: Adding soft link to
PIG-1598: Pig gobbles up error messages - Part 2 (nrai via daijy)
-PIG-1616: 'union onschema' does not use create output with correct schema
+PIG-1616: 'union onschema' does not use create output with correct schema
when udfs are involved (thejas)
PIG-1610: 'union onschema' does handle some cases involving 'namespaced'
@@ -1409,7 +1411,7 @@ PIG-1414: Problem with parameter substit
PIG-1407: Logging starts before being configured (azaroth via daijy)
-PIG-1391: pig unit tests leave behind files in temp directory because
+PIG-1391: pig unit tests leave behind files in temp directory because
MiniCluster files don't get deleted (thejas)
PIG-1211: Pig script runs half way after which it reports syntax error
@@ -1457,7 +1459,7 @@ in the path and does not consider JAVA_H
PIG-1352: piggybank UPPER udf throws exception if argument is null
-PIG-1560: Fix ant target checkstyle (gkesavan)
+PIG-1560: Fix ant target checkstyle (gkesavan)
Release 0.7.0
@@ -1551,7 +1553,7 @@ PIG-1218: Use distributed cache to store
PIG-1226: suuport for additional jar files (thejas via olgan)
PIG-1230: Streaming input in POJoinPackage should use nonspillable bag to
-collect tuples (ashutoshc)
+collect tuples (ashutoshc)
PIG-1224: Collected group should change to use new (internal) bag (ashutoshc)
@@ -1584,7 +1586,7 @@ PIG-1156: Add aliases to ExecJobs and Ph
PIG-1161: add missing license headers (dvryaboy via olgan)
-PIG-760: Add a new PigStorageSchema load/store function that
+PIG-760: Add a new PigStorageSchema load/store function that
store schemas for text files (dvryaboy via gates)
PIG-1106: FR join should not spill (ankit.modi via olgan)
@@ -1695,7 +1697,7 @@ PIG-834: incorrect plan when algebraic f
PIG-1217: Fix argToFuncMapping in Piggybank Top function (dvryaboy via gates)
-PIG-1154: Local Mode fails when hadoop config directory is specified in
+PIG-1154: Local Mode fails when hadoop config directory is specified in
classpath (ankit.modi via gates)
PIG-1124: Unable to set Custom Job Name using the -Dmapred.job.name parameter (ashutoshc)
@@ -2067,7 +2069,7 @@ PIG-697: Proposed improvements to pig's
PIG-753: Allow UDFs with no parameters (zjffdu via gates)
-PIG-765: jdiff for pig ( gkesavan
+PIG-765: jdiff for pig ( gkesavan
OPTIMIZATIONS
@@ -2082,7 +2084,7 @@ BUG FIXES
PIG-957: Tutorial is broken with 0.4 branch and trunk (pradeepkth)
PIG-955: Skewed join produces invalid results (yinghe via olgan)
-
+
PIG-954: Skewed join fails when pig.skewedjoin.reduce.memusage is not
configured(yinghe via olgan)
@@ -2111,7 +2113,7 @@ BUG FIXES
PIG-882: log level not propogated to loggers (daijy)
PIG-880: Order by is borken with complex fields (sms)
-
+
PIG-773: Empty complex constants (empty bag, empty tuple and empty map)
should be supported (ashutoshc via sms)
@@ -2135,7 +2137,7 @@ BUG FIXES
PIG-851: Map type used as return type in UDFs not recognized at all times
(zjffdu via sms)
-
+
PIG-861: POJoinPackage lose tuple in large dataset (daijy)
PIG-797: Limit with ORDER BY producing wrong results (daijy)
@@ -2249,7 +2251,7 @@ for UDFs that want to handle all simple
PIG-514: COUNT returns no results as a result of two filter statements in
FOREACH (pradeepkth)
-PIG-789: Fix dump and illustrate to work with new multi-query feature
+PIG-789: Fix dump and illustrate to work with new multi-query feature
(hagleitn via gates)
PIG-774: Pig does not handle Chinese characters (in both the parameter subsitution
@@ -2269,7 +2271,7 @@ Release 0.2.0
INCOMPATIBLE CHANGES
PIG-157: Add types and rework execution pipeline (gates)
-
+
PIG-458: integration with Hadoop 18 (olgan)
NEW FEATURES
@@ -2289,13 +2291,13 @@ NEW FEATURES
IMPROVEMENTS
PIG-270: proper line number for parse errors (daijy via olgan)
-
+
PIG-367: convinience function for UDFs to name schema
PIG-443: Illustrate for the Types branch (shubhamc via olgan)
PIG-599: Added buffering to BufferedPositionedInputStream (gates)
-
+
PIG-629: performance improvement: getting rid of targeted tuple (pradeepkth
via olgan)
@@ -2382,7 +2384,7 @@ BUG FIXES
correctly (pradeepkth vi olgan)
PIG-421: error with complex nested plan (sms via olgan)
-
+
PIG-429: Self join wth implicit split has the join output in wrong order
(pradeepkth via olgan)
@@ -2444,7 +2446,7 @@ BUG FIXES
PIG-463: POCast changes (pradeepkth via olgan)
PIG-427: casting input to UDFs
-
+
PIG-437: as in alias names causing problems (sms via olgan)
PIG-54: MIN/MAX don't deal with invalid data (pradeepkth via olgan)
@@ -2587,7 +2589,7 @@ BUG FIXES
PIG-590: error handling on the backend (sms)
- PIG-658: Data type long : When 'L' or 'l' is included with data
+ PIG-658: Data type long : When 'L' or 'l' is included with data
(123L or 123l) load produces null value. Also the case with Float (thejas
via sms)
@@ -2665,13 +2667,13 @@ Release 0.1.0 - 2008-09-11
NEW FEATURES
PIG-20 Added custom comparator functions for order by (phunt via gates)
-
+
PIG-94: Streaming implementation (arunc via olgan)
-
+
PIG-58: parameter substitution
PIG-55: added custom splitter (groves via olgan)
-
+
PIG-59: Add a new ILLUSTRATE command (shubhamc via gates)
PIG-256: Added variable argument support for UDFs (pi_song)
@@ -2679,9 +2681,9 @@ Release 0.1.0 - 2008-09-11
IMPROVEMENTS:
PIG-8 added binary comparator (olgan)
-
+
PIG-11 Add capability to search for jar file to register. (antmagna via olgan)
-
+
PIG-7: Added use of combiner in some restricted cases. (gates)
PIG-47: Added methods to DataMap to provide access to its content
@@ -2691,7 +2693,7 @@ Release 0.1.0 - 2008-09-11
PIG-12: Added time stamps to log4j messages (phunt via gates)
- PIG-44: Added adaptive decision of the number of records to hold in memory
+ PIG-44: Added adaptive decision of the number of records to hold in memory
before spilling (utkarsh)
PIG-56: Made DataBag implement Iterable. (groves via gates)
@@ -2715,7 +2717,7 @@ Release 0.1.0 - 2008-09-11
PIG-106: Change StringBuffer and String '+' to StringBuilder (francisoud via gates)
PIG-111: Reworked configuration to be setable via properties. (joa23, pi_song, oae via gates)
-
+
BUG FIXES
PIG-24 Files that were incorrectly placed under test/reports have been
removed. ant clean now cleans test/reports. (milindb via gates)
@@ -2809,7 +2811,7 @@ Release 0.1.0 - 2008-09-11
PIG-110: Replaced code accidently merged out in PIG-32 fix that handled
flattening the combiner case. (gates and oae)
- PIG-213: Remove non-static references to logger from data bags and tuples,
+ PIG-213: Remove non-static references to logger from data bags and tuples,
as it causes significant overhead (vgeschel via gates)
PIG-284: target for building source jar (oae via olgan)
Modified: pig/branches/branch-0.10/test/e2e/pig/deployers/LocalDeployer.pm
URL: http://svn.apache.org/viewvc/pig/branches/branch-0.10/test/e2e/pig/deployers/LocalDeployer.pm?rev=1305701&r1=1305700&r2=1305701&view=diff
==============================================================================
--- pig/branches/branch-0.10/test/e2e/pig/deployers/LocalDeployer.pm (original)
+++ pig/branches/branch-0.10/test/e2e/pig/deployers/LocalDeployer.pm Tue Mar 27 01:11:36 2012
@@ -208,7 +208,7 @@ sub generateData
);
# Create the target directories
- for my $dir ("singlefile", "dir", "glob/star/somegood",
+ for my $dir ("singlefile", "dir", "types", "glob/star/somegood",
"glob/star/moregood", "glob/star/bad") {
my @cmd = ("mkdir", "-p", "$cfg->{'inpathbase'}/$dir");
$self->runCmd($log, \@cmd);