You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pig.apache.org by ga...@apache.org on 2009/05/15 23:13:21 UTC
svn commit: r775346 - in /hadoop/pig/branches/pre-multiquery-phase2:
CHANGES.txt src/org/apache/pig/backend/executionengine/PigSlicer.java
Author: gates
Date: Fri May 15 21:13:20 2009
New Revision: 775346
URL: http://svn.apache.org/viewvc?rev=775346&view=rev
Log:
PIG-619: Create one InputSplit even when the input file is zero length so that hadoop runs maps and creates output for the next job.
Modified:
hadoop/pig/branches/pre-multiquery-phase2/CHANGES.txt
hadoop/pig/branches/pre-multiquery-phase2/src/org/apache/pig/backend/executionengine/PigSlicer.java
Modified: hadoop/pig/branches/pre-multiquery-phase2/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/pig/branches/pre-multiquery-phase2/CHANGES.txt?rev=775346&r1=775345&r2=775346&view=diff
==============================================================================
--- hadoop/pig/branches/pre-multiquery-phase2/CHANGES.txt (original)
+++ hadoop/pig/branches/pre-multiquery-phase2/CHANGES.txt Fri May 15 21:13:20 2009
@@ -24,6 +24,10 @@
IMPROVEMENTS
+PIG-619: Create one InputSplit even when the input file is zero length
+ so that hadoop runs maps and creates output for the next
+ job (gates).
+
PIG-693: Proposed improvements to pig's optimizer (sms)
PIG-700: To automate the pig patch test process (gkesavan via sms)
Modified: hadoop/pig/branches/pre-multiquery-phase2/src/org/apache/pig/backend/executionengine/PigSlicer.java
URL: http://svn.apache.org/viewvc/hadoop/pig/branches/pre-multiquery-phase2/src/org/apache/pig/backend/executionengine/PigSlicer.java?rev=775346&r1=775345&r2=775346&view=diff
==============================================================================
--- hadoop/pig/branches/pre-multiquery-phase2/src/org/apache/pig/backend/executionengine/PigSlicer.java (original)
+++ hadoop/pig/branches/pre-multiquery-phase2/src/org/apache/pig/backend/executionengine/PigSlicer.java Fri May 15 21:13:20 2009
@@ -100,6 +100,12 @@
// Anything that ends with a ".gz" we must process as a complete
// file
slices.add(new PigSlice(name, funcSpec, 0, size));
+ } else if (size == 0) {
+ // add one empty slice. This is a total hack to deal with the
+ // case where hadoop isn't starting maps for empty arrays of
+ // InputSplits. See PIG-619. This should be removed
+ // once we determine why this is.
+ slices.add(new PigSlice(name, funcSpec, 0, bs));
} else {
while (pos < size) {
if (pos + bs > size) {