You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Bill Graham (Created) (JIRA)" <ji...@apache.org> on 2012/03/08 03:34:59 UTC
[jira] [Created] (PIG-2574) Make reducer estimator plugable
Make reducer estimator plugable
-------------------------------
Key: PIG-2574
URL: https://issues.apache.org/jira/browse/PIG-2574
Project: Pig
Issue Type: Improvement
Reporter: Bill Graham
Assignee: Bill Graham
I'd like to refactor the logic contained in this method into a pluggable interface:
{noformat}
static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_5.patch
Per discussions on the list, I've moved the test class to a package that matches the class being tested. OK, now I'm done until further review.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2574) Make reducer estimator plugable
Posted by "Julien Le Dem (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240970#comment-13240970 ]
Julien Le Dem commented on PIG-2574:
------------------------------------
Overall, this looks good to me.
Here are my comments, mostly details. Some of them are about existing code that has been moved.
1)
{code}
return listener != null ? (PigProgressNotificationListener) listener : null;
{code}
if listener is null you can still cast it.
just return (PigProgressNotificationListener) listener;
2)
{code}
int reducers = (int)Math.ceil((totalInputFileSize+0.0) / bytesPerReducer);
reducers = Math.max(1, reducers);
reducers = Math.min(maxReducers, reducers);
{code}
use a cast (double)totalInputFileSize to convert to double.
cast the final result to int instead. That will avoid bad conversion to integer if bytesPerReducer is small.
3)
{code}
public static Object instantiateObjectFromParams(Configuration conf, String classParamKey, String argParamKey) {
{code}
you can do the following so that classes using it don't need to cast. You still get a ClassCastException if it's not the correct type.
{code}
@SuppressWarnings("unchecked")
public static <T> T instantiateObjectFromParams(Configuration conf, String classParamKey, String argParamKey) {
...
return (T)result;
}
{code}
If you want to handle the cast with a better message you can do this (notice we don't need to suppress warnings anymore):
{code}
public static <T> T get(Configuration conf, String classParamKey, String argParamKey, Class<T> clazz) {
try {
...
return clazz.cast(result);
} catch (ClassCastException e) {
throw new ExecException("the class defined by "+classParamKey+" in conf is not of type "+clazz.getName(), e);
}
}
{code}
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Release Note: Support for custom PigReducerEstimator class via pig.exec.reducer.estimator and pig.exec.reducer.estimator.args.
Status: Patch Available (was: Open)
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13254528#comment-13254528 ]
Bill Graham commented on PIG-2574:
----------------------------------
Not committed to 0.10 FYI. The patch isn't applying cleanly so I need to dig in a bit.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10.0, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch, pig-2574_6.patch, pig-2574_7.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Jonathan Coveney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Coveney updated PIG-2574:
----------------------------------
Fix Version/s: 0.11
0.10
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_3.patch
Patch #3 includes changes from PIG-2573 merged in.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_2.patch
Patch #1 had some bad diff action going on. Please disregard. Here's patch #2 for review.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: pig-2574_1.patch, pig-2574_2.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_4.patch
Here's a patch with Dmitriy's comments, plus a few other small edits:
* Incorporated the fix from PIG-2616 into {{InputSizeReducerEstimator}}.
* Added both {{@InterfaceAudience.Public}} and {{@InterfaceStability.Evolving}} to {{PigReducerEstimator}}.
* Moved some tests from {{TestJobControlCompiler}} to {{TestInputSizeReducerEstimator}}.
Also, I added the following to {{.gitignore}}. Let me know if I should back that out and put it in it's own patch:
{noformat}
+target/
+*.iml
+*.ipr
+*.iws
{noformat}
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_1.patch
Here's a first pass at a patch showing a proposed solution. It adds the following interface and default impl:
{noformat}
o.a.p.backend.hadoop.executionengine.mapReduceLayer.PigReducerEstimator
o.a.p.backend.hadoop.executionengine.mapReduceLayer.InputFileSizeReducerEstimator
{noformat}
along with the following two optional configs to override the default estimator:
{noformat}
pig.exec.reducer.estimator
pig.exec.reducer.estimator.arg
{noformat}
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: pig-2574_1.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Fix Version/s: (was: 0.10.0)
Labels: (was: 0.10_blocker)
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Fix For: 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch, pig-2574_6.patch, pig-2574_7.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed to trunk.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10.0, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch, pig-2574_6.patch, pig-2574_7.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2574) Make reducer estimator plugable
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13234021#comment-13234021 ]
Dmitriy V. Ryaboy commented on PIG-2574:
----------------------------------------
Looks good overall. I didn't run tests.
* Please mark all new APIs as evolving
* Please add all new configurable keys to pig.properties, commented out, with instructions for people regarding what they mean
* "@param job job configuration" -- it's actually a Job, not just the config, right?
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2574) Make reducer estimator plugable
Posted by "Dmitriy V. Ryaboy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13245029#comment-13245029 ]
Dmitriy V. Ryaboy commented on PIG-2574:
----------------------------------------
+1
Looks good. Please commit when you get your keys :).
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch, pig-2574_6.patch, pig-2574_7.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2574) Make reducer estimator plugable
Posted by "Travis Crawford (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224953#comment-13224953 ]
Travis Crawford commented on PIG-2574:
--------------------------------------
This is somewhat related to PIG-2573. In that bug I'm interested in making the default size-based estimator work with HCatalog. In this bug Bill is interested in using additional data to estimate the number of reducers.
For example, if you want to produce counts for some field in a large data set, you may just want one reducer. However, if your input data size is large you'll get a large number of reducers too. A plugin could look at historical stats for this job and choose an appropriate number of reducers.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_7.patch
Made two additional changes to make it cleaner when implementing {{PigReducerEstimator}}.
* Moved constants out of {{InputSizeReducerEstimator}} into it's interface {{PigReducerEstimator}}, since implementing classes might need these.
* Setting {{mapred.reduce.tasks}} in {{JobControlCompiler}} since it shouldn't be the responsibility of {{PigReducerEstimator}} to set this value, just to calculate it and return it.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch, pig-2574_6.patch, pig-2574_7.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2574) Make reducer estimator plugable
Posted by "Bill Graham (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2574:
-----------------------------
Attachment: pig-2574_6.patch
Thanks Julien, here's a revised patch.
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch, pig-2574_4.patch, pig-2574_5.patch, pig-2574_6.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf, List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira