You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mrunit.apache.org by "Dave Beech (JIRA)" <ji...@apache.org> on 2012/06/09 14:25:42 UTC
[jira] [Created] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Dave Beech created MRUNIT-116:
---------------------------------
Summary: Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Key: MRUNIT-116
URL: https://issues.apache.org/jira/browse/MRUNIT-116
Project: MRUnit
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Dave Beech
Assignee: Dave Beech
Priority: Minor
Fix For: 1.0.0
In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure in inconsistent.
The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Dave Beech (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292345#comment-13292345 ]
Dave Beech commented on MRUNIT-116:
-----------------------------------
OK, committed - 1348413. Thanks Jim. No need to call getClass actually - the ReflectionUtils.setConf method takes the object itself.
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Dave Beech (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dave Beech resolved MRUNIT-116.
-------------------------------
Resolution: Fixed
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Jim Donofrio (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292340#comment-13292340 ]
Jim Donofrio commented on MRUNIT-116:
-------------------------------------
Yes +1 I agree. I guess you are just going to do a getClass on the Mapper or Reducer that gets passed in. I think eventually we should only allow the users to setup a JobConf so they can only set classes using the existing hadoop api and cannot create any of these objects.
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated MRUNIT-116:
--------------------------------
Fix Version/s: 0.9.1
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0, 0.9.1
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Dave Beech (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13292349#comment-13292349 ]
Dave Beech commented on MRUNIT-116:
-----------------------------------
I agree about the job setup - would be better to specify classes rather than create objects and let the API create them in the same way Hadoop would.
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Brock Noland (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brock Noland updated MRUNIT-116:
--------------------------------
Fix Version/s: (was: 0.9.1)
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MRUNIT-116) Setting of configuration on
Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
Posted by "Dave Beech (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MRUNIT-116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dave Beech updated MRUNIT-116:
------------------------------
Description:
In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
Unless anybody has objections, I'll commit this change.
was:
In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure in inconsistent.
The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
Unless anybody has objections, I'll commit this change.
> Setting of configuration on Mappers/Reducers in MapDriver and ReduceDriver is inconsistent
> ------------------------------------------------------------------------------------------
>
> Key: MRUNIT-116
> URL: https://issues.apache.org/jira/browse/MRUNIT-116
> Project: MRUnit
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Dave Beech
> Assignee: Dave Beech
> Priority: Minor
> Fix For: 1.0.0
>
>
> In the run methods of MapDriver and ReduceDriver (mapred API), the calling of setConf and configure is inconsistent.
> The check for instanceof Configurable and the call to setConf is missing from ReduceDriver.
> I suggest we replace these configuration checks and sets with a call to Hadoop's ReflectionUtils.setConf, since this method contains the logic to decide what to set where, based on whether the object is Configurable or JobConfigurable etc. It also handles the proposed deprecation of JobConf, so we wouldn't have to worry about that later.
> Unless anybody has objections, I'll commit this change.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira