You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Ryan Smith (JIRA)" <ji...@apache.org> on 2008/09/10 21:51:44 UTC
[jira] Updated: (HBASE-765) Adding basic Spring DI support to
IndexConfiguration class.
[ https://issues.apache.org/jira/browse/HBASE-765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ryan Smith updated HBASE-765:
-----------------------------
Description:
Spring can configure classes/object graphs via xml. I am pretty much able to configure the entire MR object graph to launch MR jobs via spring except class IndexConfiguration.java. So instead of only using addFromXML() to configure IndexConfiguration, it would be nice to add support so Spring could set all class variables needed for initialization in IndexConfiguration without invoking addFromXML().
Since the class IndexConfiguration already has setters and getters for almost all its members, it's almost compliant for a spring configuration bean except one issue: no ability to configure columnMap outside of calling addFromXML(). The easiest way i can figure is to allow a setter for the column map and put any logic for checking the map integrity there. By adding a few methods to IndexConfiguration.java , it should solve the issue.
I will be submitting a patch for it here shortly.....
EDIT: after further review, there are 2 more classes that are using addFromXML()
IndexOutputFormat and IndexTableReduce use the xml string to configure the indexconfig object.
I will have to modify these 3 classes so I will be developing the patch over this week, maybe next week too.
EDIT 2: It seems that the distributed nature of Hadoop's MapReduce api is getting in the way of spring loading these classes.
Since a new instance of new IndexConfiguration() needs to be called remotely, (on each node in the cluster running the MR job), it makes it hard to have spring create the class when ultimately, Hadoop has to create the class for each task, etc.
I am going to submit changes to hadoop issue 2866.
http://www.mail-archive.com/core-dev@hadoop.apache.org/msg18574.html
Or maybe create a new issue for it.
There is also the possibility of getting spring to support map reduce natively. Ill ask them about it, but for now, I would say that this should go on the back burner, or just keep punting it down until remote objects can get created via factory interfaces instead of reflection internally as it is now.
was:
Spring can configure classes/object graphs via xml. I am pretty much able to configure the entire MR object graph to launch MR jobs via spring except class IndexConfiguration.java. So instead of only using addFromXML() to configure IndexConfiguration, it would be nice to add support so Spring could set all class variables needed for initialization in IndexConfiguration without invoking addFromXML().
Since the class IndexConfiguration already has setters and getters for almost all its members, it's almost compliant for a spring configuration bean except one issue: no ability to configure columnMap outside of calling addFromXML(). The easiest way i can figure is to allow a setter for the column map and put any logic for checking the map integrity there. By adding a few methods to IndexConfiguration.java , it should solve the issue.
I will be submitting a patch for it here shortly.....
EDIT: after further review, there are 2 more classes that are using addFromXML()
IndexOutputFormat and IndexTableReduce use the xml string to configure the indexconfig object.
I will have to modify these 3 classes so I will be developing the patch over this week, maybe next week too.
> Adding basic Spring DI support to IndexConfiguration class.
> -----------------------------------------------------------
>
> Key: HBASE-765
> URL: https://issues.apache.org/jira/browse/HBASE-765
> Project: Hadoop HBase
> Issue Type: Improvement
> Components: mapred
> Affects Versions: 0.16.0, 0.1.0, 0.1.1, 0.1.2, 0.1.3
> Environment: n/a
> Reporter: Ryan Smith
> Priority: Minor
> Fix For: 0.19.0, 0.2.2
>
> Original Estimate: 0.33h
> Remaining Estimate: 0.33h
>
> Spring can configure classes/object graphs via xml. I am pretty much able to configure the entire MR object graph to launch MR jobs via spring except class IndexConfiguration.java. So instead of only using addFromXML() to configure IndexConfiguration, it would be nice to add support so Spring could set all class variables needed for initialization in IndexConfiguration without invoking addFromXML().
> Since the class IndexConfiguration already has setters and getters for almost all its members, it's almost compliant for a spring configuration bean except one issue: no ability to configure columnMap outside of calling addFromXML(). The easiest way i can figure is to allow a setter for the column map and put any logic for checking the map integrity there. By adding a few methods to IndexConfiguration.java , it should solve the issue.
> I will be submitting a patch for it here shortly.....
> EDIT: after further review, there are 2 more classes that are using addFromXML()
> IndexOutputFormat and IndexTableReduce use the xml string to configure the indexconfig object.
> I will have to modify these 3 classes so I will be developing the patch over this week, maybe next week too.
> EDIT 2: It seems that the distributed nature of Hadoop's MapReduce api is getting in the way of spring loading these classes.
> Since a new instance of new IndexConfiguration() needs to be called remotely, (on each node in the cluster running the MR job), it makes it hard to have spring create the class when ultimately, Hadoop has to create the class for each task, etc.
> I am going to submit changes to hadoop issue 2866.
> http://www.mail-archive.com/core-dev@hadoop.apache.org/msg18574.html
> Or maybe create a new issue for it.
> There is also the possibility of getting spring to support map reduce natively. Ill ask them about it, but for now, I would say that this should go on the back burner, or just keep punting it down until remote objects can get created via factory interfaces instead of reflection internally as it is now.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.