You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Alejandro Abdelnur (JIRA)" <ji...@apache.org> on 2011/08/20 01:05:27 UTC
[jira] [Created] (HADOOP-7560) Make hadoop-common a POM module with
sub-modules (common & alfredo)
Make hadoop-common a POM module with sub-modules (common & alfredo)
-------------------------------------------------------------------
Key: HADOOP-7560
URL: https://issues.apache.org/jira/browse/HADOOP-7560
Project: Hadoop Common
Issue Type: Sub-task
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
I.e.:
mkdir hadoop-common-project
mv hadoop-common hadoop-common-project
mv hadoop-alfredo hadoop-common-project
hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088893#comment-13088893 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Also, is it necessary to move hadoop-common to hadoop-common-project?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Giridharan Kesavan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090532#comment-13090532 ]
Giridharan Kesavan commented on HADOOP-7560:
--------------------------------------------
+1 to Eric's proposal.
Not everyone would want to checkout the whole trunk to just build mapreduce or hdfs or yarn.
With the Hierarchical approach we get the liberty of just checking out hadoop-hdfs or hadoop-mapreduce or hadoop-yarn and build it without having to checkout the whole trunk. And, anyways maven would take care of the dependency resolutions.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090647#comment-13090647 ]
Hudson commented on HADOOP-7560:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #787 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/787/])
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
* /hadoop/common/trunk/hadoop-common-project/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
* /hadoop/common/trunk/hadoop-annotations
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
* /hadoop/common/trunk/hadoop-alfredo
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
* /hadoop/common/trunk/hadoop-mapreduce-project
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/trunk/hadoop-common-project
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth
* /hadoop/common/trunk/hadoop-hdfs
* /hadoop/common/trunk/hadoop-mapreduce
* /hadoop/common/trunk/hadoop-common
* /hadoop/common/trunk/hadoop-common-project/hadoop-common
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7560) Make hadoop-common a POM module with
sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7560:
---------------------------------------
Attachment: HADOOP-7560v2.patch
HADOOP-7560v2.sh
v2 script/patch (same instructions) set a hierarchical layout.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090454#comment-13090454 ]
Eric Yang commented on HADOOP-7560:
-----------------------------------
Alejandro, the proposed approaches are two extreme models. How about we have a sensible approach to group top level modules and submodules?
For example:
{noformat}
trunk/
hadopo-project/ (f)
hadoop-common/ (f)
hadoop-hdfs/ (h)
hadoop-hdfs-http-client/ (f) (this is hoop)
hadoop-hdfs-http-server/ (f) (this is hoop)
hadoop-mapreduce/ (h) (*)
hadoop-mr-client/ (*)
hadoop-mapreduce-client-app/
hadoop-mapreduce-client-common/
hadoop-mapreduce-client-core/
hadoop-mapreduce-client-hs/
hadoop-mapreduce-client-jobclient/
hadoop-mapreduce-client-shuffle/
hadoop-yarn/ (*)
hadoop-yarn-api/
hadoop-yarn-common/
hadoop-yarn-server/
{noformat}
In the proposed format, the structure of the tree is driven by the context and importance of the module.
IMHO, hadoop-project-dist, hadoop-assemblies, hadoop-annotations should not be modules. They are small plugins and can be grouped as hadoop-project. hadoop-auth should be part of hadoop-common. This will remove the noise for the project organization.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089743#comment-13089743 ]
Mahadev konar commented on HADOOP-7560:
---------------------------------------
bq. How about adding a top-level hadoop-extras and putting hadoop-alfredo there?
+1. I like this idea. I am also not in favor of another set of svn moves. Its very disrupting.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090593#comment-13090593 ]
Tom White commented on HADOOP-7560:
-----------------------------------
I prefer v1, however I think reasonable people can differ on flat vs. nested, and there is no overwhelming consensus on other Maven projects for one as far as I know. Having consistency is more important, and either one of these patches will improve that.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088901#comment-13088901 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
@Arun, hadoop-alfredo is a module used by hadoop-common. Until now contrib modules have been always the other way around (built on top of common/hdfs/mapreduce).
I think we need to discuss a bit the general layout of things. IMO, MR2 module approach is good, shouldn't we do something similar for common & hdfs?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089549#comment-13089549 ]
Eric Yang commented on HADOOP-7560:
-----------------------------------
Why don't we express hadoop-common depends on hadoop-alfredo and this will put hadoop-alfredo into share/hadoop/common/lib. This approach would be same as how hadoop-common includes Log4J jar files. There is no need of aggregator modules.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090409#comment-13090409 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
Note that for now the hadoop-mapreduce, hadoop-mapreduce-client and hadoop-yarn modules (which where the old aggregator modules) stay becuase there POMs are being overloaded with other uses (dependencies and properties settings).
A as part of a later consolidation of configuration they would go, but as they don't have code, they would go.
In addition, MR1 test JAR stuff is under hadoop-mapreduce still
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090457#comment-13090457 ]
Doug Cutting commented on HADOOP-7560:
--------------------------------------
Eric, a non-Maven savvy observer, using both dashes and slashes to represent the hierarchy is redundant. If the module names themselves fully represent the hierarchy, what value does the directory structure add?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Thomas Graves (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13091086#comment-13091086 ]
Thomas Graves commented on HADOOP-7560:
---------------------------------------
It doesn't looks like BUILDING.txt was updated with this, can you please update.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089724#comment-13089724 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
How about adding a top-level hadoop-extras and putting hadoop-alfredo there?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090432#comment-13090432 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
from IRC
{code}
[12:12pm] tucu: mmmhh, thought the other why around as my arguments in the JIRA to go flat were not disputed.
[12:14pm] acmurthy: sorry
[12:14pm] acmurthy: but having hadoop-yarn and hadoop-mr-client at top level seems very odd
[12:14pm] acmurthy: miscommunication
[12:14pm] tucu: still, logically still it is hierarchical, the names imply that. and IDEs (eclipse and idea) show modules flat, regardless of the hierachy. And Maven resolves the dependencies correctly as the hierachy is nothing for it
[12:19pm] acmurthy: tucu - is it hard to do it fullly heirarchical?
[12:19pm] acmurthy: having too many things at the top-level seems confusing for folks wihtout IDEs
[12:19pm] acmurthy: i liked your prev. proposal (aggregator modules) much better
[12:21pm] tucu: it is not hard, but i'd argue it is easier to work on a flat structure, ie to move file around, you'll always be at the same depth (*/src/main/java/). If you see big projects using maven the tent to use a flat set of modules.
[12:21pm] tucu: and in the past, maven was not fully functional with multi-level stuff
[12:22pm] tucu: now it is not the case, but some plugins may go bonkers
[12:22pm] tucu: and the fact that IDEs show thigns flat, is easy for a devel to do things from both environments
[12:22pm] tucu: without getting lost
[12:25pm] acmurthy: i'd really prefer heirarchical
[12:25pm] acmurthy: the project is truly heirarchical: common, hdfs, mr
[12:25pm] acmurthy: and so n
[12:25pm] acmurthy: on*
[12:28pm] tucu: arun, let me work on a patch for hierachical approach and then we can discuss the merits of both
{code}
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090347#comment-13090347 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Alejandro - thanks for the write-up.
I have a question - what were the approaches when we a) mavenized hadoop-common b) committed alfredo to trunk?
I'm more concerned about ensuring we stabilize trunk so that normal builds etc. can be resumed - it's unfair on folks like Giri to keep changing the rules, he has to continuously scramble to keep up... hence, I'd rather avoid a large change to directory/package structure. We can revisit this at a later point (for hadoop-0.24 etc.) if necessary.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089660#comment-13089660 ]
Luke Lu commented on HADOOP-7560:
---------------------------------
bq. I've introduce AG because, IMO, it would be confusing to have hadoop-common at two levels. I've change all aggregator modules to AG for consistency.
How about use -modules for aggregation, e.g. hadoop-common-modules as this doesn't require further explanation.
bq. Why don't we express hadoop-common depends on hadoop-alfredo...
The idea is to include the source into the hadoop common modules. The modules is the new (IMO better) way to include contrib sources.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090640#comment-13090640 ]
Hudson commented on HADOOP-7560:
--------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #857 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/857/])
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
* /hadoop/common/trunk/hadoop-common-project/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
* /hadoop/common/trunk/hadoop-annotations
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
* /hadoop/common/trunk/hadoop-alfredo
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
* /hadoop/common/trunk/hadoop-mapreduce-project
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/trunk/hadoop-common-project
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth
* /hadoop/common/trunk/hadoop-hdfs
* /hadoop/common/trunk/hadoop-mapreduce
* /hadoop/common/trunk/hadoop-common
* /hadoop/common/trunk/hadoop-common-project/hadoop-common
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089716#comment-13089716 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
@Arun, I'm afraid we'd have to bite another 'svn mv'
@Eric, IMO hadoop-alfredo, to be hadoop-auth, should be a separate module from hadoop-common (per reasons in my previous comment). If hadoop-common becomes an aggregator module or not it depends on how do we decide to structure Maven modules, hierarchical (like MR2) or flat (like hadoop-common, hadoop-hdfs, hadoop-alfredo, hadoop-project, ...).
Working on a comment for the last stuff, coming soon.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089808#comment-13089808 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
I'm trying to take as step back here to see the general issue of sub-modules rather than the pinpoint solution for alfredo or hoop.
Comment quite long, two main sections: Flat vs Hierarchical module structure & dual artifacts modules.
*Flat vs Hierarchical module structure*
For the purposes of the discussion, I'm leaving aside the concerns of another SVN move.
When organizing a multi-module Maven project there are 2 approaches, using a flat module structure or using a hierarchical module structure.
Our current layout is a mix of both (for the purposes of the generalization, lets assume hoop is in as well):
{code}
trunk/
hadopo-project/ (f)
hadoop-project-dist/ (f)
hadoop-assemblies/ (f)
hadoop-annotations/ (f)
hadoop-auth/ (f) (this is alfredo)
hadoop-common/ (f)
hadoop-hdfs/ (f)
hadoop-hdfs-http-client/ (f) (this is hoop)
hadoop-hdfs-http-server/ (f) (this is hoop)
hadoop-mapreduce/ (h) (*)
hadoop-mr-client/ (*)
hadoop-mapreduce-client-app/
hadoop-mapreduce-client-common/
hadoop-mapreduce-client-core/
hadoop-mapreduce-client-hs/
hadoop-mapreduce-client-jobclient/
hadoop-mapreduce-client-shuffle/
hadoop-yarn/ (*)
hadoop-yarn-api/
hadoop-yarn-common/
hadoop-yarn-server/
{code}
(f: flat modules, h: hierarchical modules, *: aggregator modules)
1. Note that the module names, flat or hierarchical, denote the hierarchy. Those names also denote the artifact name produced the module (typically a JAR). Because of this I think module names should stay as they are regardless.
2. The nice thing about a hierarchal structure is that we use directory levels to naturally represent the hierarchy. Still, because of #1, we shouldn't simplify the names. The bad thing, is that navigating through directory paths becomes a bit more cumbersome/long. Another issue is that we need additional aggregator modules (like hadoop-mapreduce, hadoop-mr-client and hadoop-yarn).
3. The nice thing about flat structure is that directory navigation is easy, the structure is in the names already. The bad thing is that the number of directories at trunk/ level is larger. The bad thing is that the sense of module scope is lost. IDEs like IntelliJ and Eclipse flatten the hierarchy. Thus, a flatten structure could be a bit more intuitive.
4. Maven doesn't care if flat or hierarchical, it resolves the order of building based on dependencies.
Personally, I'd prefer a flat structure, but I'm OK either way. What I would definitely like is to have full consistency.
*Dual Artifact Modules*
Currently hadoop-common and hadoop-hdfs modules produce 2 types of artifacts, a JAR and a TAR.
In my opinion, a Maven module should produce a single artifact, a JAR or a TAR. This is, for Common and HDFS we should have a hadoop-common-dist and a hadoop-hdfs-dist modules that only do assembly work. This decoupling will easily allow to bundle hoop with HDFS.
Otherwise, if Hoop depends on HDFS (it must for testing) there is a circular dependency (Hoop depends on HDFS for build/testing, HDFS depends on Hoop for packaging).
Because of this I'd advocate for single artifact modules.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7560) Make hadoop-common a POM module with
sub-modules (common & alfredo)
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated HADOOP-7560:
----------------------------------
Fix Version/s: 0.23.0
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090385#comment-13090385 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
As discussed over IRC:
* 1. we get the module structure correct in trunk
* 2. we get trunk to build/run testcases
* 3. we branch 0.23
* 4. on 0.23 we stabilize the build/packaging
* 5. on trunk we work on code
* 6. once #4 is done, we port build from 0.23 to trunk
* 7. we merge code changes from trunk to 0.23
It will take a few days to do this but we won't block build work 0.23 & devel work on trunk after we reach #3
We are shooting for immediate #1, #2 and #3.
Straw-man patch will be avail shortly.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089706#comment-13089706 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Alejandro - is there a way we can effect this without another 'svn mv' and renaming everything under hadoop/trunk?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090540#comment-13090540 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
Having v1 and v2 patches people can see how thing would look in a flat and hierarchical worlds. Please try both.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089701#comment-13089701 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
Arun, I'd like to keep Alfredo as a separate module. The main reason is that that its runtime dependencies are minimal:
{code}
[INFO] org.apache.hadoop:hadoop-alfredo:jar:0.23.0-SNAPSHOT
[INFO] +- org.slf4j:slf4j-api:jar:1.5.11:compile
[INFO] +- commons-codec:commons-codec:jar:1.4:compile
[INFO] +- log4j:log4j:jar:1.2.15:compile
[INFO] \- org.slf4j:slf4j-log4j12:jar:1.5.11:compile
{code}
(plus slf4j & log4j don't come when used in client mode, they are optional)
On the other hand, Hadoop Common dependencies are:
{code}
[INFO] org.apache.hadoop:hadoop-common:jar:0.23.0-SNAPSHOT
[INFO] +- com.google.guava:guava:jar:r09:compile
[INFO] +- commons-cli:commons-cli:jar:1.2:compile
[INFO] +- org.apache.commons:commons-math:jar:2.1:compile
[INFO] +- xmlenc:xmlenc:jar:0.52:compile
[INFO] +- commons-httpclient:commons-httpclient:jar:3.1:compile
[INFO] +- commons-codec:commons-codec:jar:1.4:compile
[INFO] +- commons-net:commons-net:jar:1.4.1:compile
[INFO] +- javax.servlet:servlet-api:jar:2.5:compile
[INFO] +- org.mortbay.jetty:jetty:jar:6.1.26:compile
[INFO] +- org.mortbay.jetty:jetty-util:jar:6.1.26:compile
[INFO] +- tomcat:jasper-compiler:jar:5.5.23:compile
[INFO] +- tomcat:jasper-runtime:jar:5.5.23:compile
[INFO] +- javax.servlet.jsp:jsp-api:jar:2.1:compile
[INFO] +- commons-el:commons-el:jar:1.0:compile
[INFO] +- commons-logging:commons-logging:jar:1.1.1:compile
[INFO] +- commons-logging:commons-logging-api:jar:1.1:compile
[INFO] +- log4j:log4j:jar:1.2.15:compile
[INFO] +- net.java.dev.jets3t:jets3t:jar:0.6.1:compile
[INFO] +- commons-lang:commons-lang:jar:2.5:compile
[INFO] +- commons-collections:commons-collections:jar:3.2.1:compile
[INFO] +- commons-configuration:commons-configuration:jar:1.6:compile
[INFO] | +- commons-digester:commons-digester:jar:1.8:compile
[INFO] | | \- commons-beanutils:commons-beanutils:jar:1.7.0:compile
[INFO] | \- commons-beanutils:commons-beanutils-core:jar:1.8.0:compile
[INFO] +- hsqldb:hsqldb:jar:1.8.0.7:compile
[INFO] +- org.slf4j:slf4j-api:jar:1.5.11:compile
[INFO] +- org.slf4j:slf4j-log4j12:jar:1.5.11:compile
[INFO] +- org.eclipse.jdt:core:jar:3.1.1:compile
[INFO] +- oro:oro:jar:2.0.8:compile
[INFO] +- org.codehaus.jackson:jackson-mapper-asl:jar:1.6.9:compile
[INFO] | \- org.codehaus.jackson:jackson-core-asl:jar:1.6.9:compile
[INFO] +- org.aspectj:aspectjrt:jar:1.6.5:compile
[INFO] +- org.apache.avro:avro:jar:1.5.2:compile
[INFO] | +- com.thoughtworks.paranamer:paranamer:jar:2.3:compile
[INFO] | \- org.xerial.snappy:snappy-java:jar:1.0.3-rc4:compile
[INFO] +- org.apache.avro:avro-ipc:jar:1.5.2:compile
[INFO] | +- org.jboss.netty:netty:jar:3.2.4.Final:compile
[INFO] | \- org.apache.velocity:velocity:jar:1.7:compile
[INFO] +- net.sf.kosmosfs:kfs:jar:0.3:compile
[INFO] +- com.google.protobuf:protobuf-java:jar:2.4.0a:compile
[INFO] \- org.apache.hadoop:hadoop-alfredo:jar:0.23.0-SNAPSHOT:compile
{code}
This is not really minimal. And having to exclude all of them when just wanting to use Alfredo client it not very developer friendly.
Regarding the module names and organization, I'll be posting a comment a bit later today once I finish writing it up.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088815#comment-13088815 ]
Doug Cutting commented on HADOOP-7560:
--------------------------------------
This will require updating the svn mailer configuration again, right? I can help with that again.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090574#comment-13090574 ]
Mahadev konar commented on HADOOP-7560:
---------------------------------------
v2 looks good to me! +1 for the v2 patch.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090658#comment-13090658 ]
Hudson commented on HADOOP-7560:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #765 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/765/])
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
* /hadoop/common/trunk/hadoop-common-project/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
* /hadoop/common/trunk/hadoop-annotations
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
* /hadoop/common/trunk/hadoop-alfredo
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
* /hadoop/common/trunk/hadoop-mapreduce-project
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/trunk/hadoop-common-project
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth
* /hadoop/common/trunk/hadoop-hdfs
* /hadoop/common/trunk/hadoop-mapreduce
* /hadoop/common/trunk/hadoop-common
* /hadoop/common/trunk/hadoop-common-project/hadoop-common
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088890#comment-13088890 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
So, is this the new way of doing contrib modules? Is alfredo meant to be one?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088965#comment-13088965 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
What was the idea when we committed alfredo? I want to ensure I'm not missing something in my understanding...
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090531#comment-13090531 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Thanks Alejandro - I'm looking fwd to it, your first proposal (https://issues.apache.org/jira/browse/HADOOP-7560?focusedCommentId=13089808&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13089808) seems reasonable to me.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089707#comment-13089707 ]
Eric Yang commented on HADOOP-7560:
-----------------------------------
I think making hadoop-common aggregator module would be a recursive mistake. It should be either make alfredo part of hadoop-common, or hadoop-common depends on hadoop-auth (which is the alfredo module). I am more in favor of making hadoop-http, hadoop-auth part of hadoop-common module.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090637#comment-13090637 ]
Hudson commented on HADOOP-7560:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk-Commit #782 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/782/])
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
* /hadoop/common/trunk/hadoop-common-project/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
* /hadoop/common/trunk/hadoop-annotations
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
* /hadoop/common/trunk/hadoop-alfredo
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
* /hadoop/common/trunk/hadoop-mapreduce-project
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/trunk/hadoop-common-project
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth
* /hadoop/common/trunk/hadoop-hdfs
* /hadoop/common/trunk/hadoop-mapreduce
* /hadoop/common/trunk/hadoop-common
* /hadoop/common/trunk/hadoop-common-project/hadoop-common
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Eric Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090578#comment-13090578 ]
Eric Yang commented on HADOOP-7560:
-----------------------------------
+1 on v2. Thanks
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090596#comment-13090596 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
I'm going to commit this unless anyone has reservations...
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089654#comment-13089654 ]
Doug Cutting commented on HADOOP-7560:
--------------------------------------
As noted in HDFS-2178, hadoop-alfredo should be named hadoop-auth, and hadoop-hoop should be named hadoop-hdfs-http.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090414#comment-13090414 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Oh, I was under the impression that we were going to the heirarchical model, moving everything (e.g. hadoop-mr-client*) to the top-level really doesn't make sense to me.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090536#comment-13090536 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
Just for the sake of the discussion, Yarn should not be under mapreduce but next to it, right?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090639#comment-13090639 ]
Hudson commented on HADOOP-7560:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #779 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/779/])
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
HADOOP-7560. Change src layout to be heirarchical. Contributed by Alejandro Abdelnur.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161335
Files :
* /hadoop/common/trunk/hadoop-hdfs-project/pom.xml
* /hadoop/common/trunk/hadoop-mapreduce-project/pom.xml
* /hadoop/common/trunk/hadoop-common-project/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations/pom.xml
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1161332
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mr-client
* /hadoop/common/trunk/hadoop-annotations
* /hadoop/common/trunk/hadoop-common-project/hadoop-annotations
* /hadoop/common/trunk/hadoop-alfredo
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-hdfs-project
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client
* /hadoop/common/trunk/hadoop-mapreduce-project
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs
* /hadoop/common/trunk/hadoop-common-project
* /hadoop/common/trunk/hadoop-common-project/hadoop-auth
* /hadoop/common/trunk/hadoop-hdfs
* /hadoop/common/trunk/hadoop-mapreduce
* /hadoop/common/trunk/hadoop-common
* /hadoop/common/trunk/hadoop-common-project/hadoop-common
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7560) Make hadoop-common a POM module with
sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7560:
---------------------------------------
Attachment: HADOOP-7560v1.patch
HADOOP-7560v1.sh
'v1' script & patch.
Flattens modules structure.
To apply, from trunk/ dir do:
{code}
$ sh HADOOP-7560v1.sh fs|svn
{code}
(use 'fs' if working from GIT, use 'svn' if working from SVN)
Then apply the patch.
I did minimal changes in POMs to compile.
Tested done:
{code}
$ rm -rf ~/.m2/repositories/org/apache/hadoop/*
$ mvn clean test -P-cbuild -DskipTests
{code}
This ensures all Java dependencies/code is in place.
{code}
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Project POM ......................... SUCCESS [1.664s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [1.885s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [0.002s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.309s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [1.582s]
[INFO] Apache Hadoop Common .............................. SUCCESS [17.927s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [10.260s]
[INFO] hadoop-mapreduce .................................. SUCCESS [0.064s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.004s]
[INFO] hadoop-yarn-api ................................... SUCCESS [4.713s]
[INFO] hadoop-yarn-common ................................ SUCCESS [5.545s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [2.693s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [2.444s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [1.230s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [1.768s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [1.907s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [0.424s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [4.227s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [0.838s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [0.518s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [1.063s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.003s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.003s]
[INFO] Apache Hadoop Main ................................ SUCCESS [0.005s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1:01.912s
{code}
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089668#comment-13089668 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
I'm not sure another huge set of 'svn mv's to incorporate alfredo is in order.
If hadoop-common depends on hadoop-alfredo, why don't we just move alfredo into common and be done with?
The common jar isn't that big really, so anyone who depends on hadoop-alfredo could just depend on hadoop-common?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090539#comment-13090539 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Alejandro, I think Giri's comment makes sense - for now having hadoop-yarn in hadoop-mapreduce helps folks looking for a single checkout... so, we could keep it there. Thoughts?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089736#comment-13089736 ]
Luke Lu commented on HADOOP-7560:
---------------------------------
bq. How about adding a top-level hadoop-extras and putting hadoop-alfredo there?
This is a good idea to minimize svn mv :). Though I'd like to avoid "extras", as it can imply something similar to apache-extras, which is not apache licensed. IMO, hadoop-optional-modules would be a better name.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090520#comment-13090520 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
Eric, your example (I guess c&p from a previous one of mine) lacks the aggregation level for common and hdfs.
I'm finalizing a hierarchical version patch so we can compare things side to side.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088055#comment-13088055 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
If this is done the same should be done for hadoop-hdfs, hadoop-mapreduce already does that as it had submodules at time of integration
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090562#comment-13090562 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
Awesome, v2 works flawlessly.
I'm +1 on v2.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy resolved HADOOP-7560.
-----------------------------------
Resolution: Fixed
I just committed this. Thanks Alejandro!
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
> Attachments: HADOOP-7560v1.patch, HADOOP-7560v1.sh, HADOOP-7560v2.patch, HADOOP-7560v2.sh
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Giridharan Kesavan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089723#comment-13089723 ]
Giridharan Kesavan commented on HADOOP-7560:
--------------------------------------------
svn move again?
since the beginning of this month I couldn't get a stable build out of trunk; Every week we have some new changes to make to the build env and something is broken. It's getting very painful.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13089669#comment-13089669 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
I'm not particularly disposed either way, I'd like this resolved prior to the 0.23 branch if possible.
This and MAPREDUCE-2838 are the only things blocking it IMO.
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088983#comment-13088983 ]
Alejandro Abdelnur commented on HADOOP-7560:
--------------------------------------------
@Arun, I assume you are referring to the moving common/hdfs to submodules. If so, we just started the discussion here. I don't see any proposal/agreement yet.
The current set of Maven modules is a follows (the '*' means that are aggregator Maven modules, type POM, no code in them):
{code}
trunk/
hadopo-project/
hadoop-project-dist/
hadoop-assemblies/
hadoop-annotations/
hadoop-alfredo/
hadoop-common/
hadoop-hdfs/
hadoop-mapreduce/*
hadoop-mr-client/*
hadoop-mapreduce-client-app/
hadoop-mapreduce-client-common/
hadoop-mapreduce-client-core/
hadoop-mapreduce-client-hs/
hadoop-mapreduce-client-jobclient/
hadoop-mapreduce-client-shuffle/
hadoop-yarn/*
hadoop-yarn-api/
hadoop-yarn-common/
hadoop-yarn-server/
{code}
We could do something like (adding hoop modules to HDFS to have more comprehensive examples). I'm using *AG* as a module label to indicate it is an aggregator and no-code/artifact is produced from that module:
{code}
trunk/
hadopo-project/
hadoop-project-dist/
hadoop-assemblies/
hadoop-annotations/
hadoop-common-AG/*
hadoop-alfredo/
hadoop-common/
hadoop-hdfs-AG/*
hadoop-hdfs/
hadoop-hoop-client/
hadoop-hoop-server/
hadoop-mapreduce-AG/*
hadoop-mr-client-AG/
hadoop-mapreduce-client-app/
hadoop-mapreduce-client-common/
hadoop-mapreduce-client-core/
hadoop-mapreduce-client-hs/
hadoop-mapreduce-client-jobclient/
hadoop-mapreduce-client-shuffle/
hadoop-yarn-AG/
hadoop-yarn-api/
hadoop-yarn-common/
hadoop-yarn-server/
{code}
I've introduce *AG* because, IMO, it would be confusing to have hadoop-common at two levels. I've change all aggregator modules to *AG* for consistency.
Thoughts?
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7560) Make hadoop-common a POM module
with sub-modules (common & alfredo)
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13090387#comment-13090387 ]
Arun C Murthy commented on HADOOP-7560:
---------------------------------------
+1, thanks for the effort on this Alejandro!
> Make hadoop-common a POM module with sub-modules (common & alfredo)
> -------------------------------------------------------------------
>
> Key: HADOOP-7560
> URL: https://issues.apache.org/jira/browse/HADOOP-7560
> Project: Hadoop Common
> Issue Type: Sub-task
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.0
>
>
> Currently hadoop-common is a JAR module, thus it cannot aggregate sub-modules.
> Changing it to POM module it makes it an aggregator module, all the code under hadoop-common must be moved to a sub-module.
> I.e.:
> mkdir hadoop-common-project
> mv hadoop-common hadoop-common-project
> mv hadoop-alfredo hadoop-common-project
> hadoop-common-project/pom.xml is a POM module that aggregates common & alfredo
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira