You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Luke Lu (JIRA)" <ji...@apache.org> on 2011/04/12 21:06:05 UTC
[jira] [Created] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
MR-279: Install sanitized poms for downstream sanity
----------------------------------------------------
Key: MAPREDUCE-2432
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Luke Lu
Assignee: Luke Lu
Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated MAPREDUCE-2432:
-------------------------------
Attachment: mr-2432-poms-v1.patch
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Luke Lu
> Assignee: Luke Lu
> Attachments: mr-2432-poms-v1.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar updated MAPREDUCE-2432:
-------------------------------------
Component/s: mrv2
Fix Version/s: 0.23.0
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: Luke Lu
> Assignee: Luke Lu
> Fix For: 0.23.0
>
> Attachments: mr-2432-poms-v1.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated MAPREDUCE-2432:
-------------------------------
Attachment: mr-2432-poms-v2.patch
Mahadev, thanks for catching this. I introduced an incorrect default hdfs version (should be 0.22.0-SNAPSHOT instead of 0.22-SNAPSHOT) in hadoop-mapreduce-client-core's pom.xml. I didn't catch that because I was using a setting.xml with correct version of hdfs version defined.
The v2 patch removed the superfluous default definition of the hdfs version property, which now should be inherited from parent poms.
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: Luke Lu
> Assignee: Luke Lu
> Fix For: 0.23.0
>
> Attachments: mr-2432-poms-v1.patch, mr-2432-poms-v2.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019258#comment-13019258 ]
Luke Lu commented on MAPREDUCE-2432:
------------------------------------
Thanks for taking a look Mahadev.
It's clear that the build is missing the required hdfs artifact. I recall that you were playing with hdfs build (that was broken) earlier, where you probably did a ant veryclean...
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: Luke Lu
> Assignee: Luke Lu
> Fix For: 0.23.0
>
> Attachments: mr-2432-poms-v1.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019181#comment-13019181 ]
Mahadev konar commented on MAPREDUCE-2432:
------------------------------------------
luke,
I am seeing the following error with this patch:
{noformat}
Missing:
----------
1) org.apache.hadoop:hadoop-hdfs:jar:0.22-SNAPSHOT
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=org.apache.hadoop -DartifactId=hadoop-hdfs -Dversion=0.22-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=org.apache.hadoop -DartifactId=hadoop-hdfs -Dversion=0.22-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) org.apache.hadoop:hadoop-mapreduce-client-core:jar:1.0-SNAPSHOT
2) org.apache.hadoop:hadoop-hdfs:jar:0.22-SNAPSHOT
----------
1 required artifact is missing.
for artifact:
org.apache.hadoop:hadoop-mapreduce-client-core:jar:1.0-SNAPSHOT
from the specified remote repositories:
central (http://repo1.maven.org/maven2),
repository.jboss.org (http://repository.jboss.org/nexus/content/groups/public/)
{noformat}
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Luke Lu
> Assignee: Luke Lu
> Attachments: mr-2432-poms-v1.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019182#comment-13019182 ]
Mahadev konar commented on MAPREDUCE-2432:
------------------------------------------
sorry I should have mentioned I was running:
{noformat}
mvn clean install assembly:assembly -Dmaven.test.skip.exec=true -P-cbuild
{noformat}
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Reporter: Luke Lu
> Assignee: Luke Lu
> Attachments: mr-2432-poms-v1.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated MAPREDUCE-2432:
-------------------------------
Attachment: mr-2432-poms-v3.patch
v3 patch is rebased against the current HEAD and fixed a merge issue with Vinod's container monitoring commit.
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: Luke Lu
> Assignee: Luke Lu
> Fix For: 0.23.0
>
> Attachments: mr-2432-poms-v1.patch, mr-2432-poms-v2.patch, mr-2432-poms-v3.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2432) MR-279: Install sanitized poms
for downstream sanity
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar resolved MAPREDUCE-2432.
--------------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
I just committed this. thanks luke!
> MR-279: Install sanitized poms for downstream sanity
> ----------------------------------------------------
>
> Key: MAPREDUCE-2432
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2432
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
> Reporter: Luke Lu
> Assignee: Luke Lu
> Fix For: 0.23.0
>
> Attachments: mr-2432-poms-v1.patch, mr-2432-poms-v2.patch, mr-2432-poms-v3.patch
>
>
> Due to [MNG-4223|http://jira.codehaus.org/browse/MNG-4223], the installed POMs of MR-279 is downstream hostile. E.g., it's impossible to use versions of hadoop-mapreduce-client-core.version in ivy other than 1.0-SNAPSHOT without changing the multiple POMs, rendering the version properties (hadoop-mapreduce.version and yarn.version) practically useless.
> This patch will install POMs with version (only) properties expanded. This patch also use inheritance and dependencyManagement to make POMs DRYer. It could use further cleanup to reduce "unnecessary" dependencies in some modules, but it's a working start.
> To see the patch work, apply the patch and do a mvn clean install -P-cbuild -DskipTests to make sure sane POMs are installed and then working on individual test issues.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira