You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Colin Patrick McCabe (JIRA)" <ji...@apache.org> on 2012/10/05 20:38:05 UTC
[jira] [Created] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Colin Patrick McCabe created HADOOP-8887:
--------------------------------------------
Summary: Use a Maven plugin to build the native code using CMake
Key: HADOOP-8887
URL: https://issues.apache.org/jira/browse/HADOOP-8887
Project: Hadoop Common
Issue Type: Improvement
Components: build
Affects Versions: 2.0.3-alpha
Reporter: Colin Patrick McCabe
Assignee: Colin Patrick McCabe
Priority: Minor
Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
* compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
* there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
* the antrun code is very platform specific
* there is no way to run a specific native unit test
* it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.002.patch
Get rid of some warnings (none of them were actual bugs.)
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13471990#comment-13471990 ]
Hadoop QA commented on HADOOP-8887:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548319/HADOOP-8887.003.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified test files.
{color:red}-1 javac{color}. The applied patch generated 2061 javac compiler warnings (more than the trunk's current 2053 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//testReport/
Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//console
This message is automatically generated.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472864#comment-13472864 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
bq. [defaults]
I'll add defaults for those parameters.
bq. CleanMojo for windows, I don't see how, but if that is the case we should introduce it then if needed, not now just in case.
My big concern is just that people will put their output directories somewhere other than in a directory that gets auto-cleaned. I think if we skip the clean step, we have to somehow enforce the constraint that the output directory gets cleared by mvn clean; otherwise we're just creating hidden dangers for the future.
bq. generate/make, I'd merge them in a single Mojo as there are not usecases for using them apart, plus generate does not generate sources but make files.
OK. I'm fine with merging the cmake and make steps. It probably will make the pom.xml files shorter, which we can always use.
bq. I have not run the patch yet, but I assume that ALL generated/created files are done under output and the source tree is not polluted with files created at build time.
Yep.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.005.patch
* move org.apache.maven.plugin.cmake.ng.* to org.apache.hadoop.cmake.maven.ng.*
* removed CleanMojo
* merged GenerateMojo and CompileMojo
* clearer test output.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472875#comment-13472875 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
bq. does compiled test stuff need to be segregated in target/test-native?
No.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472866#comment-13472866 ]
Alejandro Abdelnur commented on HADOOP-8887:
--------------------------------------------
got it, ok on clean. on the defaults, make sure you have tests for them.
You've missed one question from prev comment: where test code lives? does compiled test stuff need to be segregated in target/test-native?
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-8887:
---------------------------------------
Resolution: Fixed
Fix Version/s: 2.0.3-alpha
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Colin. Committed to trunk and branch-2.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470538#comment-13470538 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
A word of explanation about the {{test-container-executor.c}} change: it seems that when launched directly from Maven rather than from a shell, {{SIGQUIT}} starts off blocked, causing the test to fail. The change manually unblocks this signal-- always a good idea to do before you start using a signal.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472272#comment-13472272 ]
Tom White commented on HADOOP-8887:
-----------------------------------
This plugin looks general purpose and it's in the org.apache.maven package, so perhaps it should go in the Maven Plugins project?
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479642#comment-13479642 ]
Hudson commented on HADOOP-8887:
--------------------------------
Integrated in Hadoop-trunk-Commit #2893 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2893/])
reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Status: Open (was: Patch Available)
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch, HADOOP-8887.008.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur resolved HADOOP-8887.
----------------------------------------
Resolution: Fixed
Committed to branch-2 after tweaking POM versions, did a full build and verified all generated JARs are the correct version. Apologies for the noise.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472758#comment-13472758 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
It would be nice to spin it off as a separate project, but that seems like it would just make it more difficult to iterate on the design in the short term, which is what we will need to get it truly polished. If it's a separate project, we'll have backwards compatibility constraints, and also need two changes (one to add a feature to it, another to update Hadoop to use the new version and features) for every change.
Does it make sense to put it in Hadoop for now, and then spin it off into another project later?
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475480#comment-13475480 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
By the way, one thing that's cool about this patch is that you can now run
{code}
mvn test -Pnative -Dtest=test_native_mini_dfs
{code}
and it will run the native test, just like you would expect.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Status: Patch Available (was: Open)
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Status: Patch Available (was: Reopened)
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch, HADOOP-8887.008.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479942#comment-13479942 ]
Hudson commented on HADOOP-8887:
--------------------------------
Integrated in Hadoop-Yarn-trunk #8 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/8/])
reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470646#comment-13470646 ]
Hadoop QA commented on HADOOP-8887:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548020/HADOOP-8887.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:red}-1 javac{color}. The applied patch generated 2057 javac compiler warnings (more than the trunk's current 2053 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//artifact/trunk/patchprocess/newPatchFindbugsWarningscmake-maven-ng-plugin.html
Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//console
This message is automatically generated.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.004.patch
Need to explicitly set Maven findbugs plugin version to avoid javac warnings.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.003.patch
Clean up warnings.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.008.patch
Here's a new version which fixes the problem with "mvn site".
* It also adds the ability to recognize native test failures to test-patch.sh.
* Also fix a bug with the test classpath that some native unit tests were getting.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch, HADOOP-8887.008.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13473251#comment-13473251 ]
Tom White commented on HADOOP-8887:
-----------------------------------
bq. Does it make sense to put it in Hadoop for now, and then spin it off into another project later?
If we do that we should at least put it in a org.apache.hadoop package since the Hadoop PMC doesn't control the org.apache.maven namespace.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479198#comment-13479198 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
test failures are unrelated
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475481#comment-13475481 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
oh, also, I added the default parameters as tucu suggested.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479571#comment-13479571 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-8887:
------------------------------------------------
How about the javadoc warnings?
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.006.patch
This new version should fix the javac warnings.
The javadoc warnings will be suppressed when the patch is actually checked in (I bumped {{OK_JAVADOC_WARNINGS}} to 14). The source of the Javadoc warnings is that we're getting warnings about a few annotations which are necessary for Maven plugins (It warns about "@goal" and "@phase", but Maven needs these).
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur reopened HADOOP-8887:
----------------------------------------
I've just reverted both branch-2 and trunk, while the dist tarball is being built correctly, as Andrew pointed out site is not working, we have to fix that.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472857#comment-13472857 ]
Alejandro Abdelnur commented on HADOOP-8887:
--------------------------------------------
* Unless I'm missing something, I don't see patch anything like the following to define defaults:
{code}
* @parameter expression="${source}" default-value="${basedir}/src/main/native"
* @parameter expression="${output}" default-value="${project.build.directory}/native"
{code}
* CleanMojo for windows, I don't see how, but if that is the case we should introduce it then if needed, not now just in case.
* generate/make, I'd merge them in a single Mojo as there are not usecases for using them apart, plus generate does not generate sources but make files.
* where test code lives? does compiled test stuff need to be segregated in target/test-native?
As an additional reason for removing clean & merging generate/make is that the amount of plugins config in the POMs are reduced from 3 to 1.
I have not run the patch yet, but I assume that ALL generated/created files are done under output and the source tree is not polluted with files created at build time.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472096#comment-13472096 ]
Hadoop QA commented on HADOOP-8887:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548345/HADOOP-8887.004.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1584//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1584//console
This message is automatically generated.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478550#comment-13478550 ]
Hadoop QA commented on HADOOP-8887:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12549601/HADOOP-8887.006.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
org.apache.hadoop.hdfs.server.balancer.TestBalancer
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1642//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1642//console
This message is automatically generated.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.011.patch
after some consultation with the Maven experts, it seems that including a Maven plugin in the same repository you use it in is not supported.
I think the way forward here is to maintain cmake-maven-ng as its own Apache project.
In the meantime, I have put cmake-maven-ng on Apache Extras under http://code.google.com/a/apache-extras.org/p/cmake-maven-ng/. Maven can make use of this repository in the meantime.
The following patch implements this.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch, HADOOP-8887.008.patch, HADOOP-8887.011.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur reopened HADOOP-8887:
----------------------------------------
Thanks Suresh, my bad, missed to tweak the POM versions when merging, I'll do it again (this time properly). Nicholas, as Colin indicated, javadoc warnings are taken care by the patch, just it does not show until the patch is committed (one of these days I'll try to tweak test-patch to do what we do in Oozie, we run javadoc twice, once without the patch and once with the patch, and we compare the warning numbers, less or equal good).
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478267#comment-13478267 ]
Alejandro Abdelnur commented on HADOOP-8887:
--------------------------------------------
+1 pending jenkins.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470853#comment-13470853 ]
Hadoop QA commented on HADOOP-8887:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548065/HADOOP-8887.002.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new or modified test files.
{color:red}-1 javac{color}. The applied patch generated 2057 javac compiler warnings (more than the trunk's current 2053 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 10 new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//artifact/trunk/patchprocess/newPatchFindbugsWarningscmake-maven-ng-plugin.html
Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//console
This message is automatically generated.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480038#comment-13480038 ]
Hudson commented on HADOOP-8887:
--------------------------------
Integrated in Hadoop-Mapreduce-trunk #1230 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1230/])
reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480005#comment-13480005 ]
Hudson commented on HADOOP-8887:
--------------------------------
Integrated in Hadoop-Hdfs-trunk #1200 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1200/])
reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)
Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472786#comment-13472786 ]
Alejandro Abdelnur commented on HADOOP-8887:
--------------------------------------------
* plugin root package should be org.apache.hadoop if in Hadoop.
* source directory should be settable via an 'source' property which defaults to ${basedir}/src/main/native
* build directory should be settable via an 'outputDirectory' property which defaults to ${project.build.directory}/native if not set.
* what is the diff between the output and target params in the CompileMojo? Do we need both? see prev comment on naming.
* CleanMojo, why do we need this one? 'mvn clean' already takes care of it.
* what is the diff between cmake-generate and cmake-compile? Do we need 2 different Mojos? Do we gain something from it?
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479602#comment-13479602 ]
Suresh Srinivas commented on HADOOP-8887:
-----------------------------------------
@Alejandro no worries.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13472842#comment-13472842 ]
Colin Patrick McCabe commented on HADOOP-8887:
----------------------------------------------
bq. plugin root package should be org.apache.hadoop if in Hadoop.
OK.
bq. source directory should be settable via an 'source' property which defaults to ${basedir}/src/main/native
It is, via GenerateMojo#source
bq. build directory should be settable via an 'outputDirectory' property which defaults to ${project.build.directory}/native if not set.
It is, via GenerateMojo#output and CompileMojo#output.
bq. what is the diff between the output and target params in the CompileMojo? Do we need both? see prev comment on naming.
"Build target" would be something like Debug, Production, etc. "output" is a
directory. I will add a comment explaining this.
bq. CleanMojo, why do we need this one? 'mvn clean' already takes care of it.
'mvn clean' will delete the 'target' directory, but we don't enforce the concept that the CMake-ng output directory is inside that directory. We could enforce this, and then make get rid of the clean target? However, we also might need this for when we're supporting Windows, maybe?
bq. what is the diff between cmake-generate and cmake-compile? Do we need 2 different Mojos? Do we gain something from it?
cmake-generate runs the cmake application to create the Makefiles. cmake-compile actually runs Make on these generated files. It seems natural to separate these two steps. However, I don't have a specific reason why it has to be implemented this way -- we could combine both steps into one. I was trying to go with the spirit of Maven, which separates code generation and compilation.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Suresh Srinivas (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479570#comment-13479570 ]
Suresh Srinivas commented on HADOOP-8887:
-----------------------------------------
I reverted the branch-2 patch. Some build issues and version number issues have been identified with that merge. Perhaps you need a separate patch for branch-2.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13478341#comment-13478341 ]
Hadoop QA commented on HADOOP-8887:
-----------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12548991/HADOOP-8887.005.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new or modified test files.
{color:red}-1 javac{color}. The applied patch generated 2031 javac compiler warnings (more than the trunk's current 2030 warnings).
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 4 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//testReport/
Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//artifact/trunk/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//console
This message is automatically generated.
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13479446#comment-13479446 ]
Hudson commented on HADOOP-8887:
--------------------------------
Integrated in Hadoop-trunk-Commit #2889 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/2889/])
HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)
Result = FAILURE
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
Files :
* /hadoop/common/trunk/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
* /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
* /hadoop/common/trunk/dev-support/pom.xml
* /hadoop/common/trunk/dev-support/test-patch.sh
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
* /hadoop/common/trunk/hadoop-project/pom.xml
* /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
* /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
* /hadoop/common/trunk/pom.xml
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Fix For: 2.0.3-alpha
>
> Attachments: HADOOP-8887.001.patch, HADOOP-8887.002.patch, HADOOP-8887.003.patch, HADOOP-8887.004.patch, HADOOP-8887.005.patch, HADOOP-8887.006.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-8887) Use a Maven plugin to build the
native code using CMake
Posted by "Colin Patrick McCabe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-8887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HADOOP-8887:
-----------------------------------------
Attachment: HADOOP-8887.001.patch
> Use a Maven plugin to build the native code using CMake
> -------------------------------------------------------
>
> Key: HADOOP-8887
> URL: https://issues.apache.org/jira/browse/HADOOP-8887
> Project: Hadoop Common
> Issue Type: Improvement
> Components: build
> Affects Versions: 2.0.3-alpha
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Priority: Minor
> Attachments: HADOOP-8887.001.patch
>
>
> Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:
> * compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
> * there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
> * the antrun code is very platform specific
> * there is no way to run a specific native unit test
> * it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.
> Using a native Maven plugin would overcome these limitations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira