You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Allen Wittenauer (JIRA)" <ji...@apache.org> on 2015/06/01 22:00:18 UTC
[jira] [Updated] (HADOOP-11929) add test-patch plugin points for
customizing build layout
[ https://issues.apache.org/jira/browse/HADOOP-11929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Allen Wittenauer updated HADOOP-11929:
--------------------------------------
Attachment: HADOOP-11929.04.patch
-04:
* big rebase
* includes the current version of HADOOP-11984, just to give the patch some weight
hadoop.sh changes:
* hadoop personality forces tools to be last
* checkstyle should be working now
* mvn install run after javac run
* removed more stuff out of the linux build to match jenkins capabilities. (we'll build this back up later)
* removed bzip out of the mac build due to bzip detection in our cmake files being broken
* javadoc pre-reqs now specifically called out
* mvn install adds -DskipTests
* parallel tests support
shellcheck.sh:
* fixes for moved modules
* checkstyle
* significant update to use new per-module framework bits
test-patch.sh:
* fixes for moved modules. CHANGED_MODULES now recalculated after patch application
* New CHANGED_UNFILTERED_MODULES. CHANGED_MODULES has always had non-code modules stripped out of it. CHANGED_UNFILTERED has every module in it.
* new clock_display functions pulls code out of add_jira_table so other routines can print elapsed time themselves
* jira_footer now has protoc version
* maven log files now print the maven command used at the top of the file
* mvninstall is now separate test type. (almost, still some todo here)
* reordered test lists in determine_tests
* lots of bug fixes and API changes to the new mvn_module* routines, primarily so mvn_module_messages can be use for both success and fail, greatly reducing code elsewhere
* much of the time handling has been cleaned up, with reported times now mostly being mvn times and not test-patch.sh overhead time. global time reports maven + shell overhead.
* even more file name consistency
* almost all tests now report for *every* module *every* time rather than giving a blanket +1 for entire chunks
> add test-patch plugin points for customizing build layout
> ---------------------------------------------------------
>
> Key: HADOOP-11929
> URL: https://issues.apache.org/jira/browse/HADOOP-11929
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Sean Busbey
> Assignee: Allen Wittenauer
> Priority: Minor
> Attachments: HADOOP-11929.00.patch, HADOOP-11929.01.patch, HADOOP-11929.02.patch, HADOOP-11929.03.patch, HADOOP-11929.04.patch, hadoop.sh
>
>
> Sean Busbey and I had a chat about this at the Bug Bash. Here's the proposal:
> * Introduce the concept of a 'personality module'.
> * There can be only one personality.
> * Personalities provide a single function that takes as input the name of the test current being processed
> * This function uses two other built-in functions to define two queues: maven module name and profiles to use against those maven module names
> * If something needs to be compiled prior to this test (but not actually tested), the personality will be responsible for doing that compilation
> In hadoop, the classic example is hadoop-hdfs needs common compiled with the native bits. So prior to the javac tests, the personality would check CHANGED_MODULES, see hadoop-hdfs, and compile common w/ -Pnative prior to letting test-patch.sh do the work in hadoop-hdfs. Another example is our lack of test coverage of various native bits. Since these require profiles to be defined prior to compilation, the personality could see that something touches native code, set the appropriate profile, and let test-patch.sh be on its way.
> One way to think of it is some higher order logic on top of the automated 'figure out what modules and what tests to run' functions.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)