svn commit: r474617 - in /maven/sandbox/maven-plugin-testing-tools/src/site: ./ apt/ apt/index.apt

[jd...@apache.org]

Author: jdcasey
Date: Mon Nov 13 17:01:07 2006
New Revision: 474617

URL: http://svn.apache.org/viewvc?view=rev&rev=474617
Log:
Adding a start to the documentation for this library.

Added:
    maven/sandbox/maven-plugin-testing-tools/src/site/
    maven/sandbox/maven-plugin-testing-tools/src/site/apt/
    maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt   (with props)

Added: maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt
URL: http://svn.apache.org/viewvc/maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt?view=auto&rev=474617
==============================================================================
--- maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt (added)
+++ maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt Mon Nov 13 17:01:07 2006
@@ -0,0 +1,85 @@
+  ---
+  Maven Plugin-Testing Tools
+  ---
+  John Casey
+  ---
+  13 November 2006
+  ---
+  
+Maven Plugin-Testing Tools
+
+  This API is meant to provide a series of convenience tools for developers wishing to test Maven 
+  plugins. The primary focus of the library is on integration testing, since it is not well-supported
+  with current tools and approaches.
+
+* Special Problems with Plugin Integration Testing
+
+  Maven plugins represent a somewhat special need in terms of integration testing. Like many 
+  component-oriented systems, Maven provides much of the runtime context essential for testing a 
+  plugin in its "natural environment". This means that, in order to properly test the integration of
+  any particular plugin into the larger system, the developer should really execute Maven builds
+  that test particular features of the plugin in question.
+  
+  However, running test builds from within the plugin's own build presents some special problems.
+  Since we're using Maven to build the Maven plugin, we have to deal with the fact that a previous
+  version of the plugin could be active within the current build (not a great practice, but sometimes
+  unavoidable). This means that Maven must either be smart enough to unload the existing plugin-version
+  from the runtime context and load the newer version - and correspondingly, unload the test version
+  and reload the older version for other, later builds in the same reactor. As an alternative, the
+  integration-testing of the plugin could fork separate Maven builds, and track the output in order
+  to verify results and determine whether the build succeeds. The latter approach, while much slower
+  and somewhat dependent on local-system configuration, is currently the most robust way to avoid 
+  existing versions of the plugin under test.
+  
+  In addition to plugin-version inconsistencies, integration-testing of Maven plugins must cope with
+  the need to cleanup after themselves. Remember, Maven only loads plugins from its own local repository
+  (possibly after resolving them from a remote repository). Because of this, the only practical way 
+  to use the current plugin within an integration-test build is to install it into some sort of local 
+  repository, then direct Maven to use that local repository for the build. To avoid side effects in
+  other builds (remember, we're building Maven plugins here...and they're used to build other projects),
+  it's critical to avoid using the main local repository used for normal Maven builds. Therefore,
+  plugin integration tests require a custom local repository, to which the integration-testing builds
+  can be directed. Further complicating this testing scenario is the fact that the plugin's parent
+  POM or one of its ancestor POMs could be a snapshot which has been installed in the main local
+  repository, but which is not reachable using the plugin-POM's <<<\<relativePath/\>>>> parent-element.+
+  
+  <+ All of this seems to indicate that the best approach for ensuring the presence of all required
+  POMs and plugins is to use the normal local repository to construct a new local repository for
+  testing purposes. Work on this is ongoing.>
+  
+  Finally, in order to support integration-testing of plugins, it's important to give the developer
+  an API by which to bootstrap the testing environment (and ideally, detect when it's already been 
+  bootstrapped), and to also provide a robust API for executing builds and verifying results.
+  
+* A Note on Test-Suite Maintenance
+
+  Since failure to grasp the use cases of any system is a virtual guarantee that it will not meet
+  the needs of the user, it's important to understand how integration tests are really used.
+  
+  First and foremost, integration tests are used as a one-way valve, to ensure that no feature of
+  the application - in our case, a Maven plugin - regresses as the result of unrelated development.
+  A good set of tests - of which the integration tests are, well, integral - can provide the developer
+  with freedom to implement new features with the confidence that he's able to effectively build on
+  the foundations of previous versions.
+  
+  However, it's important to understand that regression-prevention is not the sole purpose of an
+  integration test. A secondary but critical use case deals with how integration tests are added to
+  the plugin. One has only to consider how bugs are reported on a flawed system to understand. Since
+  users usually report bugs, bug reports originate outside of the system. In the case of a Maven
+  plugin, this means users are most likely able to supply examples of the failing behavior in the
+  form of failing builds. In order to make immediate and direct use of what the user gives us, it's
+  critical that integration tests are designed to be augmented using build examples from the user,
+  with as little rewriting as possible.
+  
+  To that end, the approach for integration testing in this library will be the execution of sample
+  builds that use the plugin in question, then verify the results, either in the logs, the resulting
+  exit code, or the project files it produces.
+  
+* Integration-Testing Requirements
+
+  To summarize, this library strives to meet the following requirements for integration testing:
+  
+  * Trigger Maven builds using test projects, without incurring any sort of conflict with plugins
+    already in use within the current (this plugin's) build.
+    
+  * 

Propchange: maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt
------------------------------------------------------------------------------
    svn:eol-style = native

Propchange: maven/sandbox/maven-plugin-testing-tools/src/site/apt/index.apt
------------------------------------------------------------------------------
    svn:keywords = "Author Date Id Revision"