[Db-derby Wiki] Update of "IncrementalDevelopment" by DanDebrunner

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Db-derby Wiki" for change notification.

The following page has been changed by DanDebrunner:
http://wiki.apache.org/db-derby/IncrementalDevelopment

New page:
= Incremental Development =
[[TableOfContents]]
Wikipedia link to [http://en.wikipedia.org/wiki/Iterative_development Iterative and incremental development].

== Applying to Derby patches ==
=== A Good Thing! ===
Breaking a large change or implementation of a feature into a sequence of small independent and/or related patches
is a good thing because such patches:

 * are easier for the contributor to spot bugs in
 * are easier for the reviewers to understand and spot issues with
 * may lead to reduced amount testing before submission
 * will lead to faster commits
 * easier to identify as a cause of a regression after the fact
 * allows early testing of exposed functionality

=== Ahead of Time ===
For a feature it's normal to list a set of steps and possibly sub-steps needed to complete it, each of these steps or sub-steps
can correspond to a patch.

As an example, suppose one was planning to implement [:SqlPsmSupport:stored procedures written in SQL for Derby],
the steps might be:
 1. Implement simple functions
   a. Add a RETURN statement to the parser (not used)
   a. Add support for RoutineAliasInfo storing additional information about the routine (e.g. language SQL, etc. etc.)
   a. Modify the parser to allow SQL function definition using the RETURN statement, not worrying about parameters
   a. Add code to generate the Java class corresponding to the function
   a. Add code to store the generated class and link it to the routine
   a. Add a mechanism to load the generated class for the function
   a. Add more tests to ensure simple SQL functions work.
 1. Implement functions with parameters
   a. Add named parameter support in the compiler context obtaining the parameter names from the routine defintion
   a. Add code to resolve paramters to the current set in scope (defined by the CREATE FUNCTION statement)
 1. Add support for multi-statement procedures
   a. add parser changes to allow single statement SQL procedures
   a. enhance the code generation to support statements that don't return result sets
   a. add parser changes to allow multiple statements 
   a. enhance the code generation to support multiple statements
   a. add savepoints to generated code
 1. Add support for returning result sets in procedures from SQL statements
   a. Add parser changes
   a. etc. etc.
 1. Add control flow to the SQL supported
   a. Add parser changes
   a. etc. etc.

As development proceeds, then patches are submitted corresponding to the steps and sub-steps as needed.
After each patch, the code is left is a useful working form
but executing the actions may result in a not implemented exception, as an example after sub-step 1c) Derby would accept
a valid CREATE FUNCTION statement with LANGUAGE SQL but would throw a not implemented exception at bind time.
Most likely each page would contain tests that test out any added functionality, but that may not be a requirement for every patch.
This approach leads to a much easier set of reviews than an entire patch that implements all of the above.

Another way to split up a patch to submit tests first in one (or more) patches and then the code. This works very
well when one is replacing an existing implementation with a different (better) one. Submitting the tests against the old
code helps ensure that your changes do not change the functional behaviour. Such changes could be masked in a single patch.

=== After the Fact ===
Sometimes one can be working on a feature and realise too late that the set of changes is large, complicated and will be hard for reviewers
to understand. The trick is to spot this as soon as possible, if you are describig a patch and you have to describe more than two
or three separate items then consider splitting the patch after the fact.

First one must look at the changes and determine if a set of independent patches or steps exist, e.g. if adding some new SQL feature can
the parsing be separate, can the changes to the system table be independent etc? Once you have determined some logical way to break up
the large patch then create a separate clean code line in parallel to your modified working code. Create the large patch from your modified
code line and apply the portion of it corresponding to the first step to the clean code line. Sometimes this may be just applying a sub-set of
the changes, sometimes it may require hand editing. The Eclipse IDE allows fine grained control when applying patches. Build & test the
smaller patch in the clean code line and then submit as normal. Repeat until your large patch has been successfully applied through a number
of smaller patches.