You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by lu...@apache.org on 2009/03/15 22:34:48 UTC
svn commit: r754764 - in /commons/proper/math/trunk/src/site: site.xml
xdoc/changes.xml xdoc/userguide/estimation.xml xdoc/userguide/index.xml
xdoc/userguide/ode.xml xdoc/userguide/optimization.xml
Author: luc
Date: Sun Mar 15 21:34:47 2009
New Revision: 754764
URL: http://svn.apache.org/viewvc?rev=754764&view=rev
Log:
updated documentation after the redesign of the optimization package
Removed:
commons/proper/math/trunk/src/site/xdoc/userguide/estimation.xml
Modified:
commons/proper/math/trunk/src/site/site.xml
commons/proper/math/trunk/src/site/xdoc/changes.xml
commons/proper/math/trunk/src/site/xdoc/userguide/index.xml
commons/proper/math/trunk/src/site/xdoc/userguide/ode.xml
commons/proper/math/trunk/src/site/xdoc/userguide/optimization.xml
Modified: commons/proper/math/trunk/src/site/site.xml
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/site/site.xml?rev=754764&r1=754763&r2=754764&view=diff
==============================================================================
--- commons/proper/math/trunk/src/site/site.xml (original)
+++ commons/proper/math/trunk/src/site/site.xml Sun Mar 15 21:34:47 2009
@@ -55,7 +55,6 @@
<item name="Fractions" href="/userguide/fraction.html"/>
<item name="Transform Methods" href="/userguide/transform.html"/>
<item name="3D Geometry" href="/userguide/geometry.html"/>
- <item name="Parametric Estimation" href="/userguide/estimation.html"/>
<item name="Optimization" href="/userguide/optimization.html"/>
<item name="Ordinary Differential Equations" href="/userguide/ode.html"/>
</menu>
Modified: commons/proper/math/trunk/src/site/xdoc/changes.xml
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/site/xdoc/changes.xml?rev=754764&r1=754763&r2=754764&view=diff
==============================================================================
--- commons/proper/math/trunk/src/site/xdoc/changes.xml (original)
+++ commons/proper/math/trunk/src/site/xdoc/changes.xml Sun Mar 15 21:34:47 2009
@@ -39,6 +39,9 @@
</properties>
<body>
<release version="2.0" date="TBD" description="TBD">
+ <action dev="luc" type="fix" issue="MATH-177" >
+ Redesigned the optimization framework for a simpler yet more powerful API.
+ </action>
<action dev="luc" type="fix" issue="MATH-243" due-to="Christian Semrau">
Fixed an error in computing gcd and lcm for some extreme values at integer
range boundaries.
Modified: commons/proper/math/trunk/src/site/xdoc/userguide/index.xml
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/site/xdoc/userguide/index.xml?rev=754764&r1=754763&r2=754764&view=diff
==============================================================================
--- commons/proper/math/trunk/src/site/xdoc/userguide/index.xml (original)
+++ commons/proper/math/trunk/src/site/xdoc/userguide/index.xml Sun Mar 15 21:34:47 2009
@@ -112,28 +112,20 @@
<li><a href="geometry.html#a11.2_Vectors">11.2 Vectors</a></li>
<li><a href="geometry.html#a11.3_Rotations">11.3 Rotations</a></li>
</ul></li>
- <li><a href="estimation.html">12. Parametric Estimation</a>
+ <li><a href="optimization.html">12. Optimization</a>
<ul>
- <li><a href="estimation.html#a12.1_Overview">12.1 Overview</a></li>
- <li><a href="estimation.html#a12.2_Models">12.2 Models</a></li>
- <li><a href="estimation.html#a12.3_Parameters">12.3 Parameters</a></li>
- <li><a href="estimation.html#a12.4_Measurements">12.4 Measurements</a></li>
- <li><a href="estimation.html#a12.5_Solvers">12.5 Solvers</a></li>
+ <li><a href="optimization.html#a12.1_Overview">12.1 Overview</a></li>
+ <li><a href="analysis.html#a12.2_Univariate_Functions">12.2 Univariate Functions</a></li>
+ <li><a href="analysis.html#a12.3_Linear_Programming">12.3 Linear Programming</a></li>
+ <li><a href="optimization.html#a12.4_Direct_Methods">12.4 Direct Methods</a></li>
+ <li><a href="analysis.html#a12.5_General_Case">12.5 General Case</a></li>
</ul></li>
- <li><a href="optimization.html">13. Optimization</a>
+ <li><a href="ode.html">13. Ordinary Differential Equations Integration</a>
<ul>
- <li><a href="optimization.html#a13.1_Overview">13.1 Overview</a></li>
- <li><a href="analysis.html#a13.2_Univariate_Functions">13.2 Univariate Functions</a></li>
- <li><a href="analysis.html#a13.3_Linear_Programming">13.3 Linear Programming</a></li>
- <li><a href="optimization.html#a13.4_Direct_Methods">13.4 Direct Methods</a></li>
- <li><a href="analysis.html#a13.5_General_Case">13.5 General Case</a></li>
- </ul></li>
- <li><a href="ode.html">14. Ordinary Differential Equations Integration</a>
- <ul>
- <li><a href="ode.html#a14.1_Overview">14.1 Overview</a></li>
- <li><a href="ode.html#a14.2_Discrete_Events_Handling">14.2 Discrete Events Handling</a></li>
- <li><a href="ode.html#a14.3_ODE_Problems">14.3 ODE Problems</a></li>
- <li><a href="ode.html#a14.4_Integrators">14.4 Integrators</a></li>
+ <li><a href="ode.html#a13.1_Overview">13.1 Overview</a></li>
+ <li><a href="ode.html#a13.2_Discrete_Events_Handling">13.2 Discrete Events Handling</a></li>
+ <li><a href="ode.html#a13.3_ODE_Problems">13.3 ODE Problems</a></li>
+ <li><a href="ode.html#a13.4_Integrators">13.4 Integrators</a></li>
</ul></li>
</ul>
</section>
Modified: commons/proper/math/trunk/src/site/xdoc/userguide/ode.xml
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/site/xdoc/userguide/ode.xml?rev=754764&r1=754763&r2=754764&view=diff
==============================================================================
--- commons/proper/math/trunk/src/site/xdoc/userguide/ode.xml (original)
+++ commons/proper/math/trunk/src/site/xdoc/userguide/ode.xml Sun Mar 15 21:34:47 2009
@@ -26,8 +26,8 @@
</properties>
<body>
- <section name="14 Ordinary Differential Equations Integration">
- <subsection name="14.1 Overview" href="overview">
+ <section name="13 Ordinary Differential Equations Integration">
+ <subsection name="13.1 Overview" href="overview">
<p>
The ode package provides classes to solve Ordinary Differential Equations problems.
</p>
@@ -105,7 +105,7 @@
automatic guess is wrong.
</p>
</subsection>
- <subsection name="14.2 Discrete Events Handling" href="events">
+ <subsection name="13.2 Discrete Events Handling" href="events">
<p>
Discrete events detection is based on switching functions. The user provides
a simple <a href="../apidocs/org/apache/commons/math/ode/events/EventHandler.html">g(t, y)</a>
@@ -176,7 +176,7 @@
}
</source>
</subsection>
- <subsection name="14.3 ODE Problems" href="problems">
+ <subsection name="13.3 ODE Problems" href="problems">
<p>
First order ODE problems are defined by implementing the
<a href="../apidocs/org/apache/commons/math/ode/FirstOrderDifferentialEquations.html">FirstOrderDifferentialEquations</a>
@@ -196,7 +196,7 @@
that implement it are allowed to handle them as they want.
</p>
</subsection>
- <subsection name="14.4 Integrators" href="integrators">
+ <subsection name="13.4 Integrators" href="integrators">
<p>
The tables below show the various integrators available for non-stiff problems.
</p>
Modified: commons/proper/math/trunk/src/site/xdoc/userguide/optimization.xml
URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/site/xdoc/userguide/optimization.xml?rev=754764&r1=754763&r2=754764&view=diff
==============================================================================
--- commons/proper/math/trunk/src/site/xdoc/userguide/optimization.xml (original)
+++ commons/proper/math/trunk/src/site/xdoc/userguide/optimization.xml Sun Mar 15 21:34:47 2009
@@ -26,28 +26,93 @@
</properties>
<body>
- <section name="13 Optimization">
- <subsection name="13.1 Overview" href="overview">
+ <section name="12 Optimization">
+ <subsection name="12.1 Overview" href="overview">
<p>
- The optimization package provides algorithms to optimize (i.e. minimize) some
- objective or cost function. The package is split in several sub-packages
- dedicated to different kind of functions or algorithms.
+ The optimization package provides algorithms to optimize (i.e. either minimize
+ or maximize) some objective or cost function. The package is split in several
+ sub-packages dedicated to different kind of functions or algorithms.
<ul>
- <li>the univariate package handles univariate real functions,</li>
+ <li>the univariate package handles univariate scalar functions,</li>
<li>the linear package handles multivariate vector linear functions
with linear constraints,</li>
- <li>the direct package handles multivariate real functions using direct
- search methods (i.e. not using derivatives),</li>
- <li>the general package handles multivariate real or vector functions
+ <li>the direct package handles multivariate scalar functions
+ using direct search methods (i.e. not using derivatives),</li>
+ <li>the general package handles multivariate scalar or vector functions
using derivatives.</li>
</ul>
</p>
+ <p>
+ The top level optimization package provides common interfaces for the optimization
+ algorithms provided in sub-packages. The main interfaces defines objective functions
+ and optimizers.
+ </p>
+ <p>
+ Objective functions interfaces are intended to be implemented by
+ user code to represent the problem to minimize or maximize. When the goal is to
+ minimize, the objective function is often called a cost function. Objective
+ functions can be either scalar or vectorial and can be either differentiable or
+ not. There are four different interfaces, one for each case:
+ <ul>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/ScalarObjectiveFunction.html">
+ ScalarObjectiveFunction</a></li>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/ScalarDifferentiableObjectiveFunction.html">
+ ScalarDifferentiableObjectiveFunction</a></li>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/VectorialObjectiveFunction.html">
+ VectorialObjectiveFunction</a></li>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/VectorialDifferentiableObjectiveFunction.html">
+ VectorialDifferentiableObjectiveFunction</a></li>
+ </ul>
+ </p>
+
+ <p>
+ Optimizers are the algorithms that will either minimize or maximize, the objective function
+ by changing its input variables set until an optimal set is found. There are only three
+ interfaces defining the common behavior of optimizers, one for each type of objective
+ function except <a href="../apidocs/org/apache/commons/math/optimization/VectorialObjectiveFunction.html">
+ VectorialObjectiveFunction</a>:
+ <ul>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/ScalarOptimizer.html">
+ ScalarOptimizer</a></li>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/ScalarDifferentiableOptimizer.html">
+ ScalarDifferentiableOptimizer</a></li>
+ <li><a href="../apidocs/org/apache/commons/math/optimization/VectorialDifferentiableOptimizer.html">
+ VectorialDifferentiableOptimizer</a></li>
+ </ul>
+ </p>
+
+ <p>
+ Despite there are only three types of supported optimizers, it is possible to optimize a
+ transform a non-differentiable <a
+ href="../apidocs/org/apache/commons/math/optimization/VectorialObjectiveFunction.html">
+ VectorialObjectiveFunction</a> by transforming into a <a
+ href="../apidocs/org/apache/commons/math/optimization/ScalarObjectiveFunction.html">
+ ScalarObjectiveFunction</a> thanks to the <a
+ href="../apidocs/org/apache/commons/math/optimization/LeastSquaresConverter.html">
+ LeastSquaresConverter</a> helper class. The transformed function can be optimized using any
+ implementation of the <a href="../apidocs/org/apache/commons/math/optimization/ScalarOptimizer.html">
+ ScalarOptimizer</a> interface.
+ </p>
+
+ <p>
+ There are also three special implementations which wrap classical optimizers in order to
+ add them a multi-start feature. This feature call the underlying optimizer several times
+ in sequence with different starting points and returns the best optimum found or all optima
+ if desired. This is a classical way to prevent being trapped into a local extremum when
+ looking for a global one. The multi-start wrappers are <a
+ href="../apidocs/org/apache/commons/math/optimization/MultiStartScalarOptimizer.html">
+ MultiStartScalarOptimizer</a>, <a
+ href="../apidocs/org/apache/commons/math/optimization/MultiStartScalarDifferentiableOptimizer.html">
+ MultiStartScalarDifferentiableOptimizer</a> and <a
+ href="../apidocs/org/apache/commons/math/optimization/MultiStartVectorialDifferentiableOptimizer.html">
+ MultiStartVectorialDifferentiableOptimizer</a>.
+ </p>
</subsection>
- <subsection name="13.2 Univariate Functions" href="univariate">
+ <subsection name="12.2 Univariate Functions" href="univariate">
<p>
- A <a href="../apidocs/org/apache/commons/math/analysis/minimization/UnivariateRealMinimizer.html">
- org.apache.commons.math.optimization.univariate.UnivariateRealMinimizer</a>
- is used to find the minimal values of a univariate real-valued function <code>f</code>.
+ A <a href="../apidocs/org/apache/commons/math/optimization/univariate/UnivariateRealMinimizer.html">
+ UnivariateRealMinimizer</a> is used to find the minimal values of a univariate scalar-valued function
+ <code>f</code>.
</p>
<p>
Minimization algorithms usage is very similar to root-finding algorithms usage explained
@@ -55,11 +120,14 @@
finding algorithms is replaced by <code>minimize</code> methods.
</p>
</subsection>
- <subsection name="13.3 Linear Programming" href="linear">
+ <subsection name="12.3 Linear Programming" href="linear">
<p>
+ This package provides an implementation of George Dantzig's simplex algorithm
+ for solving linear optimization problems with linear equality and inequality
+ constraints.
</p>
</subsection>
- <subsection name="13.4 Direct Methods" href="direct">
+ <subsection name="12.4 Direct Methods" href="direct">
<p>
Direct search methods only use cost function values, they don't
need derivatives and don't either try to compute approximation of
@@ -97,8 +165,45 @@
multi-directional</a> method.
</p>
</subsection>
- <subsection name="13.5 General Case" href="general">
+ <subsection name="12.5 General Case" href="general">
+ <p>
+ The general package deals with non-linear vectorial optimization problems when
+ the partial derivatives of the objective function are available.
+ </p>
<p>
+ One important class of estimation problems is weighted least
+ squares problems. They basically consist in finding the values
+ for some parameters p<sub>k</sub> such that a cost function
+ J = sum(w<sub>i</sub>(mes<sub>i</sub> - mod<sub>i</sub>)<sup>2</sup>) is
+ minimized. The various (target<sub>i</sub> - model<sub>i</sub>(p<sub>k</sub>))
+ terms are called residuals. They represent the deviation between a set of
+ target values target<sub>i</sub> and theoretical values computed from
+ models model<sub>i</sub> depending on free parameters p<sub>k</sub>.
+ The w<sub>i</sub> factors are weights. One classical use case is when the
+ target values are experimental observations or measurements.
+ </p>
+ <p>
+ Solving a least-squares problem is finding the free parameters p<sub>k</sub>
+ of the theoretical models such that they are close to the target values, i.e.
+ when the residual are small.
+ </p>
+ <p>
+ Two optimizers are available in the general package, both devoted to least-squares
+ problems. The first one is based on the <a
+ href="../apidocs/org/apache/commons/math/optimization/general/GaussNewtonOptimizer.html">
+ Gauss-Newton</a> method. The second one is the <a
+ href="../apidocs/org/apache/commons/math/optimization/general/LevenbergMarquardtOptimizer.html">
+ Levenberg-Marquardt</a> method.
+ </p>
+ <p>
+ In order to solve a vectorial optimization problem, the user must provide it as
+ an object implementing the <a
+ href="../apidocs/org/apache/commons/math/optimization/VectorialDifferentiableObjectiveFunction.html">
+ VectorialDifferentiableObjectiveFunction</a> interface. The object will be provided to
+ the <code>estimate</code> method of the optimizer, along with the target and weight arrays,
+ thus allowing the optimizer to compute the residuals at will. The last parameter to the
+ <code>estimate</code> method is the point from which the optimizer will start its
+ search for the optimal point.
</p>
</subsection>
</section>