You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@commons.apache.org by lu...@apache.org on 2015/07/27 21:21:29 UTC
[1/2] [math] Typo.
Repository: commons-math
Updated Branches:
refs/heads/MATH_3_X 265ed5dd4 -> d84862558
Typo.
Project: http://git-wip-us.apache.org/repos/asf/commons-math/repo
Commit: http://git-wip-us.apache.org/repos/asf/commons-math/commit/4e828dbe
Tree: http://git-wip-us.apache.org/repos/asf/commons-math/tree/4e828dbe
Diff: http://git-wip-us.apache.org/repos/asf/commons-math/diff/4e828dbe
Branch: refs/heads/MATH_3_X
Commit: 4e828dbe231bed7491eeeb36b817c2d6385dbc6e
Parents: 265ed5d
Author: Luc Maisonobe <lu...@apache.org>
Authored: Mon Jul 27 17:14:00 2015 +0200
Committer: Luc Maisonobe <lu...@apache.org>
Committed: Mon Jul 27 17:14:00 2015 +0200
----------------------------------------------------------------------
src/changes/changes.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/commons-math/blob/4e828dbe/src/changes/changes.xml
----------------------------------------------------------------------
diff --git a/src/changes/changes.xml b/src/changes/changes.xml
index d613ede..03cfec2 100644
--- a/src/changes/changes.xml
+++ b/src/changes/changes.xml
@@ -51,7 +51,7 @@ If the output is not quite correct, check for invisible trailing spaces!
</properties>
<body>
<release version="3.6" date="XXXX-XX-XX" description="">
- <action dev="psteitz" type="fix" issue="MATH-1252 due-to="John Bay">
+ <action dev="psteitz" type="fix" issue="MATH-1252" due-to="John Bay">
ResizableDoubleArray constructor does not work with double array of size 1.
</action>
<action dev="erans" type="fix" issue="MATH-1251">
[2/2] [math] Added a section for least squares fitting in the user
guide.
Posted by lu...@apache.org.
Added a section for least squares fitting in the user guide.
Project: http://git-wip-us.apache.org/repos/asf/commons-math/repo
Commit: http://git-wip-us.apache.org/repos/asf/commons-math/commit/d8486255
Tree: http://git-wip-us.apache.org/repos/asf/commons-math/tree/d8486255
Diff: http://git-wip-us.apache.org/repos/asf/commons-math/diff/d8486255
Branch: refs/heads/MATH_3_X
Commit: d848625585f15a1b53618ebf4f0c66930bb01dea
Parents: 4e828db
Author: Luc Maisonobe <lu...@apache.org>
Authored: Mon Jul 27 20:50:55 2015 +0200
Committer: Luc Maisonobe <lu...@apache.org>
Committed: Mon Jul 27 21:09:36 2015 +0200
----------------------------------------------------------------------
src/site/site.xml | 6 +-
src/site/xdoc/userguide/exceptions.xml | 10 +-
src/site/xdoc/userguide/filter.xml | 6 +-
src/site/xdoc/userguide/fitting.xml | 8 +-
src/site/xdoc/userguide/genetics.xml | 8 +-
src/site/xdoc/userguide/index.xml | 64 +++--
src/site/xdoc/userguide/leastsquares.xml | 366 ++++++++++++++++++++++++++
src/site/xdoc/userguide/ode.xml | 12 +-
src/site/xdoc/userguide/optimization.xml | 7 +-
9 files changed, 436 insertions(+), 51 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/site.xml
----------------------------------------------------------------------
diff --git a/src/site/site.xml b/src/site/site.xml
index 6f45d47..b6e0f7a 100644
--- a/src/site/site.xml
+++ b/src/site/site.xml
@@ -67,13 +67,15 @@
<item name="Distributions" href="/userguide/distribution.html"/>
<item name="Fractions" href="/userguide/fraction.html"/>
<item name="Transform Methods" href="/userguide/transform.html"/>
- <item name="3D Geometry" href="/userguide/geometry.html"/>
+ <item name="Geometry" href="/userguide/geometry.html"/>
<item name="Optimization" href="/userguide/optimization.html"/>
+ <item name="Curve Fitting" href="/userguide/fitting.html"/>
+ <item name="Least Squares" href="/userguide/leastsquares.html"/>
<item name="Ordinary Differential Equations" href="/userguide/ode.html"/>
<item name="Genetic Algorithms" href="/userguide/genetics.html"/>
<item name="Filters" href="/userguide/filter.html"/>
- <item name="Fitting" href="/userguide/fitting.html"/>
<item name="Machine Learning" href="/userguide/ml.html"/>
+ <item name="Exceptions" href="/userguide/exceptions.html"/>
</menu>
<head>
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/exceptions.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/exceptions.xml b/src/site/xdoc/userguide/exceptions.xml
index 55dabf0..3d31810 100644
--- a/src/site/xdoc/userguide/exceptions.xml
+++ b/src/site/xdoc/userguide/exceptions.xml
@@ -23,14 +23,14 @@
<title>The Commons Math User Guide - Exceptions</title>
</properties>
<body>
- <section name="16 Exceptions">
+ <section name="19 Exceptions">
- <subsection name="16.1 Overview" href="overview">
+ <subsection name="19.1 Overview" href="overview">
Commons Math defines a set of exceptions in order to convey the
precise low-level cause of failure.
</subsection>
- <subsection name="16.2 Unchecked Exceptions" href="unchecked">
+ <subsection name="19.2 Unchecked Exceptions" href="unchecked">
<p>
Starting from version 3.0, all exceptions generated by the
Commons Math code are <em>unchecked</em> (i.e. they inherit from
@@ -46,7 +46,7 @@
</p>
</subsection>
- <subsection name="16.3 Hierarchies" href="hierarchies">
+ <subsection name="19.3 Hierarchies" href="hierarchies">
<p>
The exceptions defined by Commons Math follow the Java standard
hierarchies:
@@ -88,7 +88,7 @@
</p>
</subsection>
- <subsection name="16.4 Features" href="features">
+ <subsection name="19.4 Features" href="features">
<ul>
<li>Localization
<p>
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/filter.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/filter.xml b/src/site/xdoc/userguide/filter.xml
index c1919af..78420dc 100644
--- a/src/site/xdoc/userguide/filter.xml
+++ b/src/site/xdoc/userguide/filter.xml
@@ -23,13 +23,13 @@
<title>The Commons Math User Guide - Filters</title>
</properties>
<body>
- <section name="15 Filters">
- <subsection name="15.1 Overview" href="overview">
+ <section name="17 Filters">
+ <subsection name="17.1 Overview" href="overview">
<p>
The filter package currently provides only an implementation of a Kalman filter.
</p>
</subsection>
- <subsection name="15.2 Kalman Filter" href="kalman">
+ <subsection name="17.2 Kalman Filter" href="kalman">
<p>
<a href="../apidocs/org/apache/commons/math3/filter/KalmanFilter.html">
KalmanFilter</a> provides a discrete-time filter to estimate
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/fitting.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/fitting.xml b/src/site/xdoc/userguide/fitting.xml
index 2e38b20..b67aa1e 100644
--- a/src/site/xdoc/userguide/fitting.xml
+++ b/src/site/xdoc/userguide/fitting.xml
@@ -25,8 +25,8 @@
</properties>
<body>
- <section name="17 Curve Fitting">
- <subsection name="17.1 Overview" href="overview">
+ <section name="13 Curve Fitting">
+ <subsection name="13.1 Overview" href="overview">
<p>
The fitting package deals with curve fitting for univariate real functions.
When a univariate real function y = f(x) does depend on some unknown parameters
@@ -76,7 +76,7 @@
</subsection>
- <subsection name="17.2 Implemented Functions" href="special">
+ <subsection name="13.2 Implemented Functions" href="special">
<p>
Fitting of specific functions are provided through the following classes:
@@ -128,7 +128,7 @@ final double[] coeff = fitter.fit(obs.toList());
</subsection>
- <subsection name="17.3 General Case" href="general">
+ <subsection name="13.3 General Case" href="general">
<p>
The <a href="../apidocs/org/apache/commons/math3/fitting/AbstractCurveFitter.html">
AbstractCurveFitter</a> class provides the basic functionality for implementing other
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/genetics.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/genetics.xml b/src/site/xdoc/userguide/genetics.xml
index ed29867..ba87ad6 100644
--- a/src/site/xdoc/userguide/genetics.xml
+++ b/src/site/xdoc/userguide/genetics.xml
@@ -23,14 +23,14 @@
<title>The Commons Math User Guide - Genetic Algorithms</title>
</properties>
<body>
- <section name="14 Genetic Algorithms">
- <subsection name="14.1 Overview" href="overview">
+ <section name="16 Genetic Algorithms">
+ <subsection name="16.1 Overview" href="overview">
<p>
The genetics package provides a framework and implementations for
genetic algorithms.
</p>
</subsection>
- <subsection name="14.2 GA Framework">
+ <subsection name="16.2 GA Framework">
<p>
<a href="../apidocs/org/apache/commons/math3/genetics/GeneticAlgorithm.html">
GeneticAlgorithm</a> provides an execution framework for Genetic Algorithms (GA).
@@ -76,7 +76,7 @@
</ol>
</p>
</subsection>
- <subsection name="14.3 Implementation">
+ <subsection name="16.3 Implementation">
<p>
Here is an example GA execution:
<source>
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/index.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/index.xml b/src/site/xdoc/userguide/index.xml
index a80f40a..0dfd8e6 100644
--- a/src/site/xdoc/userguide/index.xml
+++ b/src/site/xdoc/userguide/index.xml
@@ -128,39 +128,43 @@
<li><a href="optimization.html#a12.4_Direct_Methods">12.4 Direct Methods</a></li>
<li><a href="optimization.html#a12.5_General_Case">12.5 General Case</a></li>
</ul></li>
- <li><a href="ode.html">13. Ordinary Differential Equations Integration</a>
- <ul>
- <li><a href="ode.html#a13.1_Overview">13.1 Overview</a></li>
- <li><a href="ode.html#a13.2_Continuous_Output">13.2 Continuous Output</a></li>
- <li><a href="ode.html#a13.3_Discrete_Events_Handling">13.3 Discrete Events Handling</a></li>
- <li><a href="ode.html#a13.4_Available_Integrators">13.4 Available Integrators</a></li>
- <li><a href="ode.html#a13.5_Derivatives">13.5 Derivatives</a></li>
- </ul></li>
- <li><a href="genetics.html">14. Genetic Algorithms</a>
- <ul>
- <li><a href="genetics.html#a14.1_Overview">14.1 Overview</a></li>
- <li><a href="genetics.html#a14.2_GA_Framework">14.2 GA Framework</a></li>
- <li><a href="genetics.html#a14.3_Implementation">14.3 Implementation and Examples</a></li>
- </ul></li>
- <li><a href="filter.html">15. Filters</a>
+ <li><a href="fitting.html">13. Curve Fitting</a>
<ul>
- <li><a href="filter.html#a15.1_Overview">15.1 Overview</a></li>
- <li><a href="filter.html#a15.2_Kalman_Filter">15.2 Kalman Filter</a></li>
+ <li><a href="fitting.html#a13.1_Overview">13.1 Overview</a></li>
+ <li><a href="fitting.html#a13.2_Implemented_Functions">13.2 Implemented Functions</a></li>
+ <li><a href="fitting.html#a13.3_General_Case">13.3 General Case</a></li>
</ul>
</li>
- <li><a href="exceptions.html">16. Exceptions</a>
+ <li><a href="leastsquares.html">14. Least Squares</a>
<ul>
- <li><a href="exceptions.html#a16.1_Overview">16.1 Overview</a></li>
- <li><a href="exceptions.html#a16.2_Unchecked_Exceptions">16.2 Unchecked Exceptions</a></li>
- <li><a href="exceptions.html#a16.3_Hierarchies">16.3 Hierarchies</a></li>
- <li><a href="exceptions.html#a16.4_Features">16.4 Features</a></li>
+ <li><a href="fitting.html#a14.1_Overview">14.1 Overview</a></li>
+ <li><a href="fitting.html#a14.2_LeastSquaresBuilder_and_LeastSquaresFactory">14.2 LeastSquaresBuilder and LeastSquaresFactory</a></li>
+ <li><a href="fitting.html#a14.3_Model_Function">14.3 Model Function</a></li>
+ <li><a href="fitting.html#a14.4_Parameters_Validation">14.4 Parameters Validation</a></li>
+ <li><a href="fitting.html#a14.5_Tuning">14.5 Tuning</a></li>
+ <li><a href="fitting.html#a14.6_Optimization_Engine">14.6 Optimization Engine</a></li>
+ <li><a href="fitting.html#a14.7_Solving">14.7 Solving</a></li>
+ <li><a href="fitting.html#a14.8_Example">14.8 Example</a></li>
</ul>
</li>
- <li><a href="fitting.html">17. Curve Fitting</a>
+ <li><a href="ode.html">15. Ordinary Differential Equations Integration</a>
+ <ul>
+ <li><a href="ode.html#a15.1_Overview">15.1 Overview</a></li>
+ <li><a href="ode.html#a15.2_Continuous_Output">15.2 Continuous Output</a></li>
+ <li><a href="ode.html#a15.3_Discrete_Events_Handling">15.3 Discrete Events Handling</a></li>
+ <li><a href="ode.html#a15.4_Available_Integrators">15.4 Available Integrators</a></li>
+ <li><a href="ode.html#a15.5_Derivatives">15.5 Derivatives</a></li>
+ </ul></li>
+ <li><a href="genetics.html">16. Genetic Algorithms</a>
+ <ul>
+ <li><a href="genetics.html#a16.1_Overview">16.1 Overview</a></li>
+ <li><a href="genetics.html#a16.2_GA_Framework">16.2 GA Framework</a></li>
+ <li><a href="genetics.html#a16.3_Implementation">16.3 Implementation and Examples</a></li>
+ </ul></li>
+ <li><a href="filter.html">17. Filters</a>
<ul>
- <li><a href="fitting.html#a17.1_Overview">17.1 Overview</a></li>
- <li><a href="fitting.html#a17.2_General_Case">17.2 General Case</a></li>
- <li><a href="fitting.html#a17.3_Special_Cases">17.3 Special Cases</a></li>
+ <li><a href="filter.html#a17.1_Overview">17.1 Overview</a></li>
+ <li><a href="filter.html#a17.2_Kalman_Filter">17.2 Kalman Filter</a></li>
</ul>
</li>
<li><a href="ml.html">18. Machine Learning</a>
@@ -170,6 +174,14 @@
<li><a href="ml.html#implementation">18.3 Implementation</a></li>
</ul>
</li>
+ <li><a href="exceptions.html">19. Exceptions</a>
+ <ul>
+ <li><a href="exceptions.html#a19.1_Overview">19.1 Overview</a></li>
+ <li><a href="exceptions.html#a19.2_Unchecked_Exceptions">19.2 Unchecked Exceptions</a></li>
+ <li><a href="exceptions.html#a19.3_Hierarchies">19.3 Hierarchies</a></li>
+ <li><a href="exceptions.html#a19.4_Features">19.4 Features</a></li>
+ </ul>
+ </li>
</ul>
</section>
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/leastsquares.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/leastsquares.xml b/src/site/xdoc/userguide/leastsquares.xml
new file mode 100644
index 0000000..f61b82e
--- /dev/null
+++ b/src/site/xdoc/userguide/leastsquares.xml
@@ -0,0 +1,366 @@
+<?xml version="1.0"?>
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ -->
+
+<?xml-stylesheet type="text/xsl" href="./xdoc.xsl"?>
+<document url="fitting.html">
+
+ <properties>
+ <title>The Commons Math User Guide - Least squares</title>
+ </properties>
+
+ <body>
+ <section name="14 Least squares">
+ <subsection name="14.1 Overview">
+ <p>
+ The least squares package fits a parametric model to a set of observed
+ values by minimizing a cost function with a specific form.
+ The fitting basically consists in finding the values
+ for some parameters p<sub>k</sub> such that a cost function
+ J = sum(w<sub>i</sub>(target<sub>i</sub> - model<sub>i</sub>)<sup>2</sup>) is
+ minimized. The various (target<sub>i</sub> - model<sub>i</sub>(p<sub>k</sub>))
+ terms are called residuals. They represent the deviation between a set of
+ target values target<sub>i</sub> and theoretical values computed from
+ models model<sub>i</sub> depending on free parameters p<sub>k</sub>.
+ The w<sub>i</sub> factors are weights. One classical use case is when the
+ target values are experimental observations or measurements.
+ </p>
+ <p>
+ Two engines devoted to least-squares problems are available. The first one is
+ based on the <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/GaussNewtonOptimizer.html">
+ Gauss-Newton</a> method. The second one is the <a
+ href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LevenbergMarquardtOptimizer.html">
+ Levenberg-Marquardt</a> method.
+ </p>
+ </subsection>
+
+ <subsection name="14.2 LeastSquaresBuilder and LeastSquaresFactory">
+
+ <p>
+ In order to solve a least-squares fitting problem, the user must provide the following elements:
+ <ul>
+ <li>a mean to evaluate all the components of the model for a given set of parameters:
+ model<sub>i</sub> = f<sub>i</sub>(p<sub>1</sub>, p<sub>2</sub>, ... p<sub>k</sub>),
+ this is code</li>
+ <li>the target (or observed) components: target<sub>i</sub>, this is data</li>
+ <li>the start values for all p<sub>k</sub> parameters: s<sub>k</sub>, this is data</li>
+ <li>optionally a validator for the p<sub>k</sub> parameters, this is code</li>
+ <li>optionally the weight for sample point i: w<sub>i</sub>, this is data and defaults to 1.0 if not provided</li>
+ <li>a maximum number of iterations, this is data</li>
+ <li>a maximum number of evaluations, this is data</li>
+ <li>a convergence criterion, this is code</li>
+ </ul>
+ </p>
+ <p>
+ The elements of the list above can be provided as an implementation of the
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresProblem.html">
+ LeastSquaresProblem</a> interface. However, this is cumbersome to do directly, so some helper
+ classes are available. The first helper is a mutable builder:
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresBuilder.html">
+ LeastSquaresBuilder</a>. The second helper is an utility factory:
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresFactory.html">
+ LeastSquaresFactory</a>.
+ </p>
+ <p>
+ The builder class is better suited when setting the various elements of the least squares
+ problem is done progressively in different places in the user code. In this case, the user
+ would create first an empty builder andconfigure it progressively by calling its methods
+ (<code>start</code>, <code>target</code>, <code>model</code>, ...). Once the configuration
+ is complete, calling the <code>build</code> method would create the least squares problem.
+ </p>
+ <p>
+ The factory utility is better suited when the various elements of the least squares
+ problem are all known at one place and the problem can be built in just one sweep, calling
+ to one of the static <code>LeastSquaresFactory.create</code> method.
+ </p>
+ </subsection>
+
+ <subsection name="14.3 Model Function">
+ <p>
+ The model function is used by the least squares engine to evaluate the model components
+ model<sub>i</sub> given some test parameters p<sub>k</sub>. It is therefore a multivariate
+ function (it depends on the various p<sub>k</sub>) and it is vector-valued (it has several
+ components model<sub>i</sub>). There must be exactly one component model<sub>i</sub> for
+ each target (or observed) component target<sub>i</sub>, otherwise some residuals to be
+ squared and summed could not be computed. In order for the problem to be well defined, the
+ number of parameters p<sub>k</sub> must be less than the number of components model<sub>i</sub>.
+ Failing to ensure this may lead to the engine throwing an exception as the underlying linear
+ algebra operations may encounter singular matrices. It is not unusual to have a large number
+ of components (several thousands) and only a dozen parameters. There are no limitations on these
+ numbers, though.
+ </p>
+ <p>
+ As the least squares engine needs to create Jacobians matrices for the model function, both
+ its value and its derivatives <em>with respect to the p<sub>k</sub> parameters</em> must
+ be available. There are two ways to provide this:
+ <ul>
+ <li>provide one
+ <a href="../apidocs/org/apache/commons/math3/analysis/MultivariateVectorFunction.html">MultivariateVectorFunction</a>
+ instance for computing the components values and one
+ <a href="../apidocs/org/apache/commons/math3/analysis/MultivariateMatrixFunction.html">MultivariateMatrixFunction</a>
+ instance for computing the components derivatives (i.e. the Jacobian matrix) with
+ respect to the parameters,</li>
+ <li>or provide one
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/MultivariateJacobianFunction.html">MultivariateJacobianFunction</a>
+ instance for computing both the components values and their derivatives simultaneously.</li>
+ </ul>
+ The first alternative is best suited for models which are not computationally intensive
+ as it allows more modularized code with one method for each type of computation. The second
+ alternative is best suited for models which are computationally intensive and evaluating
+ both the values and derivatives in one sweep saves a lot of work.
+ </p>
+ <p>
+ The <code>point</code> parameter of the <code>value</code> methods in the
+ <a href="../apidocs/org/apache/commons/math3/analysis/MultivariateVectorFunction.html">MultivariateVectorFunction</a>,
+ <a href="../apidocs/org/apache/commons/math3/analysis/MultivariateMatrixFunction.html">MultivariateMatrixFunction</a>,
+ or <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/MultivariateJacobianFunction.html">MultivariateJacobianFunction</a>
+ interfaces will contain the parameters p<sub>k</sub>. The values will be the model components
+ model<sub>i</sub> and the derivatives will be the derivatives of the model components
+ with respect to the parameters dmodel<sub>i</sub>/dp<sub>k</sub>.
+ </p>
+ <p>
+ There are no requirements on how to compute value and derivatives. The
+ <a href="../apidocs/org/apache/commons/math3/analysis/differentiation/DerivativeStructure.html">
+ DerivativeStructure</a> class may be useful to compute analytically derivatives in
+ difficult cases, but this class is not mandated by the API which only expects the derivatives
+ as a Jacobian matrix containing primitive double entries.
+ </p>
+ <p>
+ One non-obvious feature provided by both the builder and the factory is lazy evaluation. This feature
+ allows to defer calls to the model functions until they are really needed by the engine. This
+ can save some calls for engines that evaluate the value and the Jacobians in different loops
+ (this is the case for Levenberg-Marquardt). However, lazy evaluation is possible <em>only</em>
+ if the model functions are themselves separated, i.e. it can be used only with the first
+ alternative above. Setting up the <code>lazyEvaluation</code> flag to <code>true</code> in the builder
+ or factory and setting up the model function as one
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/MultivariateJacobianFunction.html">MultivariateJacobianFunction</a>
+ instance at the same time will trigger an illegal state exception telling that the model function
+ misses required functionality.
+ </p>
+ </subsection>
+
+ <subsection name="14.4 Parameters Validation">
+ <p>
+ In some cases, the model function requires parameters to lie within a specific domain. For example
+ a parameter may be used in a square root and needs to be positive, or another parameter represents
+ the sine of an angle and should be within -1 and +1, or several parameters may need to remain in
+ the unit circle and the sum of their squares must be smaller than 1. The least square solvers available
+ in Apache Commons Math currently don't allow to set up constraints on the parameters. This is a
+ known missing feature. There are two ways to circumvent this.
+ </p>
+ <p>
+ Both ways are achieved by setting up a
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/ParameterValidator.html">ParameterValidator</a>
+ instance. The input of the value and jacobian model functions will always be the output of
+ the parameter validator if one exists.
+ </p>
+ <p>
+ One way to constrain parameters is to use a continuous mapping between the parameters that the
+ least squares solver will handle and the real parameters of the mathematical model. Using mapping
+ functions like <code>logit</code> and <code>sigmoid</code>, one can map a finite range to the
+ infinite real line. Using mapping functions based on <code>log</code> and <code>exp</code>, one
+ can map a semi-infinite range to the infinite real line. It is possible to use such a mapping so
+ that the engine will always see unbounded parameters, whereas on the other side of the mapping the
+ mathematical model will always see parameters mapped correctly to the expected range. Care must be
+ taken with derivatives as one must remember that the parameters have been mapped. Care must also
+ be taken with convergence status. This may be tricky.
+ </p>
+ <p>
+ Another way to constrain parameters is to simply truncate the parameters back to the domain when
+ one search point escapes from it and not care about derivatives. This works <em>only</em> if the
+ solution is expected to be inside the domain and not at the boundary, as points out of the domain
+ will only be temporary test points with a cost function higher than the real solution and will soon
+ be dropped by the underlying engine. As a rule of thumb, these conditions are met only when the
+ domain boundaries correspond to unrealistic values that will never be achieved (null distances,
+ negative masses, ...) but they will not be met when the domain boundaries are more operational
+ limits (a maximum weight that can be handled by a device, a minimum temperature that can be
+ sustained by an instrument, ...).
+ </p>
+ </subsection>
+
+ <subsection name="14.5 Tuning">
+ <p>
+ Among the elements to be provided to the least squares problem builder or factory
+ are some tuning parameters for the solver.
+ </p>
+ <p>
+ The maximum number of iterations refers to the engine algorithm main loop, whereas the
+ maximum number of iterations refers to the number of calls to the model method. Some
+ algorithms (like Levenberg-Marquardt) have two embedded loops, with iteration number
+ being incremented at outer loop level, but a new evaluation being done at each inner
+ loop. In this case, the number of evaluations will be greater than the number of iterations.
+ Other algorithms (like Gauss-Newton) have only one level of loops. In this case, the
+ number of evaluations will equal to the number of iterations. In any case, the maximum
+ numbers are really only intended as safeguard to prevent infinite loops, so the exact
+ value of the limit is not important so it is common to select some almost arbitrary number
+ much larger than the expected number of evaluations and use it for both
+ <code>maxIterations</code> and <code>maxEvaluations</code>. As an example, if the least
+ squares solver usually finds a solution in 50 iterations, setting a maximum value to 1000
+ is probably safe and prevents infinite loops. If the least squares solver needs several
+ hundreds of evaluations, it would probably be safer to set the maximum value to 10000 or
+ even 1000000 to avoid failures in slightly more demanding cases. Very fine tuning of
+ these maximum numbers is often worthless, they are only intended as safeguards.
+ </p>
+ <p>
+ Convergence checking is delegated to a dedicated interface from the <code>optim</code>
+ package: <a href="../apidocs/org/apache/commons/math3/optim/ConvergenceChecker.html">
+ ConvergenceChecker</a>, parameterized with either the specific
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresProblem.Evaluation.html">Evaluation</a>
+ class used for least squares problems or the general
+ <a href="../apidocs/org/apache/commons/math3/optim/PointVectorValuePair.html">PointVectorValuePair</a>.
+ Each time convergence is checked, both the previous
+ and the current evaluations of the least squares problem are provided, so the checker can
+ compare them and decide whereas convergence has been reached or not. The predefined convergence
+ checker implementations that can be useful for least squares fitting are:
+ <ul>
+ <li><a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/EvaluationRmsChecker.html">EvaluationRmsChecker</a>,
+ which uses only the normalized cost (square-root of the sum of squared of the residuals,
+ divided by the number of measurements),</li>
+ <li><a href="../apidocs/org/apache/commons/math3/optim/SimpleVectorValueChecker.html">SimpleVectorValueChecker</a>,
+ which uses the model components themselves (<em>not</em> the residuals),</li>
+ <li><a href="../apidocs/org/apache/commons/math3/optim/SimplePointChecker.html">SimplePointChecker<PointVectorValuePair></a>,
+ which uses the parameters.</li>
+ </ul>
+ Of course, users can also provide their own implementation of the
+ <a href="../apidocs/org/apache/commons/math3/optim/ConvergenceChecker.html">ConvergenceChecker</a>
+ interface.
+ </p>
+ </subsection>
+
+ <subsection name="14.6 Optimization Engine">
+ <p>
+ Once the least squares problem has been created, using either the builder or the factory,
+ it is passed to an optimization engine for solving. Two engines devoted to least-squares
+ problems are available. The first one is
+ based on the <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/GaussNewtonOptimizer.html">
+ Gauss-Newton</a> method. The second one is the <a
+ href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LevenbergMarquardtOptimizer.html">
+ Levenberg-Marquardt</a> method. For both increased readability and in order to leverage
+ possible future changes in the configuration, it is recommended to use the fluent-style API to
+ build and configure the optimizers. This means creating a first temporary version of the optimizer
+ with a default parameterless constructor, and then to set up the various configuration parameters
+ using the available <code>withXxx</code> methods that all return a new optimizer instance. Only the
+ final fully configured instance is used. As an example, setting up a Levenberg-Marquardt with
+ all configuration set to default except the cost relative tolerance and parameter relative tolerance
+ would be done as follows:
+ </p>
+ <source>
+ LeastSquaresOptimizer optimizer = new LevenbergMarquardtOptimizer().
+ withCostRelativeTolerance(1.0e-12).
+ withParameterRelativeTolerance(1.0e-12);
+ </source>
+
+ <p>
+ As another example, setting up a Gauss-Newton optimizer and forcing the decomposition to SVD (the
+ default is QR decomposition) would be done as follows:
+ </p>
+ <source>
+ LeastSquaresOptimizer optimizer = new GaussNewtonOptimizer().
+ withwithDecomposition(GaussNewtonOptimizer.Decomposition.QR);
+ </source>
+
+ </subsection>
+
+ <subsection name="14.7 Solving">
+ <p>
+ Solving the least squares problem is done by calling the <code>optimize</code> method of the
+ optimizer and passing the least squares problem as the single parameter:
+ </p>
+ <source>
+ LeastSquaresOptimizer.Optimum optimum = optimizer.optimize(leastSquaresProblem);
+ </source>
+
+ <p>
+ The <a
+ href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresOptimizer.Optimum.html">
+ LeastSquaresOptimizer.Optimum</a> class is a specialized
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresProblem.Evaluation.html">Evaluation</a>
+ with additional methods te retrieve the number of evaluations and number of iterations performed.
+ The most important methods are inherited from the
+ <a href="../apidocs/org/apache/commons/math3/fitting/leastsquares/LeastSquaresProblem.Evaluation.html">Evaluation</a>
+ class and correspond to the point (i.e. the parameters), cost, Jacobian, RMS, covariance ...
+ </p>
+ </subsection>
+
+ <subsection name="14.8 Example">
+ <p>
+ The following simple example shows how to find the center of a circle of known radius to
+ to best fit observed 2D points. It is a simplified version of one of the JUnit test cases.
+ In the complete test case, both the circle center and its radius are fitted, here the
+ radius is fixed.
+ </p>
+ <source>
+ final double radius = 70.0;
+ final Vector2D[] observedPoints = new Vector2D[] {
+ new Vector2D( 30.0, 68.0),
+ new Vector2D( 50.0, -6.0),
+ new Vector2D(110.0, -20.0),
+ new Vector2D( 35.0, 15.0),
+ new Vector2D( 45.0, 97.0)
+ };
+
+ // the model function components are the distances to current estimated center,
+ // they should be as close as possible to the specified radius
+ MultivariateJacobianFunction distancesToCurrentCenter = new MultivariateJacobianFunction() {
+ public Pair<RealVector, RealMatrix> value(final RealVector point) {
+
+ Vector2D center = new Vector2D(point.getEntry(0), point.getEntry(1));
+
+ RealVector value = new ArrayRealVector(observedPoints.length);
+ RealMatrix jacobian = new Array2DRowRealMatrix(observedPoints.length, 2);
+
+ for (int i = 0; i < observedPoints.length; ++i) {
+ Vector2D o = observedPoints[i];
+ double modelI = Vector2D.distance(o, center);
+ value.setEntry(i, modelI);
+ // derivative with respect to p0 = x center
+ jacobian.setEntry(i, 0, (center.getX() - o.getX()) / modelI);
+ // derivative with respect to p1 = y center
+ jacobian.setEntry(i, 1, (center.getX() - o.getX()) / modelI);
+ }
+
+ return new Pair<RealVector, RealMatrix>(value, jacobian);
+
+ }
+ };
+
+ // the target is to have all points at the specified radius from the center
+ double[] prescribedDistances = new double[observedPoints.length];
+ Arrays.fill(prescribedDistances, radius);
+
+ // least squares problem to solve : modeled radius should be close to target radius
+ LeastSquaresProblem problem = new LeastSquaresBuilder().
+ start(new double[] { 100.0, 50.0 }).
+ model(distancesToCurrentCenter).
+ target(prescribedDistances).
+ lazyEvaluation(false).
+ maxEvaluations(1000).
+ maxIterations(1000).
+ build();
+ LeastSquaresOptimizer.Optimum optimum = new LevenbergMarquardtOptimizer().optimize(problem);
+ Vector2D fittedCenter = new Vector2D(optimum.getPoint().getEntry(0), optimum.getPoint().getEntry(1));
+ System.out.println("fitted center: " + fittedCenter.getX() + " " + fittedCenter.getY());
+ System.out.println("RMS: " + optimum.getRMS());
+ System.out.println("evaluations: " + optimum.getEvaluations());
+ System.out.println("iterations: " + optimum.getIterations());
+ </source>
+ </subsection>
+
+ </section>
+ </body>
+</document>
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/ode.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/ode.xml b/src/site/xdoc/userguide/ode.xml
index d019e12..0cd246b 100644
--- a/src/site/xdoc/userguide/ode.xml
+++ b/src/site/xdoc/userguide/ode.xml
@@ -25,8 +25,8 @@
</properties>
<body>
- <section name="13 Ordinary Differential Equations Integration">
- <subsection name="13.1 Overview" href="overview">
+ <section name="15 Ordinary Differential Equations Integration">
+ <subsection name="15.1 Overview" href="overview">
<p>
The ode package provides classes to solve Ordinary Differential Equations problems.
</p>
@@ -115,7 +115,7 @@ double[] y = new double[] { 0.0, 1.0 }; // initial state
dp853.integrate(ode, 0.0, y, 16.0, y); // now y contains final state at time t=16.0
</source>
</subsection>
- <subsection name="13.2 Continuous Output" href="continuous">
+ <subsection name="15.2 Continuous Output" href="continuous">
<p>
The solution of the integration problem is provided by two means. The first one is aimed towards
simple use: the state vector at the end of the integration process is copied in the y array of the
@@ -178,7 +178,7 @@ integrator.addStepHandler(stepHandler);
automatic guess is wrong.
</p>
</subsection>
- <subsection name="13.3 Discrete Events Handling" href="events">
+ <subsection name="15.3 Discrete Events Handling" href="events">
<p>
ODE problems are continuous ones. However, sometimes discrete events must be
taken into account. The most frequent case is the stop condition of the integrator
@@ -256,7 +256,7 @@ public int eventOccurred(double t, double[] y, boolean increasing) {
}
</source>
</subsection>
- <subsection name="13.4 Available Integrators" href="integrators">
+ <subsection name="15.4 Available Integrators" href="integrators">
<p>
The tables below show the various integrators available for non-stiff problems. Note that the
implementations of Adams-Bashforth and Adams-Moulton are adaptive stepsize, not fixed stepsize
@@ -288,7 +288,7 @@ public int eventOccurred(double t, double[] y, boolean increasing) {
</table>
</p>
</subsection>
- <subsection name="13.5 Derivatives" href="derivatives">
+ <subsection name="15.5 Derivatives" href="derivatives">
<p>
If in addition to state y(t) the user needs to compute the sensitivity of the final state with respect to
the initial state (dy/dy<sub>0</sub>) or the sensitivity of the final state with respect to some parameters
http://git-wip-us.apache.org/repos/asf/commons-math/blob/d8486255/src/site/xdoc/userguide/optimization.xml
----------------------------------------------------------------------
diff --git a/src/site/xdoc/userguide/optimization.xml b/src/site/xdoc/userguide/optimization.xml
index 16202d9..43dc16f 100644
--- a/src/site/xdoc/userguide/optimization.xml
+++ b/src/site/xdoc/userguide/optimization.xml
@@ -27,7 +27,12 @@
<body>
<section name="12 Optimization">
<p><em>The contents of this section currently describes deprecated classes.</em>
- Please refer to the new <a href="../apidocs/org/apache/commons/math3/optim/package-summary.html">API description</a>.
+ Please refer to the new <a href="../apidocs/org/apache/commons/math3/optim/package-summary.html">API
+ description</a>.
+ </p>
+ <p>Least squares optimizers are not in this package anymore, they have been moved
+ in a dedicated least-squares sub-package described in the <a href="./leastsquares.html">least squares</a>
+ section.
</p>
<subsection name="12.1 Overview" href="overview">