Re: [math] Estimator capabilities

[Al Lelopath <jd...@gmail.com>]

Anent tangent and quadratic estimates:

Specifies the approach used to obtain initial estimates of the basic
variables in each one-dimensional search.

Tangent Uses linear extrapolation from a tangent vector.

Quadratic Uses quadratic extrapolation, which can improve the results
on highly nonlinear problems. (Excel does this).

Please let me know if this explanation is still insufficient.

On Nov 30, 2007 12:36 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> luc.maisonobe@free.fr a écrit :
>
>
>
> > Selon Al Lelopath <jd...@gmail.com>:
> >
> >> I see that apache commons has Estimators (aka Solvers, yes?):
> >> EstimatedParameter
> >> GaussNewtonEstimator
> >> LevenbergMarquardtEstimator
> >> SimpleEstimationProblem
> >> WeightedMeasurement
> >
> > Not all these classes represent solvers. Only the ones that implement Estimator
> > (i.e. GaussNewtonEstimator and LevenbergMarquardtEstimator) are estimators. The
> > other classes are used for estimation but are not the solvers themselves.
> > EStimatedParameters represent the parameters of the models, and the job of the
> > solver is to update these parameters in order to minimize a cost function. When
> > the solvers has finished its work, retrieving the results is a matter of reading
> > the updated value hold by the instances of this class. SimpleEstimationProblem
> > is mostly a container for the parameters and the measurements. The basic usage
> > is for the user to create a problem-specific class extending this one, the
> > created class implementing the logic of the model (relation between parameters
> > and model, relation between model and theoretical measurements, container for
> > the observed measurements). The cost function is automatically computed from the
> > residuals between theoretical and observed measurements). The
> > WeightedMeasurements class represent a measurement, it should also be
> > specialized by creating a class extending this one for each type of
> > measurements.
> >> Our mathematician has specified features we will need:
> >> 1. Assume non-negative parameters,
> >
> > This feature is a simple bound constraint. The estimators do not
> > handle any constraints yet (neither simple bounds, nor linear, nor non-linear).
> > This will most probably be added one day as it is rather important.
> >
> > Simple bounds constraints can be simulated artificially though, by adding a
> > layer between the solved parameters and the real parameters of the model. For
> > example, if your model needs three parameters p1, p2 and p3 with p1 that should
> > be positive, you can really solve using q1, p2, p3 with an additional transform
> > p1 = q1^2 implemented by your model. Similar transforms can be built for double
> > bounds constraints.
> >
> >> 2. forward and central derivatives,
> >
> > The derivatives are not computed by the classes provided by commons-math, they
> > should be provided by the user when extending the SimpleEstimationProblem class
> > (or implementing directly the EstimationProblem interface if prefered). This
> > means you can use any method you prefer for computeing them : forward, central
> > or backward finite differences or analycally computed derivatives.
> >
> >> 3. tangent estimates
> >
> > I'm not sure I understand what you mean here. The solvers are based on gradient
> > computation at the current point which is updated according to the selected
> > solver algorithm.
> >
> >> 4. quadratic estimates,
> >
> > I'm also not sure to understand what you mean here. The problem are defined in
> > terms of observed measurements which are defined at problem start and
> > theoretical measurements which are automatically recomputed during the search as
> > the current point moves. The algorithms compute a quadratic cost function as the
> > sum of the weighted residuals between theoretical measurements and observed
>
> I meant sum of weighted SQUARED residuals, of course.
>
>
> > measurements. This cost function is minimized (i.e. parameters are adjusted in
> > such a way theoretically recomputed measurements match the observations). The
> > quadratic structure of the cost function is fixed, but the measurements involved
> > and the weights can be specified by the user.
> >
> >> 5. Newton and conjugate search direction
> >
> > The GaussNewtonEstimator is based on ... the Gauss-Newton algorithm which
> > iterates over a Newton search.
> >
> > The LevenbergMarquardtEstimator is a much more elaborate algorithm which
> > interpolates between a Gauss-Newton and a gradient method. It is more robust
> > than Gauss-Newton and highly recommended.
> >
> > Commons-math currently does not yet provide a simple conjugate-gradient method.
> >
> >
> >> Does apache commons have these capabilities?
> >
> > To conclude, commons-math provide some of these capabilities, but not all of
> > them.
> >
> > Luc
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> > For additional commands, e-mail: user-help@commons.apache.org
> >
> >
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: [math] Estimator capabilities

[Luc Maisonobe <Lu...@free.fr>]

Al Lelopath wrote:
> Anent tangent and quadratic estimates:
> 
> Specifies the approach used to obtain initial estimates of the basic
> variables in each one-dimensional search.
> 
> Tangent Uses linear extrapolation from a tangent vector.
> 
> Quadratic Uses quadratic extrapolation, which can improve the results
> on highly nonlinear problems. (Excel does this).

The methods implemented here are really popular and widely known 
methods. They do use linear models based on tangents (gradients) or 
quadratic models based on curvature (approximated Hessian). However, 
during the one-dimensional search they also re-evaluate the real 
function several times as needed, they do not simply rely on the linear 
or quadratic estimates, which would be very bad for ill-conditioned 
problems.

> 
> Please let me know if this explanation is still insufficient.

I'm still not sure I understand, sorry.

If this can help you understand what is provided, we implemented widely 
known existing methods: the basic Gauss-Newton method and the much more 
elaborate Levenberg-Marquardt method. We did not develop some custom 
made method based on very basic features such as tangents or curvature.

Does your mathematician want some other method like for example 
conjugate gradient ? If so, this could really be added easily. Does he 
want something else ?

Luc

> 
> On Nov 30, 2007 12:36 PM, Luc Maisonobe <Lu...@free.fr> wrote:
>> luc.maisonobe@free.fr a écrit :
>>
>>
>>
>>> Selon Al Lelopath <jd...@gmail.com>:
>>>
>>>> I see that apache commons has Estimators (aka Solvers, yes?):
>>>> EstimatedParameter
>>>> GaussNewtonEstimator
>>>> LevenbergMarquardtEstimator
>>>> SimpleEstimationProblem
>>>> WeightedMeasurement
>>> Not all these classes represent solvers. Only the ones that implement Estimator
>>> (i.e. GaussNewtonEstimator and LevenbergMarquardtEstimator) are estimators. The
>>> other classes are used for estimation but are not the solvers themselves.
>>> EStimatedParameters represent the parameters of the models, and the job of the
>>> solver is to update these parameters in order to minimize a cost function. When
>>> the solvers has finished its work, retrieving the results is a matter of reading
>>> the updated value hold by the instances of this class. SimpleEstimationProblem
>>> is mostly a container for the parameters and the measurements. The basic usage
>>> is for the user to create a problem-specific class extending this one, the
>>> created class implementing the logic of the model (relation between parameters
>>> and model, relation between model and theoretical measurements, container for
>>> the observed measurements). The cost function is automatically computed from the
>>> residuals between theoretical and observed measurements). The
>>> WeightedMeasurements class represent a measurement, it should also be
>>> specialized by creating a class extending this one for each type of
>>> measurements.
>>>> Our mathematician has specified features we will need:
>>>> 1. Assume non-negative parameters,
>>> This feature is a simple bound constraint. The estimators do not
>>> handle any constraints yet (neither simple bounds, nor linear, nor non-linear).
>>> This will most probably be added one day as it is rather important.
>>>
>>> Simple bounds constraints can be simulated artificially though, by adding a
>>> layer between the solved parameters and the real parameters of the model. For
>>> example, if your model needs three parameters p1, p2 and p3 with p1 that should
>>> be positive, you can really solve using q1, p2, p3 with an additional transform
>>> p1 = q1^2 implemented by your model. Similar transforms can be built for double
>>> bounds constraints.
>>>
>>>> 2. forward and central derivatives,
>>> The derivatives are not computed by the classes provided by commons-math, they
>>> should be provided by the user when extending the SimpleEstimationProblem class
>>> (or implementing directly the EstimationProblem interface if prefered). This
>>> means you can use any method you prefer for computeing them : forward, central
>>> or backward finite differences or analycally computed derivatives.
>>>
>>>> 3. tangent estimates
>>> I'm not sure I understand what you mean here. The solvers are based on gradient
>>> computation at the current point which is updated according to the selected
>>> solver algorithm.
>>>
>>>> 4. quadratic estimates,
>>> I'm also not sure to understand what you mean here. The problem are defined in
>>> terms of observed measurements which are defined at problem start and
>>> theoretical measurements which are automatically recomputed during the search as
>>> the current point moves. The algorithms compute a quadratic cost function as the
>>> sum of the weighted residuals between theoretical measurements and observed
>> I meant sum of weighted SQUARED residuals, of course.
>>
>>
>>> measurements. This cost function is minimized (i.e. parameters are adjusted in
>>> such a way theoretically recomputed measurements match the observations). The
>>> quadratic structure of the cost function is fixed, but the measurements involved
>>> and the weights can be specified by the user.
>>>
>>>> 5. Newton and conjugate search direction
>>> The GaussNewtonEstimator is based on ... the Gauss-Newton algorithm which
>>> iterates over a Newton search.
>>>
>>> The LevenbergMarquardtEstimator is a much more elaborate algorithm which
>>> interpolates between a Gauss-Newton and a gradient method. It is more robust
>>> than Gauss-Newton and highly recommended.
>>>
>>> Commons-math currently does not yet provide a simple conjugate-gradient method.
>>>
>>>
>>>> Does apache commons have these capabilities?
>>> To conclude, commons-math provide some of these capabilities, but not all of
>>> them.
>>>
>>> Luc
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>>
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org