You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Christoph Läubrich <la...@googlemail.com.INVALID> on 2020/08/10 15:09:00 UTC
[math] example for constrain parameters for Least squares
The userguide [1] mentions that it is currently not directly possible to
contrain parameters directly but suggest one can use the
ParameterValidator, is there any example code for both mentioned
alternatives?
For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
archive that the mean is within a closed bound e.g from 5 to 6 where my
datapoints ranges from 0..90, how would this be archived?
I'm especially interested because the FUNCTION inside
GaussianCurveFitter seems to reject invalid values (e.g. negative
valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
either approach described in the user-docs.
[1]
https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Christoph Läubrich <la...@googlemail.com.INVALID>.
LSB = LeasSquaresBuilder :-)
It seems to work with INF in my testings for one exception: If the
starting point is in an invalid range, an exception is thrown (unable to
perform Q.R decomposition) so if using this technique one must take care
that the initial guess is always inside valid bounds!
Am 11.08.20 um 16:11 schrieb Gilles Sadowski:
> Hello.
>
> Le mar. 11 août 2020 à 12:08, Christoph Läubrich
> <la...@googlemail.com.invalid> a écrit :
>>
>> Thanks for your patience, maybe a better/simpler example would be [1], I
>> want to find the best fit using LSB
>
> "LSB" ?
>
>> under the constraint that the height
>> of the curve never is above 0.7 (even though without constrains the
>> optimizer would find a better solution around 8.5).
>
> It occurs to me that this would be more properly defined as a
> least-square problem by assigning an error bar (weight) to each
> data point.
>
>> So my first idea way to copy/extend GausianCurveFitter to accept some
>> kind of "maxHeight(...)", adjust the function to return INF for
>> h>maxHeight, I was just unsure that return INF is allowed (per
>> definition) as it is not mentioned anywhere in the userdocs. And if I
>> want to limit others (like max position) it would work the same way.
>
> Hopefully "infinity" will not cause an issue; you could try and check.
> [Then if it does, just use any suitably large value.]
>
>> In my final problem I need to limit the height, width and position of
>> the bell-curve to fall into certain bounds, but there is no direct
>> relation (e.g. width must be three times the height).
>
> Then, width is totally correlated (to height) and there are only
> 2 parameters to fit (height and mean); hence it's probably better
> (and more efficient) to use that fact instead of defining the
> correlation as a constraint i.e. define (untested):
> ---CUT---
> public class MyFunc implements ParametricUnivariateFunction {
> public double value(double x, double ... param) {
> final double diff = x - param[1]; // param[1] is the mean.
> final double norm = param[0]; // param[0] is the height.
> final double s = 3 * norm; // "constraint".
> final double i2s2 = 1 / (2 * s * s);
> return Gaussian.value(diff, norm, i2s2);
> }
>
> // And similarly for the "gradient":
> // https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/analysis/function/Gaussian.java;h=08dcac0d4d37179bc85a7071c84a6cb289c09f02;hb=HEAD#l143
> }
> ---CUT---
> to be passed to "SimpleCurveFitter".
>
> Regards,
> Gilles
>
>>
>>
>>
>> [1]
>> https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294
>>
>>
>>
>>
>> Am 11.08.20 um 11:18 schrieb Gilles Sadowski:
>>> Hi.
>>>
>>> 2020-08-11 8:51 UTC+02:00, Christoph Läubrich <la...@googlemail.com.invalid>:
>>>> Hi Gilles,
>>>>
>>>>
>>>> Just to make clear I don't suspect any error with GausianCurveFitter, I
>>>> just don't understand how the advice in the user-doc to restrict
>>>> parameter (for general problems) could be applied to a concrete problem
>>>> and thus chosen GausianCurvefitter as an example as it uses
>>>> LeastSquaresBuilder.
>>>>
>>>> I also noticed that Gaussian Fitter has a restriction on parameters
>>>> (norm can't be negative) that is handled in a third way (returning
>>>> Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned
>>>> in the userdoc at all, so I wonder if this is a general purpose solution
>>>> for restricting parameters (seems the simplest approach).
>>>
>>> I'd indeed suggest to first try the same trick as in "GaussianCurveFitter"
>>> (i.e. return a "high" value for arguments outside a known range).
>>> That way, you only have to define a suitable "ParametricUnivariateFunction"
>>> and pass it to "SimpleCurveFitter".
>>>
>>> One case for the "ParameterValidator" is when some of the model
>>> parameters might be correlated to others.
>>> But using it makes it necessary that you handle yourself all the
>>> arguments to be passed to the "LeastSquaresProblem".
>>>
>>>> To take the gausian example for my use case, consider an observed signal
>>>> similar to [1], given I know (from other source as the plain data) for
>>>> example that the result must be found in the range of 2...3 and I wanted
>>>> to restrict valid solutions to this area. The same might apply to the
>>>> norm: I know it must be between a given range and I want to restrict the
>>>> optimizer here even though there might be a solution outside of the
>>>> range that (compared of the R^2) fits better, e.g. a gausian fit well
>>>> inside the -1..1.
>>>>
>>>> I hope it is a little bit clearer.
>>>
>>> I'm not sure. The picture shows a function that is not a Gaussian.
>>> Do you mean that you want to fit only *part* of the data with a
>>> function that would not fit well *all* the data?
>>>
>>> Regards,
>>> Gilles
>>>
>>>>
>>>>
>>>> [1]
>>>> https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg
>>>>
>>>> Am 11.08.20 um 00:42 schrieb Gilles Sadowski:
>>>>> Hello.
>>>>>
>>>>> Le lun. 10 août 2020 à 17:09, Christoph Läubrich
>>>>> <la...@googlemail.com.invalid> a écrit :
>>>>>>
>>>>>> The userguide [1] mentions that it is currently not directly possible to
>>>>>> contrain parameters directly but suggest one can use the
>>>>>> ParameterValidator, is there any example code for both mentioned
>>>>>> alternatives?
>>>>>>
>>>>>> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
>>>>>> archive that the mean is within a closed bound e.g from 5 to 6 where my
>>>>>> datapoints ranges from 0..90, how would this be archived?
>>>>>
>>>>> Could you set up a unit test as a practical example of what
>>>>> you need to achieve?
>>>>>
>>>>>> I'm especially interested because the FUNCTION inside
>>>>>> GaussianCurveFitter seems to reject invalid values (e.g. negative
>>>>>> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
>>>>>> either approach described in the user-docs.
>>>>>
>>>>> What I don't quite get is why you need to force the mean within a
>>>>> certain range; if the data match a Gaussian with a mean within that
>>>>> range, I would assume that the fitter will find the correct value...
>>>>> Sorry if I missed something. Hopefully the example will clarify.
>>>>>
>>>>> Best,
>>>>> Gilles
>>>>>
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Christoph Läubrich <la...@googlemail.com.INVALID>.
Hi Gilles and Randy,
that sounds really interesting I'll try this out, both the sigmodial and
the scaling stuff!
I'm relative new to all this least squares things and I'm not a
mathematician so I need to learn a lot, so don't hesitate to point out
things that might seem obvious to the advanced user, that's really
welcome :-)
Am 08.06.21 um 15:23 schrieb Gilles Sadowski:
> Hello.
>
> Le mar. 8 juin 2021 à 08:14, Christoph Läubrich
> <la...@googlemail.com.invalid> a écrit :
>>
>> Hi Gilles,
>>
>> I have used the the INFINITY approach for a while now and it works quite
>> good. I just recently found a problem where I got very bad fits after a
>> handful of iterations using the LevenbergMarquardtOptimizer.
>>
>> The problem arises whenever there is a relative small range of valid
>> values for *one* parameter.
>>
>> Too keep up with the Gausian example assume that the mean is only valid
>> in a small window, but norm and sigma are completely free.
>>
>> My guess is, if in the data there are outlier that indicates a strong
>> maximum outside this range the optimizer try to go in that 'direction'
>> because I reject this solution it 'gives up' as it seems evident that
>> there is no better solution. This then can result in a gausian that is
>> very thin and a really bad fit (cost e.g about 1E4).
>>
>> If I help the optimizer (e.g. by adjusting the initial guess of sigma)
>> it finds a much better solution (cost about 1E-9).
>>
>> So what I would need to tell the Optimizer (not sure if this is possible
>> at all!) that not the *whole* solution is bad, but only the choice of
>> *one* variable so it could use larger increments for the other variables.
>
> If you want to restrict the range of, say, the mean:
>
> public class MyFunc implements ParametricUnivariateFunction {
> private final Sigmoid meanTransform;
>
> public MyFunc(double minMean, double maxMean) {
> meanTransform = new Sigmoid(minMean, maxMean);
> }
>
> public double value(double x, double ... param) {
> final double mu = meanTransform.value(param[1]); // param[1]
> is the mean.
> final double diff = x - mu;
> final double norm = param[0]; // param[0] is the height.
> final double s = param[2]; // param[2] is the standard deviation.
> final double i2s2 = 1 / (2 * s * s);
> return Gaussian.value(diff, norm, i2s2);
> }
> }
>
> // ...
> final MyFunc f = new MyFunc(min, max);
> final double[] best = fitter.fit(f); // Perform fit.
> final double bestMean = new Logit(min, max).value(best[1]);
>
> HTH,
> Gilles
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Randy Motluck <r_...@yahoo.com.INVALID>.
Hi Gilles,
I may be wrong in understanding your issue, but in general, I believe you should scale your features.
https://towardsdatascience.com/gradient-descent-the-learning-rate-and-the-importance-of-feature-scaling-6c0b416596e1
https://societyofai.medium.com/simplest-way-for-feature-scaling-in-gradient-descent-ae0aaa383039
It becomes important when one feature is of greatly different scale than another. Say for example X is 0-100.0 while Y is 0-1000000000 or something.
Hope this finds you well and best of luck,
Randy Motluck
On Tuesday, June 8, 2021, 08:24:54 AM CDT, Gilles Sadowski <gi...@gmail.com> wrote:
Hello.
Le mar. 8 juin 2021 à 08:14, Christoph Läubrich
<la...@googlemail.com.invalid> a écrit :
>
> Hi Gilles,
>
> I have used the the INFINITY approach for a while now and it works quite
> good. I just recently found a problem where I got very bad fits after a
> handful of iterations using the LevenbergMarquardtOptimizer.
>
> The problem arises whenever there is a relative small range of valid
> values for *one* parameter.
>
> Too keep up with the Gausian example assume that the mean is only valid
> in a small window, but norm and sigma are completely free.
>
> My guess is, if in the data there are outlier that indicates a strong
> maximum outside this range the optimizer try to go in that 'direction'
> because I reject this solution it 'gives up' as it seems evident that
> there is no better solution. This then can result in a gausian that is
> very thin and a really bad fit (cost e.g about 1E4).
>
> If I help the optimizer (e.g. by adjusting the initial guess of sigma)
> it finds a much better solution (cost about 1E-9).
>
> So what I would need to tell the Optimizer (not sure if this is possible
> at all!) that not the *whole* solution is bad, but only the choice of
> *one* variable so it could use larger increments for the other variables.
If you want to restrict the range of, say, the mean:
public class MyFunc implements ParametricUnivariateFunction {
private final Sigmoid meanTransform;
public MyFunc(double minMean, double maxMean) {
meanTransform = new Sigmoid(minMean, maxMean);
}
public double value(double x, double ... param) {
final double mu = meanTransform.value(param[1]); // param[1]
is the mean.
final double diff = x - mu;
final double norm = param[0]; // param[0] is the height.
final double s = param[2]; // param[2] is the standard deviation.
final double i2s2 = 1 / (2 * s * s);
return Gaussian.value(diff, norm, i2s2);
}
}
// ...
final MyFunc f = new MyFunc(min, max);
final double[] best = fitter.fit(f); // Perform fit.
final double bestMean = new Logit(min, max).value(best[1]);
HTH,
Gilles
>>>> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Gilles Sadowski <gi...@gmail.com>.
Hello.
Le mar. 8 juin 2021 à 08:14, Christoph Läubrich
<la...@googlemail.com.invalid> a écrit :
>
> Hi Gilles,
>
> I have used the the INFINITY approach for a while now and it works quite
> good. I just recently found a problem where I got very bad fits after a
> handful of iterations using the LevenbergMarquardtOptimizer.
>
> The problem arises whenever there is a relative small range of valid
> values for *one* parameter.
>
> Too keep up with the Gausian example assume that the mean is only valid
> in a small window, but norm and sigma are completely free.
>
> My guess is, if in the data there are outlier that indicates a strong
> maximum outside this range the optimizer try to go in that 'direction'
> because I reject this solution it 'gives up' as it seems evident that
> there is no better solution. This then can result in a gausian that is
> very thin and a really bad fit (cost e.g about 1E4).
>
> If I help the optimizer (e.g. by adjusting the initial guess of sigma)
> it finds a much better solution (cost about 1E-9).
>
> So what I would need to tell the Optimizer (not sure if this is possible
> at all!) that not the *whole* solution is bad, but only the choice of
> *one* variable so it could use larger increments for the other variables.
If you want to restrict the range of, say, the mean:
public class MyFunc implements ParametricUnivariateFunction {
private final Sigmoid meanTransform;
public MyFunc(double minMean, double maxMean) {
meanTransform = new Sigmoid(minMean, maxMean);
}
public double value(double x, double ... param) {
final double mu = meanTransform.value(param[1]); // param[1]
is the mean.
final double diff = x - mu;
final double norm = param[0]; // param[0] is the height.
final double s = param[2]; // param[2] is the standard deviation.
final double i2s2 = 1 / (2 * s * s);
return Gaussian.value(diff, norm, i2s2);
}
}
// ...
final MyFunc f = new MyFunc(min, max);
final double[] best = fitter.fit(f); // Perform fit.
final double bestMean = new Logit(min, max).value(best[1]);
HTH,
Gilles
>>>> [...]
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Christoph Läubrich <la...@googlemail.com.INVALID>.
Hi Gilles,
I have used the the INFINITY approach for a while now and it works quite
good. I just recently found a problem where I got very bad fits after a
handful of iterations using the LevenbergMarquardtOptimizer.
The problem arises whenever there is a relative small range of valid
values for *one* parameter.
Too keep up with the Gausian example assume that the mean is only valid
in a small window, but norm and sigma are completely free.
My guess is, if in the data there are outlier that indicates a strong
maximum outside this range the optimizer try to go in that 'direction'
because I reject this solution it 'gives up' as it seems evident that
there is no better solution. This then can result in a gausian that is
very thin and a really bad fit (cost e.g about 1E4).
If I help the optimizer (e.g. by adjusting the initial guess of sigma)
it finds a much better solution (cost about 1E-9).
So what I would need to tell the Optimizer (not sure if this is possible
at all!) that not the *whole* solution is bad, but only the choice of
*one* variable so it could use larger increments for the other variables.
Am 11.08.20 um 16:11 schrieb Gilles Sadowski:
> Hello.
>
> Le mar. 11 août 2020 à 12:08, Christoph Läubrich
> <la...@googlemail.com.invalid> a écrit :
>>
>> Thanks for your patience, maybe a better/simpler example would be [1], I
>> want to find the best fit using LSB
>
> "LSB" ?
>
>> under the constraint that the height
>> of the curve never is above 0.7 (even though without constrains the
>> optimizer would find a better solution around 8.5).
>
> It occurs to me that this would be more properly defined as a
> least-square problem by assigning an error bar (weight) to each
> data point.
>
>> So my first idea way to copy/extend GausianCurveFitter to accept some
>> kind of "maxHeight(...)", adjust the function to return INF for
>> h>maxHeight, I was just unsure that return INF is allowed (per
>> definition) as it is not mentioned anywhere in the userdocs. And if I
>> want to limit others (like max position) it would work the same way.
>
> Hopefully "infinity" will not cause an issue; you could try and check.
> [Then if it does, just use any suitably large value.]
>
>> In my final problem I need to limit the height, width and position of
>> the bell-curve to fall into certain bounds, but there is no direct
>> relation (e.g. width must be three times the height).
>
> Then, width is totally correlated (to height) and there are only
> 2 parameters to fit (height and mean); hence it's probably better
> (and more efficient) to use that fact instead of defining the
> correlation as a constraint i.e. define (untested):
> ---CUT---
> public class MyFunc implements ParametricUnivariateFunction {
> public double value(double x, double ... param) {
> final double diff = x - param[1]; // param[1] is the mean.
> final double norm = param[0]; // param[0] is the height.
> final double s = 3 * norm; // "constraint".
> final double i2s2 = 1 / (2 * s * s);
> return Gaussian.value(diff, norm, i2s2);
> }
>
> // And similarly for the "gradient":
> // https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/analysis/function/Gaussian.java;h=08dcac0d4d37179bc85a7071c84a6cb289c09f02;hb=HEAD#l143
> }
> ---CUT---
> to be passed to "SimpleCurveFitter".
>
> Regards,
> Gilles
>
>>
>>
>>
>> [1]
>> https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294
>>
>>
>>
>>
>> Am 11.08.20 um 11:18 schrieb Gilles Sadowski:
>>> Hi.
>>>
>>> 2020-08-11 8:51 UTC+02:00, Christoph Läubrich <la...@googlemail.com.invalid>:
>>>> Hi Gilles,
>>>>
>>>>
>>>> Just to make clear I don't suspect any error with GausianCurveFitter, I
>>>> just don't understand how the advice in the user-doc to restrict
>>>> parameter (for general problems) could be applied to a concrete problem
>>>> and thus chosen GausianCurvefitter as an example as it uses
>>>> LeastSquaresBuilder.
>>>>
>>>> I also noticed that Gaussian Fitter has a restriction on parameters
>>>> (norm can't be negative) that is handled in a third way (returning
>>>> Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned
>>>> in the userdoc at all, so I wonder if this is a general purpose solution
>>>> for restricting parameters (seems the simplest approach).
>>>
>>> I'd indeed suggest to first try the same trick as in "GaussianCurveFitter"
>>> (i.e. return a "high" value for arguments outside a known range).
>>> That way, you only have to define a suitable "ParametricUnivariateFunction"
>>> and pass it to "SimpleCurveFitter".
>>>
>>> One case for the "ParameterValidator" is when some of the model
>>> parameters might be correlated to others.
>>> But using it makes it necessary that you handle yourself all the
>>> arguments to be passed to the "LeastSquaresProblem".
>>>
>>>> To take the gausian example for my use case, consider an observed signal
>>>> similar to [1], given I know (from other source as the plain data) for
>>>> example that the result must be found in the range of 2...3 and I wanted
>>>> to restrict valid solutions to this area. The same might apply to the
>>>> norm: I know it must be between a given range and I want to restrict the
>>>> optimizer here even though there might be a solution outside of the
>>>> range that (compared of the R^2) fits better, e.g. a gausian fit well
>>>> inside the -1..1.
>>>>
>>>> I hope it is a little bit clearer.
>>>
>>> I'm not sure. The picture shows a function that is not a Gaussian.
>>> Do you mean that you want to fit only *part* of the data with a
>>> function that would not fit well *all* the data?
>>>
>>> Regards,
>>> Gilles
>>>
>>>>
>>>>
>>>> [1]
>>>> https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg
>>>>
>>>> Am 11.08.20 um 00:42 schrieb Gilles Sadowski:
>>>>> Hello.
>>>>>
>>>>> Le lun. 10 août 2020 à 17:09, Christoph Läubrich
>>>>> <la...@googlemail.com.invalid> a écrit :
>>>>>>
>>>>>> The userguide [1] mentions that it is currently not directly possible to
>>>>>> contrain parameters directly but suggest one can use the
>>>>>> ParameterValidator, is there any example code for both mentioned
>>>>>> alternatives?
>>>>>>
>>>>>> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
>>>>>> archive that the mean is within a closed bound e.g from 5 to 6 where my
>>>>>> datapoints ranges from 0..90, how would this be archived?
>>>>>
>>>>> Could you set up a unit test as a practical example of what
>>>>> you need to achieve?
>>>>>
>>>>>> I'm especially interested because the FUNCTION inside
>>>>>> GaussianCurveFitter seems to reject invalid values (e.g. negative
>>>>>> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
>>>>>> either approach described in the user-docs.
>>>>>
>>>>> What I don't quite get is why you need to force the mean within a
>>>>> certain range; if the data match a Gaussian with a mean within that
>>>>> range, I would assume that the fitter will find the correct value...
>>>>> Sorry if I missed something. Hopefully the example will clarify.
>>>>>
>>>>> Best,
>>>>> Gilles
>>>>>
>>>>>>
>>>>>>
>>>>>> [1]
>>>>>> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Gilles Sadowski <gi...@gmail.com>.
Hello.
Le mar. 11 août 2020 à 12:08, Christoph Läubrich
<la...@googlemail.com.invalid> a écrit :
>
> Thanks for your patience, maybe a better/simpler example would be [1], I
> want to find the best fit using LSB
"LSB" ?
> under the constraint that the height
> of the curve never is above 0.7 (even though without constrains the
> optimizer would find a better solution around 8.5).
It occurs to me that this would be more properly defined as a
least-square problem by assigning an error bar (weight) to each
data point.
> So my first idea way to copy/extend GausianCurveFitter to accept some
> kind of "maxHeight(...)", adjust the function to return INF for
> h>maxHeight, I was just unsure that return INF is allowed (per
> definition) as it is not mentioned anywhere in the userdocs. And if I
> want to limit others (like max position) it would work the same way.
Hopefully "infinity" will not cause an issue; you could try and check.
[Then if it does, just use any suitably large value.]
> In my final problem I need to limit the height, width and position of
> the bell-curve to fall into certain bounds, but there is no direct
> relation (e.g. width must be three times the height).
Then, width is totally correlated (to height) and there are only
2 parameters to fit (height and mean); hence it's probably better
(and more efficient) to use that fact instead of defining the
correlation as a constraint i.e. define (untested):
---CUT---
public class MyFunc implements ParametricUnivariateFunction {
public double value(double x, double ... param) {
final double diff = x - param[1]; // param[1] is the mean.
final double norm = param[0]; // param[0] is the height.
final double s = 3 * norm; // "constraint".
final double i2s2 = 1 / (2 * s * s);
return Gaussian.value(diff, norm, i2s2);
}
// And similarly for the "gradient":
// https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/main/java/org/apache/commons/math4/analysis/function/Gaussian.java;h=08dcac0d4d37179bc85a7071c84a6cb289c09f02;hb=HEAD#l143
}
---CUT---
to be passed to "SimpleCurveFitter".
Regards,
Gilles
>
>
>
> [1]
> https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294
>
>
>
>
> Am 11.08.20 um 11:18 schrieb Gilles Sadowski:
> > Hi.
> >
> > 2020-08-11 8:51 UTC+02:00, Christoph Läubrich <la...@googlemail.com.invalid>:
> >> Hi Gilles,
> >>
> >>
> >> Just to make clear I don't suspect any error with GausianCurveFitter, I
> >> just don't understand how the advice in the user-doc to restrict
> >> parameter (for general problems) could be applied to a concrete problem
> >> and thus chosen GausianCurvefitter as an example as it uses
> >> LeastSquaresBuilder.
> >>
> >> I also noticed that Gaussian Fitter has a restriction on parameters
> >> (norm can't be negative) that is handled in a third way (returning
> >> Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned
> >> in the userdoc at all, so I wonder if this is a general purpose solution
> >> for restricting parameters (seems the simplest approach).
> >
> > I'd indeed suggest to first try the same trick as in "GaussianCurveFitter"
> > (i.e. return a "high" value for arguments outside a known range).
> > That way, you only have to define a suitable "ParametricUnivariateFunction"
> > and pass it to "SimpleCurveFitter".
> >
> > One case for the "ParameterValidator" is when some of the model
> > parameters might be correlated to others.
> > But using it makes it necessary that you handle yourself all the
> > arguments to be passed to the "LeastSquaresProblem".
> >
> >> To take the gausian example for my use case, consider an observed signal
> >> similar to [1], given I know (from other source as the plain data) for
> >> example that the result must be found in the range of 2...3 and I wanted
> >> to restrict valid solutions to this area. The same might apply to the
> >> norm: I know it must be between a given range and I want to restrict the
> >> optimizer here even though there might be a solution outside of the
> >> range that (compared of the R^2) fits better, e.g. a gausian fit well
> >> inside the -1..1.
> >>
> >> I hope it is a little bit clearer.
> >
> > I'm not sure. The picture shows a function that is not a Gaussian.
> > Do you mean that you want to fit only *part* of the data with a
> > function that would not fit well *all* the data?
> >
> > Regards,
> > Gilles
> >
> >>
> >>
> >> [1]
> >> https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg
> >>
> >> Am 11.08.20 um 00:42 schrieb Gilles Sadowski:
> >>> Hello.
> >>>
> >>> Le lun. 10 août 2020 à 17:09, Christoph Läubrich
> >>> <la...@googlemail.com.invalid> a écrit :
> >>>>
> >>>> The userguide [1] mentions that it is currently not directly possible to
> >>>> contrain parameters directly but suggest one can use the
> >>>> ParameterValidator, is there any example code for both mentioned
> >>>> alternatives?
> >>>>
> >>>> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
> >>>> archive that the mean is within a closed bound e.g from 5 to 6 where my
> >>>> datapoints ranges from 0..90, how would this be archived?
> >>>
> >>> Could you set up a unit test as a practical example of what
> >>> you need to achieve?
> >>>
> >>>> I'm especially interested because the FUNCTION inside
> >>>> GaussianCurveFitter seems to reject invalid values (e.g. negative
> >>>> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
> >>>> either approach described in the user-docs.
> >>>
> >>> What I don't quite get is why you need to force the mean within a
> >>> certain range; if the data match a Gaussian with a mean within that
> >>> range, I would assume that the fitter will find the correct value...
> >>> Sorry if I missed something. Hopefully the example will clarify.
> >>>
> >>> Best,
> >>> Gilles
> >>>
> >>>>
> >>>>
> >>>> [1]
> >>>> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Christoph Läubrich <la...@googlemail.com.INVALID>.
Thanks for your patience, maybe a better/simpler example would be [1], I
want to find the best fit using LSB under the constraint that the height
of the curve never is above 0.7 (even though without constrains the
optimizer would find a better solution around 8.5).
So my first idea way to copy/extend GausianCurveFitter to accept some
kind of "maxHeight(...)", adjust the function to return INF for
h>maxHeight, I was just unsure that return INF is allowed (per
definition) as it is not mentioned anywhere in the userdocs. And if I
want to limit others (like max position) it would work the same way.
In my final problem I need to limit the height, width and position of
the bell-curve to fall into certain bounds, but there is no direct
relation (e.g. width must be three times the height).
[1]
https://crlbucophysics101.files.wordpress.com/2015/02/gaussian.png?w=538&h=294
Am 11.08.20 um 11:18 schrieb Gilles Sadowski:
> Hi.
>
> 2020-08-11 8:51 UTC+02:00, Christoph Läubrich <la...@googlemail.com.invalid>:
>> Hi Gilles,
>>
>>
>> Just to make clear I don't suspect any error with GausianCurveFitter, I
>> just don't understand how the advice in the user-doc to restrict
>> parameter (for general problems) could be applied to a concrete problem
>> and thus chosen GausianCurvefitter as an example as it uses
>> LeastSquaresBuilder.
>>
>> I also noticed that Gaussian Fitter has a restriction on parameters
>> (norm can't be negative) that is handled in a third way (returning
>> Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned
>> in the userdoc at all, so I wonder if this is a general purpose solution
>> for restricting parameters (seems the simplest approach).
>
> I'd indeed suggest to first try the same trick as in "GaussianCurveFitter"
> (i.e. return a "high" value for arguments outside a known range).
> That way, you only have to define a suitable "ParametricUnivariateFunction"
> and pass it to "SimpleCurveFitter".
>
> One case for the "ParameterValidator" is when some of the model
> parameters might be correlated to others.
> But using it makes it necessary that you handle yourself all the
> arguments to be passed to the "LeastSquaresProblem".
>
>> To take the gausian example for my use case, consider an observed signal
>> similar to [1], given I know (from other source as the plain data) for
>> example that the result must be found in the range of 2...3 and I wanted
>> to restrict valid solutions to this area. The same might apply to the
>> norm: I know it must be between a given range and I want to restrict the
>> optimizer here even though there might be a solution outside of the
>> range that (compared of the R^2) fits better, e.g. a gausian fit well
>> inside the -1..1.
>>
>> I hope it is a little bit clearer.
>
> I'm not sure. The picture shows a function that is not a Gaussian.
> Do you mean that you want to fit only *part* of the data with a
> function that would not fit well *all* the data?
>
> Regards,
> Gilles
>
>>
>>
>> [1]
>> https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg
>>
>> Am 11.08.20 um 00:42 schrieb Gilles Sadowski:
>>> Hello.
>>>
>>> Le lun. 10 août 2020 à 17:09, Christoph Läubrich
>>> <la...@googlemail.com.invalid> a écrit :
>>>>
>>>> The userguide [1] mentions that it is currently not directly possible to
>>>> contrain parameters directly but suggest one can use the
>>>> ParameterValidator, is there any example code for both mentioned
>>>> alternatives?
>>>>
>>>> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
>>>> archive that the mean is within a closed bound e.g from 5 to 6 where my
>>>> datapoints ranges from 0..90, how would this be archived?
>>>
>>> Could you set up a unit test as a practical example of what
>>> you need to achieve?
>>>
>>>> I'm especially interested because the FUNCTION inside
>>>> GaussianCurveFitter seems to reject invalid values (e.g. negative
>>>> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
>>>> either approach described in the user-docs.
>>>
>>> What I don't quite get is why you need to force the mean within a
>>> certain range; if the data match a Gaussian with a mean within that
>>> range, I would assume that the fitter will find the correct value...
>>> Sorry if I missed something. Hopefully the example will clarify.
>>>
>>> Best,
>>> Gilles
>>>
>>>>
>>>>
>>>> [1]
>>>> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
>>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Gilles Sadowski <gi...@gmail.com>.
Hi.
2020-08-11 8:51 UTC+02:00, Christoph Läubrich <la...@googlemail.com.invalid>:
> Hi Gilles,
>
>
> Just to make clear I don't suspect any error with GausianCurveFitter, I
> just don't understand how the advice in the user-doc to restrict
> parameter (for general problems) could be applied to a concrete problem
> and thus chosen GausianCurvefitter as an example as it uses
> LeastSquaresBuilder.
>
> I also noticed that Gaussian Fitter has a restriction on parameters
> (norm can't be negative) that is handled in a third way (returning
> Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned
> in the userdoc at all, so I wonder if this is a general purpose solution
> for restricting parameters (seems the simplest approach).
I'd indeed suggest to first try the same trick as in "GaussianCurveFitter"
(i.e. return a "high" value for arguments outside a known range).
That way, you only have to define a suitable "ParametricUnivariateFunction"
and pass it to "SimpleCurveFitter".
One case for the "ParameterValidator" is when some of the model
parameters might be correlated to others.
But using it makes it necessary that you handle yourself all the
arguments to be passed to the "LeastSquaresProblem".
> To take the gausian example for my use case, consider an observed signal
> similar to [1], given I know (from other source as the plain data) for
> example that the result must be found in the range of 2...3 and I wanted
> to restrict valid solutions to this area. The same might apply to the
> norm: I know it must be between a given range and I want to restrict the
> optimizer here even though there might be a solution outside of the
> range that (compared of the R^2) fits better, e.g. a gausian fit well
> inside the -1..1.
>
> I hope it is a little bit clearer.
I'm not sure. The picture shows a function that is not a Gaussian.
Do you mean that you want to fit only *part* of the data with a
function that would not fit well *all* the data?
Regards,
Gilles
>
>
> [1]
> https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg
>
> Am 11.08.20 um 00:42 schrieb Gilles Sadowski:
>> Hello.
>>
>> Le lun. 10 août 2020 à 17:09, Christoph Läubrich
>> <la...@googlemail.com.invalid> a écrit :
>>>
>>> The userguide [1] mentions that it is currently not directly possible to
>>> contrain parameters directly but suggest one can use the
>>> ParameterValidator, is there any example code for both mentioned
>>> alternatives?
>>>
>>> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
>>> archive that the mean is within a closed bound e.g from 5 to 6 where my
>>> datapoints ranges from 0..90, how would this be archived?
>>
>> Could you set up a unit test as a practical example of what
>> you need to achieve?
>>
>>> I'm especially interested because the FUNCTION inside
>>> GaussianCurveFitter seems to reject invalid values (e.g. negative
>>> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
>>> either approach described in the user-docs.
>>
>> What I don't quite get is why you need to force the mean within a
>> certain range; if the data match a Gaussian with a mean within that
>> range, I would assume that the fitter will find the correct value...
>> Sorry if I missed something. Hopefully the example will clarify.
>>
>> Best,
>> Gilles
>>
>>>
>>>
>>> [1]
>>> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Christoph Läubrich <la...@googlemail.com.INVALID>.
Hi Gilles,
Just to make clear I don't suspect any error with GausianCurveFitter, I
just don't understand how the advice in the user-doc to restrict
parameter (for general problems) could be applied to a concrete problem
and thus chosen GausianCurvefitter as an example as it uses
LeastSquaresBuilder.
I also noticed that Gaussian Fitter has a restriction on parameters
(norm can't be negative) that is handled in a third way (returning
Double.POSITIVE_INFINITY instead of Parameter Validator) not mentioned
in the userdoc at all, so I wonder if this is a general purpose solution
for restricting parameters (seems the simplest approach).
To take the gausian example for my use case, consider an observed signal
similar to [1], given I know (from other source as the plain data) for
example that the result must be found in the range of 2...3 and I wanted
to restrict valid solutions to this area. The same might apply to the
norm: I know it must be between a given range and I want to restrict the
optimizer here even though there might be a solution outside of the
range that (compared of the R^2) fits better, e.g. a gausian fit well
inside the -1..1.
I hope it is a little bit clearer.
[1]
https://ascelibrary.org/cms/asset/6ca2b016-1a4f-4eed-80da-71219777cac1/1.jpg
Am 11.08.20 um 00:42 schrieb Gilles Sadowski:
> Hello.
>
> Le lun. 10 août 2020 à 17:09, Christoph Läubrich
> <la...@googlemail.com.invalid> a écrit :
>>
>> The userguide [1] mentions that it is currently not directly possible to
>> contrain parameters directly but suggest one can use the
>> ParameterValidator, is there any example code for both mentioned
>> alternatives?
>>
>> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
>> archive that the mean is within a closed bound e.g from 5 to 6 where my
>> datapoints ranges from 0..90, how would this be archived?
>
> Could you set up a unit test as a practical example of what
> you need to achieve?
>
>> I'm especially interested because the FUNCTION inside
>> GaussianCurveFitter seems to reject invalid values (e.g. negative
>> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
>> either approach described in the user-docs.
>
> What I don't quite get is why you need to force the mean within a
> certain range; if the data match a Gaussian with a mean within that
> range, I would assume that the fitter will find the correct value...
> Sorry if I missed something. Hopefully the example will clarify.
>
> Best,
> Gilles
>
>>
>>
>> [1]
>> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] example for constrain parameters for Least squares
Posted by Gilles Sadowski <gi...@gmail.com>.
Hello.
Le lun. 10 août 2020 à 17:09, Christoph Läubrich
<la...@googlemail.com.invalid> a écrit :
>
> The userguide [1] mentions that it is currently not directly possible to
> contrain parameters directly but suggest one can use the
> ParameterValidator, is there any example code for both mentioned
> alternatives?
>
> For example GaussianCurveFitter uses LeastSquaresBuilder and I wan't to
> archive that the mean is within a closed bound e.g from 5 to 6 where my
> datapoints ranges from 0..90, how would this be archived?
Could you set up a unit test as a practical example of what
you need to achieve?
> I'm especially interested because the FUNCTION inside
> GaussianCurveFitter seems to reject invalid values (e.g. negative
> valuenorm) by simply return Double.POSITIVE_INFINITY instead of using
> either approach described in the user-docs.
What I don't quite get is why you need to force the mean within a
certain range; if the data match a Gaussian with a mean within that
range, I would assume that the fitter will find the correct value...
Sorry if I missed something. Hopefully the example will clarify.
Best,
Gilles
>
>
> [1]
> https://commons.apache.org/proper/commons-math/userguide/leastsquares.html
>
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org