You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Andreas Niekler <an...@fbm.htwk-leipzig.de> on 2012/05/14 10:02:29 UTC
[math] Usage of DifferentiableMultivariateRealFunction
Hello,
i'm currently developing a gaussian process implementation for my PhD
thesis. Within this package i need to calculate and maximize the
marginal likelihood of the model given some data. This marginal
likelihood is dependent on the parameters of a covariance function which
is used. For a certain covariance function one can calculate the
derivitives and the gradients for the marginal likelihood to optimize
this function.
My Question: Can someone show me an example for the usage of
DifferentiableMultivariateRealFunction and a matching optimizer. I
understand the concept but the implementational details are hard to
find. Here is what i understood:
value: the value of the function.
gradient(): an array containing all partial derivitivs gradients for the
parameters.
partialDerivitive: Value containing only one gradient of the function
if i provide this basic implementation i should pass this to the
optimizer. Or do i have to provide an extra target function to the
optimizer as inicated by some optinmizers with the <FUNC> operator?
Thnak you
--
Dipl.-Ing. (FH) Andreas Niekler
Mitarbeiter und Promovend
Bereich Multimedia-Produktionssysteme und -technologien
Hochschule für Technik, Wirtschaft und Kultur Leipzig
Fachbereich Medien
Besucher
Gustav-Freytag-Straße 40
04277 Leipzig
Telefon: +49 0341 30 76 2378
Email: aniekler@fbm.htwk-leipzig.de
http://www.fbm.htwk-leipzig.de
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Mon, May 14, 2012 at 11:14:59AM +0200, Andreas Niekler wrote:
> Hello,
>
> thanks for the reply. But i wonder what is the input for value and
> gradient. in DifferentiableMultivariateRealFunction this needs to be
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Which version of Commons Math do you use?
The latest one is 3.0; it is more likely that we can help you if we talk
about the same code.
> a double array but what needs to be provided there? The parameters
> for the function to optimize?
Please give code examples; then, name the arguments which you have trouble
understanding. Otherwise, we spend a lot of time trying to guess what your
question is.
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Andreas Niekler <an...@fbm.htwk-leipzig.de>.
Thank you all very much.
I have updated my DifferentiableMultivariateRealFunction and this is
what it looks like now (appended). I use this and stick it into a
MultiStartDifferentiableMultivariateRealOptimizer. Unfortunately this
procedure end with the creation of some negative parameters (impossible
by method) which i cannot oversee right now (wrong gradients, wrong
procedure?). Here is the procedure i'm using. Hope this clarifies my
task and maybe to get some more hints to optimize my usage of commons Math:
parameter[2] = 1.0;
parameter[1] = 1.0;
parameter[0] = 1.0;
NonLinearConjugateGradientOptimizer underlying = new
NonLinearConjugateGradientOptimizer(ConjugateGradientFormula.FLETCHER_REEVES);
JDKRandomGenerator g = new JDKRandomGenerator();
g.setSeed(753289573253l);
RandomVectorGenerator generator =
new UncorrelatedRandomVectorGenerator(3, new GaussianRandomGenerator(g));
MultiStartDifferentiableMultivariateRealOptimizer optimizer = new
MultiStartDifferentiableMultivariateRealOptimizer(underlying, 10,
generator);
GaussianProcessRegressionMarginalLikelihood gprml = new
GaussianProcessRegressionMarginalLikelihood(input, X);
RealPointValuePair pair = null;
try {
pair = optimizer.optimize(gprml, GoalType.MAXIMIZE, parameter);
} catch (OptimizationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (FunctionEvaluationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Am 14.05.2012 15:42, schrieb Luc Maisonobe:
> Le 14/05/2012 14:11, Andreas Niekler a �crit :
>> Hello,
>
> Hi Andreas,
>
>>
>> after reading a lot through the tutorial this is the code that i came up
>> with regarding the implementation of a gaussian process regression
>> optimisation (File appended):
>>
>> initCovarianceAndGradients(): initialisation of matrices and
>> calculations which are needed by both marginal likelihood calculation
>> and gradient calculation:
>>
>> Within this function i calculate some things globally which are strongly
>> reused by the value() and gradient() functions. What i do not really
>> understand is the passing of the double[] argument to the value()
>> function and the value() function of the gradient() method. Are those
>> methods called by the optimizer with the updated parameters? If this is
>> the case i have to recalculate the global calculations with each call to
>> the value() and gradient() methods.
>
> Yes, the double[] argument is updated at each call and correspond to the
> current estimate as the algorithm iterates towards the solution.
>
> You cannot even rely on the calls being always scheduled in the same
> way. As an example, the Gauss-Newton optimizer performs the two calls
> with function first and gradient afterwards at each iteration, but the
> Levenberg-Marquardt optimizer has two embedded loops and computes
> Jacobians on the external loop and the function value on the internal
> loop. So you should probably not compute everything beforehand in the
> hope it will be used later on.
>
> Luc
>
>
>>
>> Thanks for clarification
>>
>> Am 14.05.2012 12:53, schrieb Gilles Sadowski:
>>> Hello.
>>>
>>>>>
>>>>> thanks for the reply. But i wonder what is the input for value and
>>>>> gradient.
>>>>> in DifferentiableMultivariateRealFunction this needs to be a double
>>>>> array
>>>>> but what needs to be provided there? The parameters for the function to
>>>>> optimize?
>>>>>
>>>>> Thank you very much again
>>>>>
>>>>> Andreas
>>>>>
>>>> Do please have a look to the examples, as your question (and my
>>>> answer) is too vague if not supported by proper code. I guess the
>>>> answer to your question is 'yes', the double[] array is indeed the set
>>>> of parameters, but again, do check the examples, I would not like to
>>>> be misguiding you. Besides the user guide which should provide you
>>>> with the answer, have a look to this implementation [1], line 153. In
>>>> this implementation, x[i] and y[i] are the data points, yhat[i] are
>>>> the model predictions, and a[] are the parameters. You should be able
>>>> to find your way with this example.
>>>
>>> I've also just added another bit of code show-casing the usage of the
>>> "non-linear least-squares" optimizers (svn revision 1338144).
>>>
>>>
>>> Best regards,
>>> Gilles
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: user-help@commons.apache.org
>>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
--
Dipl.-Ing. (FH) Andreas Niekler
Mitarbeiter und Promovend
Bereich Multimedia-Produktionssysteme und -technologien
Hochschule f�r Technik, Wirtschaft und Kultur Leipzig
Fachbereich Medien
Besucher
Gustav-Freytag-Stra�e 40
04277 Leipzig
Telefon: +49 0341 30 76 2378
Email: aniekler@fbm.htwk-leipzig.de
http://www.fbm.htwk-leipzig.de
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Luc Maisonobe <lu...@spaceroots.org>.
Le 14/05/2012 14:11, Andreas Niekler a écrit :
> Hello,
Hi Andreas,
>
> after reading a lot through the tutorial this is the code that i came up
> with regarding the implementation of a gaussian process regression
> optimisation (File appended):
>
> initCovarianceAndGradients(): initialisation of matrices and
> calculations which are needed by both marginal likelihood calculation
> and gradient calculation:
>
> Within this function i calculate some things globally which are strongly
> reused by the value() and gradient() functions. What i do not really
> understand is the passing of the double[] argument to the value()
> function and the value() function of the gradient() method. Are those
> methods called by the optimizer with the updated parameters? If this is
> the case i have to recalculate the global calculations with each call to
> the value() and gradient() methods.
Yes, the double[] argument is updated at each call and correspond to the
current estimate as the algorithm iterates towards the solution.
You cannot even rely on the calls being always scheduled in the same
way. As an example, the Gauss-Newton optimizer performs the two calls
with function first and gradient afterwards at each iteration, but the
Levenberg-Marquardt optimizer has two embedded loops and computes
Jacobians on the external loop and the function value on the internal
loop. So you should probably not compute everything beforehand in the
hope it will be used later on.
Luc
>
> Thanks for clarification
>
> Am 14.05.2012 12:53, schrieb Gilles Sadowski:
>> Hello.
>>
>>>>
>>>> thanks for the reply. But i wonder what is the input for value and
>>>> gradient.
>>>> in DifferentiableMultivariateRealFunction this needs to be a double
>>>> array
>>>> but what needs to be provided there? The parameters for the function to
>>>> optimize?
>>>>
>>>> Thank you very much again
>>>>
>>>> Andreas
>>>>
>>> Do please have a look to the examples, as your question (and my
>>> answer) is too vague if not supported by proper code. I guess the
>>> answer to your question is 'yes', the double[] array is indeed the set
>>> of parameters, but again, do check the examples, I would not like to
>>> be misguiding you. Besides the user guide which should provide you
>>> with the answer, have a look to this implementation [1], line 153. In
>>> this implementation, x[i] and y[i] are the data points, yhat[i] are
>>> the model predictions, and a[] are the parameters. You should be able
>>> to find your way with this example.
>>
>> I've also just added another bit of code show-casing the usage of the
>> "non-linear least-squares" optimizers (svn revision 1338144).
>>
>>
>> Best regards,
>> Gilles
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
>> For additional commands, e-mail: user-help@commons.apache.org
>>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Andreas Niekler <an...@fbm.htwk-leipzig.de>.
Hello,
after reading a lot through the tutorial this is the code that i came up
with regarding the implementation of a gaussian process regression
optimisation (File appended):
initCovarianceAndGradients(): initialisation of matrices and
calculations which are needed by both marginal likelihood calculation
and gradient calculation:
Within this function i calculate some things globally which are strongly
reused by the value() and gradient() functions. What i do not really
understand is the passing of the double[] argument to the value()
function and the value() function of the gradient() method. Are those
methods called by the optimizer with the updated parameters? If this is
the case i have to recalculate the global calculations with each call to
the value() and gradient() methods.
Thanks for clarification
Am 14.05.2012 12:53, schrieb Gilles Sadowski:
> Hello.
>
>>>
>>> thanks for the reply. But i wonder what is the input for value and gradient.
>>> in DifferentiableMultivariateRealFunction this needs to be a double array
>>> but what needs to be provided there? The parameters for the function to
>>> optimize?
>>>
>>> Thank you very much again
>>>
>>> Andreas
>>>
>> Do please have a look to the examples, as your question (and my
>> answer) is too vague if not supported by proper code. I guess the
>> answer to your question is 'yes', the double[] array is indeed the set
>> of parameters, but again, do check the examples, I would not like to
>> be misguiding you. Besides the user guide which should provide you
>> with the answer, have a look to this implementation [1], line 153. In
>> this implementation, x[i] and y[i] are the data points, yhat[i] are
>> the model predictions, and a[] are the parameters. You should be able
>> to find your way with this example.
>
> I've also just added another bit of code show-casing the usage of the
> "non-linear least-squares" optimizers (svn revision 1338144).
>
>
> Best regards,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
--
Dipl.-Ing. (FH) Andreas Niekler
Mitarbeiter und Promovend
Bereich Multimedia-Produktionssysteme und -technologien
Hochschule f�r Technik, Wirtschaft und Kultur Leipzig
Fachbereich Medien
Besucher
Gustav-Freytag-Stra�e 40
04277 Leipzig
Telefon: +49 0341 30 76 2378
Email: aniekler@fbm.htwk-leipzig.de
http://www.fbm.htwk-leipzig.de
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
Hello.
> >
> > thanks for the reply. But i wonder what is the input for value and gradient.
> > in DifferentiableMultivariateRealFunction this needs to be a double array
> > but what needs to be provided there? The parameters for the function to
> > optimize?
> >
> > Thank you very much again
> >
> > Andreas
> >
> Do please have a look to the examples, as your question (and my
> answer) is too vague if not supported by proper code. I guess the
> answer to your question is 'yes', the double[] array is indeed the set
> of parameters, but again, do check the examples, I would not like to
> be misguiding you. Besides the user guide which should provide you
> with the answer, have a look to this implementation [1], line 153. In
> this implementation, x[i] and y[i] are the data points, yhat[i] are
> the model predictions, and a[] are the parameters. You should be able
> to find your way with this example.
I've also just added another bit of code show-casing the usage of the
"non-linear least-squares" optimizers (svn revision 1338144).
Best regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Sébastien Brisard <se...@m4x.org>.
Hi,
>
> thanks for the reply. But i wonder what is the input for value and gradient.
> in DifferentiableMultivariateRealFunction this needs to be a double array
> but what needs to be provided there? The parameters for the function to
> optimize?
>
> Thank you very much again
>
> Andreas
>
Do please have a look to the examples, as your question (and my
answer) is too vague if not supported by proper code. I guess the
answer to your question is 'yes', the double[] array is indeed the set
of parameters, but again, do check the examples, I would not like to
be misguiding you. Besides the user guide which should provide you
with the answer, have a look to this implementation [1], line 153. In
this implementation, x[i] and y[i] are the data points, yhat[i] are
the model predictions, and a[] are the parameters. You should be able
to find your way with this example.
Good luck,
Sébastien
[1] http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math3/optimization/general/StatisticalReferenceDataset.java?view=markup
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Andreas Niekler <an...@fbm.htwk-leipzig.de>.
Hello,
thanks for the reply. But i wonder what is the input for value and
gradient. in DifferentiableMultivariateRealFunction this needs to be a
double array but what needs to be provided there? The parameters for the
function to optimize?
Thank you very much again
Andreas
Am 14.05.2012 10:34, schrieb Sébastien Brisard:
> Hi Andreas,
>>
>> My Question: Can someone show me an example for the usage of
>> DifferentiableMultivariateRealFunction and a matching optimizer. I
>> understand the concept but the implementational details are hard to find.
>> Here is what i understood:
>>
>> value: the value of the function.
>>
>> gradient(): an array containing all partial derivitivs gradients for the
>> parameters.
>>
>> partialDerivitive: Value containing only one gradient of the function
>>
>>
>> if i provide this basic implementation i should pass this to the optimizer.
>> Or do i have to provide an extra target function to the optimizer as
>> inicated by some optinmizers with the<FUNC> operator?
>>
>
> The user guide [1] shows a detailed example of the use of the
> Levenberg-Marquardt optimized. Also, you can have a look to the new
> AbstractLeastSquaresOptimizerAbstractTest [2] (available only through
> svn). The doTestStRD is another illustration.
>
> Hope this helps,
> Sébastien
>
> [1] http://commons.apache.org/math/userguide/optimization.html
> [2] http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math3/optimization/general/AbstractLeastSquaresOptimizerAbstractTest.java?view=markup,
> line 470
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>
--
Dipl.-Ing. (FH) Andreas Niekler
Mitarbeiter und Promovend
Bereich Multimedia-Produktionssysteme und -technologien
Hochschule für Technik, Wirtschaft und Kultur Leipzig
Fachbereich Medien
Besucher
Gustav-Freytag-Straße 40
04277 Leipzig
Telefon: +49 0341 30 76 2378
Email: aniekler@fbm.htwk-leipzig.de
http://www.fbm.htwk-leipzig.de
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [math] Usage of DifferentiableMultivariateRealFunction
Posted by Sébastien Brisard <se...@m4x.org>.
Hi Andreas,
>
> My Question: Can someone show me an example for the usage of
> DifferentiableMultivariateRealFunction and a matching optimizer. I
> understand the concept but the implementational details are hard to find.
> Here is what i understood:
>
> value: the value of the function.
>
> gradient(): an array containing all partial derivitivs gradients for the
> parameters.
>
> partialDerivitive: Value containing only one gradient of the function
>
>
> if i provide this basic implementation i should pass this to the optimizer.
> Or do i have to provide an extra target function to the optimizer as
> inicated by some optinmizers with the <FUNC> operator?
>
The user guide [1] shows a detailed example of the use of the
Levenberg-Marquardt optimized. Also, you can have a look to the new
AbstractLeastSquaresOptimizerAbstractTest [2] (available only through
svn). The doTestStRD is another illustration.
Hope this helps,
Sébastien
[1] http://commons.apache.org/math/userguide/optimization.html
[2] http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math3/optimization/general/AbstractLeastSquaresOptimizerAbstractTest.java?view=markup,
line 470
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org