You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Ajo Fod <aj...@gmail.com> on 2013/08/15 22:59:21 UTC
[math] Using the hessian in scalar unconstrained optimization
Hello,
Is'nt there an advantage to being able to compute the Jacobian of the
gradient precisely at a point?
If so, is there a class that uses the Jacobian instead of estimating the
jacobian from the last few iteration as NonLinearConjugateGradientOptimizer
does?
Thanks,
-Ajo
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Ajo Fod <aj...@gmail.com>.
If you're talking about analysis.differentiation package. I'm looking
forward to using it.
Though, my main point in this thread was that if one can compute the
hessian, it should be possible in certain cases to speed up convergence
rather than depending on an update rule to estimate an *approximate*
hessian in the Newton step.
Cheers,
-Ajo
On Fri, Aug 16, 2013 at 10:03 AM, Luc Maisonobe <Lu...@free.fr>wrote:
> Le 16/08/2013 18:55, Ajo Fod a écrit :
> > The algorithm computes the Hessian using an update rule. My question was
> > what if you can compute the hessian analytically?
> >
> > Hessian: http://en.wikipedia.org/wiki/Hessian_matrix
> > Gradient: http://en.wikipedia.org/wiki/Gradient
>
> We do have support to help generating second derivatives in the analysis
> package, but don't use them yet in the optimizers.
>
> best regards,
> Luc
>
> >
> > Cheers,
> > -Ajo
> >
> >
> > On Fri, Aug 16, 2013 at 9:39 AM, Luc Maisonobe <Luc.Maisonobe@free.fr
> >wrote:
> >
> >> Le 15/08/2013 22:59, Ajo Fod a écrit :
> >>> Hello,
> >>>
> >>> Is'nt there an advantage to being able to compute the Jacobian of the
> >>> gradient precisely at a point?
> >>>
> >>> If so, is there a class that uses the Jacobian instead of estimating
> the
> >>> jacobian from the last few iteration as
> >> NonLinearConjugateGradientOptimizer
> >>> does?
> >>
> >> I'm not sure what you really mean, but you can always pass an
> >> ObjectiveFunctionGradient holding any MultivariateVectorFunction to be
> >> used by the algorithm.
> >>
> >> Luc
> >>
> >>>
> >>> Thanks,
> >>> -Ajo
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 16/08/2013 18:55, Ajo Fod a écrit :
> The algorithm computes the Hessian using an update rule. My question was
> what if you can compute the hessian analytically?
>
> Hessian: http://en.wikipedia.org/wiki/Hessian_matrix
> Gradient: http://en.wikipedia.org/wiki/Gradient
We do have support to help generating second derivatives in the analysis
package, but don't use them yet in the optimizers.
best regards,
Luc
>
> Cheers,
> -Ajo
>
>
> On Fri, Aug 16, 2013 at 9:39 AM, Luc Maisonobe <Lu...@free.fr>wrote:
>
>> Le 15/08/2013 22:59, Ajo Fod a écrit :
>>> Hello,
>>>
>>> Is'nt there an advantage to being able to compute the Jacobian of the
>>> gradient precisely at a point?
>>>
>>> If so, is there a class that uses the Jacobian instead of estimating the
>>> jacobian from the last few iteration as
>> NonLinearConjugateGradientOptimizer
>>> does?
>>
>> I'm not sure what you really mean, but you can always pass an
>> ObjectiveFunctionGradient holding any MultivariateVectorFunction to be
>> used by the algorithm.
>>
>> Luc
>>
>>>
>>> Thanks,
>>> -Ajo
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Ajo Fod <aj...@gmail.com>.
The algorithm computes the Hessian using an update rule. My question was
what if you can compute the hessian analytically?
Hessian: http://en.wikipedia.org/wiki/Hessian_matrix
Gradient: http://en.wikipedia.org/wiki/Gradient
Cheers,
-Ajo
On Fri, Aug 16, 2013 at 9:39 AM, Luc Maisonobe <Lu...@free.fr>wrote:
> Le 15/08/2013 22:59, Ajo Fod a écrit :
> > Hello,
> >
> > Is'nt there an advantage to being able to compute the Jacobian of the
> > gradient precisely at a point?
> >
> > If so, is there a class that uses the Jacobian instead of estimating the
> > jacobian from the last few iteration as
> NonLinearConjugateGradientOptimizer
> > does?
>
> I'm not sure what you really mean, but you can always pass an
> ObjectiveFunctionGradient holding any MultivariateVectorFunction to be
> used by the algorithm.
>
> Luc
>
> >
> > Thanks,
> > -Ajo
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 15/08/2013 22:59, Ajo Fod a écrit :
> Hello,
>
> Is'nt there an advantage to being able to compute the Jacobian of the
> gradient precisely at a point?
>
> If so, is there a class that uses the Jacobian instead of estimating the
> jacobian from the last few iteration as NonLinearConjugateGradientOptimizer
> does?
I'm not sure what you really mean, but you can always pass an
ObjectiveFunctionGradient holding any MultivariateVectorFunction to be
used by the algorithm.
Luc
>
> Thanks,
> -Ajo
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Ajo Fod <aj...@gmail.com>.
Yes that works. Thanks!
On Thursday, August 15, 2013, Konstantin Berlin wrote:
> There would be an advantage, true. I don't know if commons has one
> (doesn't look like it). You can also try http://www.joptimizer.com/
>
> On Thu, Aug 15, 2013 at 4:59 PM, Ajo Fod <ajo.fod@gmail.com <javascript:;>>
> wrote:
> > Hello,
> >
> > Is'nt there an advantage to being able to compute the Jacobian of the
> > gradient precisely at a point?
> >
> > If so, is there a class that uses the Jacobian instead of estimating the
> > jacobian from the last few iteration as
> NonLinearConjugateGradientOptimizer
> > does?
> >
> > Thanks,
> > -Ajo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org <javascript:;>
> For additional commands, e-mail: dev-help@commons.apache.org<javascript:;>
>
>
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Konstantin Berlin <kb...@gmail.com>.
The general problem is not computationally tractable. You can trying
stochastic algorithms, like simulated annealing or genetic programmig, but
results depend on the problem. There is no point in computing derivatives
in that case either.
On Sunday, August 18, 2013, Ajo Fod wrote:
> Looks like Joptimizer is restricted to solving convex problems.
>
> My application is to minimize a generic non-linear function with linear
> constraints. Know of anything that does it?
>
> -Ajo
>
>
> On Thu, Aug 15, 2013 at 5:48 PM, Konstantin Berlin <kberlin@gmail.com<javascript:;>
> >wrote:
>
> > There would be an advantage, true. I don't know if commons has one
> > (doesn't look like it). You can also try http://www.joptimizer.com/
> >
> > On Thu, Aug 15, 2013 at 4:59 PM, Ajo Fod <ajo.fod@gmail.com<javascript:;>>
> wrote:
> > > Hello,
> > >
> > > Is'nt there an advantage to being able to compute the Jacobian of the
> > > gradient precisely at a point?
> > >
> > > If so, is there a class that uses the Jacobian instead of estimating
> the
> > > jacobian from the last few iteration as
> > NonLinearConjugateGradientOptimizer
> > > does?
> > >
> > > Thanks,
> > > -Ajo
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org<javascript:;>
> > For additional commands, e-mail: dev-help@commons.apache.org<javascript:;>
> >
> >
>
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Ajo Fod <aj...@gmail.com>.
Looks like Joptimizer is restricted to solving convex problems.
My application is to minimize a generic non-linear function with linear
constraints. Know of anything that does it?
-Ajo
On Thu, Aug 15, 2013 at 5:48 PM, Konstantin Berlin <kb...@gmail.com>wrote:
> There would be an advantage, true. I don't know if commons has one
> (doesn't look like it). You can also try http://www.joptimizer.com/
>
> On Thu, Aug 15, 2013 at 4:59 PM, Ajo Fod <aj...@gmail.com> wrote:
> > Hello,
> >
> > Is'nt there an advantage to being able to compute the Jacobian of the
> > gradient precisely at a point?
> >
> > If so, is there a class that uses the Jacobian instead of estimating the
> > jacobian from the last few iteration as
> NonLinearConjugateGradientOptimizer
> > does?
> >
> > Thanks,
> > -Ajo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Using the hessian in scalar unconstrained optimization
Posted by Konstantin Berlin <kb...@gmail.com>.
There would be an advantage, true. I don't know if commons has one
(doesn't look like it). You can also try http://www.joptimizer.com/
On Thu, Aug 15, 2013 at 4:59 PM, Ajo Fod <aj...@gmail.com> wrote:
> Hello,
>
> Is'nt there an advantage to being able to compute the Jacobian of the
> gradient precisely at a point?
>
> If so, is there a class that uses the Jacobian instead of estimating the
> jacobian from the last few iteration as NonLinearConjugateGradientOptimizer
> does?
>
> Thanks,
> -Ajo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org