You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Ajo Fod <aj...@gmail.com> on 2013/07/21 17:04:05 UTC
Re: [math] new feature to allow infinite limits in numerical integration.
The patches for Math-994 have been reworked ... slightly better design.
Here is some numerical analysis on the issue:
Laguerre is defined only in [0,+ve Inf]
Hermite is defined in [-Inf,+Inf]
I have two issues with the above:
1: Cant imagine how someone would use AQ. Which means as Gilles noticed,
you can't focus on the hard to converge sections of the integral.
2: If you use the integration without AQ. Any function that has a high
frequency region somewhere off the region where the polynomial focuses, the
integral probably won't converge. For Hermite with its weighting in
e^(-x^2) ... good luck with convergence with say computing CDF of N(0,100)
or for that matter N(100,1).
For an idea look at :
https://en.wikipedia.org/wiki/Gauss%E2%80%93Hermite_quadrature
On Fri, Jun 28, 2013 at 11:13 AM, Phil Steitz <ph...@gmail.com> wrote:
> On 6/28/13 7:44 AM, Gilles wrote:
> > On Mon, 24 Jun 2013 07:43:22 -0700, Ajo Fod wrote:
> >> As I read through the Wikipedia articles on Gauss-Hermite and
> >> Laguerre, I
> >> notice that they are talking about basis functions with
> >> infinity/s in its
> >> domain. How would this would solve the problem addressed in the
> >> MATH-994
> >> which is to restrict the bounds of the function being integrated
> >> so that
> >> numerical integration is possible.
> >>
> >>
> >>
> http://en.wikipedia.org/wiki/Gauss-Hermite_quadraturehttp://en.wikipedia.org/wiki/Gauss-Laguerre_quadrature
> >>
> >
> > The references rather state that those schemes are indeed used to
> > approximate improper integrals.
> >
> >> In any case, MATH-994 is a working solution to a major class of
> >> integration problems. A solution that passes the unit tests in the
> >> example is quite desirable.
> >
> > I do not deny that this approach could be useful, but since CM
> > already
> > provides a framework that should accommodate the approaches
> > referred to
> > above, and since we have some initial implementations for some of
> > those
> > schemes (Gauss-Hermite, Gauss-Chebyshev), it not unreasonable to
> > try and
> > reuse (or improve upon) the existing code; at least until you or
> > someone
> > else provides a compelling case where your alternative is indeed
> > better.
>
> +1 What would be really great is some numerical analysis references
> provided indicating which of the methods performs the best. This
> should not be that hard to research.
>
> Phil
> >
> > [Your use-case is until now reduced to the integral of the density
> > of the
> > normal distribution, which we know in advance is equal to one.]
> >
> >> So, I think the commons would be better of with modifications to
> >> MATH-994 to confirm to the design philosophy than say postponing a
> >> more complex integration scheme is available.
> >
> > The point is indeed that we have something _now_ (cf. above), and
> > helpful
> > work would be to examine its usefulness.
> >
> > Then, if we come to agree that the "change of variable" approach
> > should be
> > implemented in CM, we still need to do it in accordance with the
> > current
> > design of the "o.a.c.m.integration" package (or show that
> > something is wrong
> > or incomplete there and should consequently be modified or removed).
> >
> > Another aspect is to explore the obvious extensions (as you
> > mentioned in the
> > code comments) of the code which you propose, i.e. how it can be
> > made most
> > useful by increasing its flexibility.
> > In the case of "InfiniteIntegral", this could include:
> > * Providing suggestions on how to extend the existing framework so
> > that users
> > can easily pick what they need.
> > * Working out the details of generalizing to other types of
> > improper integrals
> > (sometimes this allows to readily figure out how to improve the
> > design).
> > * Showing how to apply the code in non-trivial examples
> > (eventually to be
> > included in the userguide).
> >
> >> The code can always be
> >> deprecated once a better implementation is available.
> >
> > Agreed, but in this case, another road is clearly visible and we
> > have to
> > try and keep everything consistent. It doesn't make sense, from a
> > design
> > and maintenance viewpoint, to postpone this work to a later time.
> >
> >> (You don't have
> >> to do it on my account, I already have what I need.)
> >
> > If not even you are going to use this code, what good would it do
> > to CM?
> > As I tried to explain above, in order to be useful to a wide range of
> > users, CM must provide code that is flexible. We all have to try
> > hard not
> > to consider a general purpose library as a repository for
> > disparate ad-hoc
> > code snippets.
> >
> >
> > Best regards,
> > Gilles
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] new feature to allow infinite limits in numerical integration.
Posted by Ajo Fod <aj...@gmail.com>.
You mean just move the Infinite.class to the test section?
Sure ... go ahead.
-Ajo
On Thu, Jul 25, 2013 at 5:33 AM, Gilles <gi...@harfang.homelinux.org>wrote:
> On Wed, 24 Jul 2013 07:50:25 -0700, Ajo Fod wrote:
>
>> It would be nice to know whether the Gauss-Hermite implementation recently
>>
>> added proves useful for that purpose; and if so, whether it could be
>>> added
>>> as a non-trivial example in the user guide.
>>>
>>>
>> I'll check it out when I get back to it.
>>
>>
>>
>>> How do you propose to do the test without a change of variables?
>>>
>>>>
>>>>
>>> The test would consist in assessing how different adaptive strategies
>>> perform
>>> when integrating the transformed Gaussian over [-1, 1].
>>>
>>>
>>> How do you propose that I make the transformation to [-1,1] available
>> for
>> the tests?
>>
>
> For the time being (i.e. until we gather ideas on how to represent a
> change of variable in CM that would not necessarily be tied to the
> computation of improper integrals), the test function can be stored in
> the "test" part of the repository (see how functions are defined there
> for e.g. testing root solvers), within the package that will provide
> unit tests for adaptive strategies.
>
> f(x) = N(mu, sigma; x)
> tsq(x) = x * x
> tcp(x) = (1 - tsq(x))
> g(x) = x / tcp(x)
> h(x) = (1 + tsq(x)) / (tcp(x) * tcp(x))
> fc(x) = f(g(x)) * h(x)
>
> So, for the caller (unit test), there is just the univariate function
> "fc(x)" (probably parameterized with "mu" and "sigma") without any
> reference to a change of variable.
>
>
> HTH,
>
> Gilles
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@commons.**apache.org<de...@commons.apache.org>
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] new feature to allow infinite limits in numerical
integration.
Posted by Gilles <gi...@harfang.homelinux.org>.
On Wed, 24 Jul 2013 07:50:25 -0700, Ajo Fod wrote:
> It would be nice to know whether the Gauss-Hermite implementation
> recently
>
>> added proves useful for that purpose; and if so, whether it could be
>> added
>> as a non-trivial example in the user guide.
>>
>
> I'll check it out when I get back to it.
>
>
>>
>> How do you propose to do the test without a change of variables?
>>>
>>
>> The test would consist in assessing how different adaptive
>> strategies
>> perform
>> when integrating the transformed Gaussian over [-1, 1].
>>
>>
> How do you propose that I make the transformation to [-1,1] available
> for
> the tests?
For the time being (i.e. until we gather ideas on how to represent a
change of variable in CM that would not necessarily be tied to the
computation of improper integrals), the test function can be stored in
the "test" part of the repository (see how functions are defined there
for e.g. testing root solvers), within the package that will provide
unit tests for adaptive strategies.
f(x) = N(mu, sigma; x)
tsq(x) = x * x
tcp(x) = (1 - tsq(x))
g(x) = x / tcp(x)
h(x) = (1 + tsq(x)) / (tcp(x) * tcp(x))
fc(x) = f(g(x)) * h(x)
So, for the caller (unit test), there is just the univariate function
"fc(x)" (probably parameterized with "mu" and "sigma") without any
reference to a change of variable.
HTH,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] new feature to allow infinite limits in numerical integration.
Posted by Ajo Fod <aj...@gmail.com>.
It would be nice to know whether the Gauss-Hermite implementation recently
> added proves useful for that purpose; and if so, whether it could be added
> as a non-trivial example in the user guide.
>
I'll check it out when I get back to it.
>
> How do you propose to do the test without a change of variables?
>>
>
> The test would consist in assessing how different adaptive strategies
> perform
> when integrating the transformed Gaussian over [-1, 1].
>
>
How do you propose that I make the transformation to [-1,1] available for
the tests?
-Ajo
Re: [math] new feature to allow infinite limits in numerical
integration.
Posted by Gilles <gi...@harfang.homelinux.org>.
On Tue, 23 Jul 2013 09:03:15 -0700, Ajo Fod wrote:
>>
>> So, given this development, I'm not particularly motivated to
>> develop AQ
>>> further.
>>>
>>
>> So, all the fuss just to use a numerical method for computing a
>> value
>> known in advance?
>
>
> The fuss was about the ability to compute the partition function for
> the
> exponential family. It has closed form values for some members of
> the
> family such as the normal.
It would be nice to know whether the Gauss-Hermite implementation
recently
added proves useful for that purpose; and if so, whether it could be
added
as a non-trivial example in the user guide.
>
>> We already know one, don't we?
>> [Hint: the plots in
>> https://issues.apache.org/**jira/browse/MATH-995<https://issues.apache.org/jira/browse/MATH-995>
>> I mean that this function could serve as a validation test and
>> benchmark (since we know that the answer is 1).]
>>
>
> How do you propose to do the test without a change of variables?
The test would consist in assessing how different adaptive strategies
perform
when integrating the transformed Gaussian over [-1, 1].
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] new feature to allow infinite limits in numerical integration.
Posted by Ajo Fod <aj...@gmail.com>.
>
> So, given this development, I'm not particularly motivated to develop AQ
>> further.
>>
>
> So, all the fuss just to use a numerical method for computing a value
> known in advance?
The fuss was about the ability to compute the partition function for the
exponential family. It has closed form values for some members of the
family such as the normal.
> We already know one, don't we?
> [Hint: the plots in https://issues.apache.org/**jira/browse/MATH-995<https://issues.apache.org/jira/browse/MATH-995>
> I mean that this function could serve as a validation test and
> benchmark (since we know that the answer is 1).]
>
How do you propose to do the test without a change of variables?
Cheers,
Ajo.
Re: [math] new feature to allow infinite limits in numerical integration.
Posted by Ajo Fod <aj...@gmail.com>.
However, the infinite class would have allowed integration from [-Inf,
>> C!=0].
>> Can you manage that with Gauss Hermite?
>>
>
> I don't think so (non-authoritative answer).
> that the answer is 1).]
>
>
Just Posted MATH-1015 which describes how you'd do this with your
integration methods.
Cheers,
Ajo.
Re: [math] new feature to allow infinite limits in numerical
integration.
Posted by Gilles <gi...@harfang.homelinux.org>.
On Sun, 21 Jul 2013 17:41:16 -0700, Ajo Fod wrote:
> Based on the tests you posted it seems like Gauss-Hermite fills the
> capability CM was missing of doing one class of improper integrals.
>
> However, the infinite class would have allowed integration from
> [-Inf,
> C!=0].
> Can you manage that with Gauss Hermite?
I don't think so (non-authoritative answer).
> So, given this development, I'm not particularly motivated to develop
> AQ
> further.
So, all the fuss just to use a numerical method for computing a value
known in advance?
> I'll get back if I hit a "high frequency" function to bother with
> it.
We already know one, don't we?
[Hint: the plots in https://issues.apache.org/jira/browse/MATH-995
I mean that this function could serve as a validation test and
benchmark (since we know that the answer is 1).]
> [...]
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] new feature to allow infinite limits in numerical integration.
Posted by Ajo Fod <aj...@gmail.com>.
Based on the tests you posted it seems like Gauss-Hermite fills the
capability CM was missing of doing one class of improper integrals.
However, the infinite class would have allowed integration from [-Inf,
C!=0].
Can you manage that with Gauss Hermite?
So, given this development, I'm not particularly motivated to develop AQ
further. I'll get back if I hit a "high frequency" function to bother with
it.
However, if you chose to adapt the code I provided note that it:
1) avoids using recursive function calls, so you stack size is not a
constraint as implementations typically do.
2) it uses Simpson's rule, but you should be able to swap in other
quadrature rules.
Cheers,
Ajo
On Sun, Jul 21, 2013 at 5:16 PM, Gilles <gi...@harfang.homelinux.org>wrote:
> On Sun, 21 Jul 2013 08:04:05 -0700, Ajo Fod wrote:
>
>> [...]
>>
>>
>> Here is some numerical analysis on the issue:
>>
>> Laguerre is defined only in [0,+ve Inf]
>> Hermite is defined in [-Inf,+Inf]
>>
>> I have two issues with the above:
>> 1: Cant imagine how someone would use AQ. Which means as Gilles noticed,
>> you can't focus on the hard to converge sections of the integral.
>> 2: If you use the integration without AQ. Any function that has a high
>> frequency region somewhere off the region where the polynomial focuses,
>> the
>> integral probably won't converge. For Hermite with its weighting in
>> e^(-x^2) ... good luck with convergence with say computing CDF of N(0,100)
>> or for that matter N(100,1).
>>
>
> I'm afraid that these are counter-examples to your suppositions.
> Please have a look at
> https://issues.apache.org/**jira/browse/MATH-997<https://issues.apache.org/jira/browse/MATH-997>
>
>
>
> For an idea look at :
>> https://en.wikipedia.org/wiki/**Gauss%E2%80%93Hermite_**quadrature<https://en.wikipedia.org/wiki/Gauss%E2%80%93Hermite_quadrature>
>>
>>
> Look at what, precisely?
>
>
>
> Regards,
> Gilles
>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@commons.**apache.org<de...@commons.apache.org>
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] new feature to allow infinite limits in numerical
integration.
Posted by Gilles <gi...@harfang.homelinux.org>.
On Sun, 21 Jul 2013 08:04:05 -0700, Ajo Fod wrote:
> [...]
>
> Here is some numerical analysis on the issue:
>
> Laguerre is defined only in [0,+ve Inf]
> Hermite is defined in [-Inf,+Inf]
>
> I have two issues with the above:
> 1: Cant imagine how someone would use AQ. Which means as Gilles
> noticed,
> you can't focus on the hard to converge sections of the integral.
> 2: If you use the integration without AQ. Any function that has a
> high
> frequency region somewhere off the region where the polynomial
> focuses, the
> integral probably won't converge. For Hermite with its weighting in
> e^(-x^2) ... good luck with convergence with say computing CDF of
> N(0,100)
> or for that matter N(100,1).
I'm afraid that these are counter-examples to your suppositions.
Please have a look at
https://issues.apache.org/jira/browse/MATH-997
> For an idea look at :
> https://en.wikipedia.org/wiki/Gauss%E2%80%93Hermite_quadrature
>
Look at what, precisely?
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] new feature to allow infinite limits in numerical
integration.
Posted by Gilles <gi...@harfang.homelinux.org>.
On Sun, 21 Jul 2013 08:04:05 -0700, Ajo Fod wrote:
> The patches for Math-994 have been reworked ... slightly better
> design.
Sorry but I don't understand the purpose of adding a patch to
a closed issue...
> Here is some numerical analysis on the issue:
>
> Laguerre is defined only in [0,+ve Inf]
> Hermite is defined in [-Inf,+Inf]
>
> I have two issues with the above:
> 1: Cant imagine how someone would use AQ. Which means as Gilles
> noticed,
> you can't focus on the hard to converge sections of the integral.
> 2: If you use the integration without AQ. Any function that has a
> high
> frequency region somewhere off the region where the polynomial
> focuses, the
> integral probably won't converge. For Hermite with its weighting in
> e^(-x^2) ... good luck with convergence with say computing CDF of
> N(0,100)
> or for that matter N(100,1).
> For an idea look at :
> https://en.wikipedia.org/wiki/Gauss%E2%80%93Hermite_quadrature
>
You are again mixing two concepts:
* improper integrals
* adaptive quadrature
[We already talked about that long ago.]
Let's be practical: I propose that you focus on adaptive quadrature
_first_ because we know that it is needed (a building block) in order
to perform integration on an infinite interval, using a change of
variables.
The goal would be to create a code similar to
"IterativeGaussLegendreIntegrator"
but with a different adaptive strategy: instead of dividing the
interval
into equal-length sub-intervals, it would... (well, you know).
What do you think?
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org