You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@commons.apache.org by "Pavel Ryzhov (JIRA)" <ji...@apache.org> on 2010/12/22 22:29:00 UTC
[jira] Created: (MATH-460) Levy Distribution
Levy Distribution
-----------------
Key: MATH-460
URL: https://issues.apache.org/jira/browse/MATH-460
Project: Commons Math
Issue Type: New Feature
Reporter: Pavel Ryzhov
Priority: Minor
Fix For: 3.0
Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Mikkel Meyer Andersen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291700#comment-13291700 ]
Mikkel Meyer Andersen commented on MATH-460:
--------------------------------------------
Any news on this? I think the easiest way to fix this is to simple copy an existing distribution and its test and modify it according to the Levy distribution. If Andrew is still up for it, I warmly welcome him. If not, I will do it.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Assignee: Mikkel Meyer Andersen
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264652#comment-13264652 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
I've been able to sample what looks to be a proper a Levy Distribution using the T-Distribution (student's) in R with the following code:
> N<- 100
> distribution-vector<- rt(N,df=2)
[modified from http://zoonek2.free.fr/UNIX/48_R/07.html]
When graphed as an XY, I'm able to plot what looks to be a "Levy Flight" (see citation for details) as used in ecological modeling. This could serve as the basis to implement the mathematical expectations for the missing unit tests.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463636#comment-13463636 ]
Gilles commented on MATH-460:
-----------------------------
Could you please edit your last comment, as the output is garbled (see [wiki markup help|https://issues.apache.org/jira/secure/WikiRendererHelpAction.jspa?section=advanced])?
bq. It seems like it should be overridden in my own test case, [...]
No, it is the test is based on the definition of "quartiles".
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263984#comment-13263984 ]
Andrew Waterman edited comment on MATH-460 at 4/27/12 9:31 PM:
---------------------------------------------------------------
Hi Sebastian,
I've been working with a local mathematician, and we've been able to resolve a few of the tests. It looks like the mean and standard deviation for Levy should return +∞. From my friend (in response to a few questions):
(1) According to the summary of properties of the Lévy distrib given in the right-hand margin of the Wikipedia article (https://en.wikipedia.org/wiki/L%C3%A9vy_distribution), the mean for the Lévy distrib is indeed +∞.
(2) If you search the article https://en.wikipedia.org/wiki/Standard_deviation for the word "variance", you'll find several places that say that the standard deviation is the square root of the variance. Since the summary mentioned in (1) says that the variance of the Lévy distrib is +∞, its standard deviation should be +∞ as well. (just as Mathematica says). From what I recall of statistics and probability, that conclusion is consistent with the statement in the article mentioned in (1), that the formula for the moments of a Levy distrib "diverges for all n > 0 so that the moments of the Lévy distribution do not exist." (The variance is the second moment [n=2].)
The GNU Scientific Library has an implementation of Levy for "alpha-stable distributions" and for "skew alpha-stable Distribution" which might provide some insight and provides something to test against and compare to: [http://bzr.savannah.gnu.org/lh/gsl/trunk/annotate/head:/randist/levy.c]
I'll continue to discuss with my friend, but I'm still not quite sure how to implement the failed tests for:
testQuantiles
testCumulativeProbabilities
testInverseCumulativeProbabilities
testConsistency
testSampling
Any feedback and insight/suggestions you might want to pass on would be very helpful.
Of course, if Pavel wanted to take this up again, that would work as well.
was (Author: awaterma):
Hi Sebastian,
I've been working with a local mathematician, and we've been able to resolve a few of the tests. It looks like the mean and standard deviation for Levy should return +∞. From my friend (in response to a few questions):
(1) According to the summary of properties of the Lévy distrib given in the right-hand margin of the Wikipedia article (https://en.wikipedia.org/wiki/L%C3%A9vy_distribution), the mean for the Lévy distrib is indeed +∞.
(2) If you search the article https://en.wikipedia.org/wiki/Standard_deviation for the word "variance", you'll find several places that say that the standard deviation is the square root of the variance. Since the summary mentioned in (1) says that the variance of the Lévy distrib is +∞, its standard deviation should be +∞ as well. (just as Mathematica says). From what I recall of statistics and probability, that conclusion is consistent with the statement in the article mentioned in (1), that the formula for the moments of a Levy distrib "diverges for all n > 0 so that the moments of the Lévy distribution do not exist." (The variance is the second moment [n=2].)
The GNU Scientific Library has an implementation of Levy for "alpha-stable distributions" and for "skew alpha-stable Distribution" which might provide some insight and provides something to test against and compare to: [http://bzr.savannah.gnu.org/lh/gsl/trunk/annotate/head:/randist/levy.c]
I'll continue to discuss with my friend, but I'm still not quite sure how to implement the failed tests for:
testQuantiles
testCumulativeProbabilities
testInverseCumulativeProbabilities
testConsistency
testSampling
Any feedback and insight/suggestions you might want to pass on would be very helpful.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264685#comment-13264685 ]
Sébastien Brisard commented on MATH-460:
----------------------------------------
Hi Andrew,
right now, I'm busy working on MATH-784, but will gladly help in a few days if you need support. I take it you already have an implementation of the Levy distribution which now compiles (even if it does not pass all the tests?).
Thanks for your involvement,
Sébastien
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MATH-460) Levy Distribution
Posted by "Phil Steitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448513#comment-13448513 ]
Phil Steitz edited comment on MATH-460 at 9/27/12 8:25 AM:
-----------------------------------------------------------
Thanks, Andrew!
What we are trying to capture in "support" is the smallest closed set whose complement has probability zero. The definitions of getSupportUpperBound and getSupportLowerBound in the RealDistribution javadoc roughly correspond to this in the case where the distribution is continuous with connected support (every interval of positive length between lower and upper support bounds has positive mass). The isSupportConnected property says that there are no such gaps. This is used by the default inverse cumulative probability method. The isSupportXxxBoundInclusive methods are no longer used; but should be interpreted to mean that these values are within the domain of the density function (i.e. density does not return NaN, an infinity or throw).
was (Author: psteitz):
Thanks, Andrew!
What we are trying to capture in "support" is the smallest closed set whose complement has probability zero. The definitions of getSupportUpperBound and getSupportLowerBound in the RealDistribution javadoc roughly correspond to this in the case where the distribution is continuous with connected support (every interval of positive length between lower and upper support bounds has positive mass). The isSupportConnected property says that there are no such gaps. This is used by the default inverse cumulative probability method. The isSupportXxxBoundInclusive methods are no longer used; but should be interpreted to mean that these values are within the domain of the density function (i.e. density does not return NaN, and infinity or throw).
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13264652#comment-13264652 ]
Andrew Waterman edited comment on MATH-460 at 4/30/12 12:31 AM:
----------------------------------------------------------------
I've been able to sample what looks to be a proper a Levy Distribution using the T-Distribution (student's) in R with the following code:
> N<- 100
> distribution-vector<- rt(N,df=2)
[modified from http://zoonek2.free.fr/UNIX/48_R/07.html]
When graphed as an XY, I'm able to plot what looks to be a "Levy Flight" (see citation for details) as used in ecological modeling. This could serve as the basis to implement the mathematical expectations for the missing unit tests (with a shared random seed).
was (Author: awaterma):
I've been able to sample what looks to be a proper a Levy Distribution using the T-Distribution (student's) in R with the following code:
> N<- 100
> distribution-vector<- rt(N,df=2)
[modified from http://zoonek2.free.fr/UNIX/48_R/07.html]
When graphed as an XY, I'm able to plot what looks to be a "Levy Flight" (see citation for details) as used in ecological modeling. This could serve as the basis to implement the mathematical expectations for the missing unit tests.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463470#comment-13463470 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
Thanks for the help Phil! Do you think you could explain the following test? It seems like it should be overridden in my own test case, as I'm a bit confused about the set [250,250,250,250] for the variable "expected" below:
public abstract class RealDistributionAbstractTest {
....
/**
* Test sampling
*/
@Test
public void testSampling() {
final int sampleSize = 1000;
distribution.reseedRandomGenerator(1000); // Use fixed seed
double[] sample = distribution.sample(sampleSize);
double[] quartiles = TestUtils.getDistributionQuartiles(distribution);
double[] expected = {250, 250, 250, 250};
long[] counts = new long[4];
for (int i = 0; i < sampleSize; i++) {
TestUtils.updateCounts(sample[i], counts, quartiles);
}
TestUtils.assertChiSquareAccept(expected, counts, 0.001);
}
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MATH-460) Levy Distribution
Posted by "Pavel Ryzhov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pavel Ryzhov updated MATH-460:
------------------------------
Attachment: levy_math_460.patch
The patch includes:
1. implementation of Levy distribution
2. Unit-tests for it
Public methods are javadoc'd
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.0
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Issue Comment Edited] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266859#comment-13266859 ]
Andrew Waterman edited comment on MATH-460 at 5/2/12 7:44 PM:
--------------------------------------------------------------
Hi Sébastien,
Things are moving along. I just noticed that there is already a T-Distribution available in the Math Commons, which we are currently using in R for both x and y coordinates in a plot that looks very similar to what I expect from a Lévy distribution. This may just suffice for our work. I can pass on the modifications that I made to reach this current point if you think that would be worthwhile. Love to hear your opinion on using the T-Distribution in order to extract a vector comparable to what one would find in "Lévy-T" (e.g., rt (10,df=2) versus a 10 length vector directly from Lévy).
was (Author: awaterma):
Hi Sébastien,
Things are moving along. I just noticed that there is already a T-Distribution available in the Math Commons, which we are currently using in R for both x and y coordinates in a plot that looks very similar to what I expect from a Lévy distribution. This may just suffice for our work. I can pass on the modifications that I made to reach this current point if you think that would be worthwhile. Love to hear your opinion on using the T-Distribution in order to extract an RPM comparable to Lévy (e.g., rt (10,df=2) versus a 10 length vector directly from Lévy).
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448179#comment-13448179 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
I've taken another stab at most of the class, doing an independent implementation working from the equations presented in Wikipedia and modifying the implementation where it differed from Pavel's. I haven't been able to implement the following, the JavaDoc is a bit vague in RealDistribution:
/**
* Use this method to get information about whether the lower bound
* of the support is inclusive or not.
*
* @return whether the lower bound of the support is inclusive or not
*/
boolean isSupportLowerBoundInclusive();
/**
* Use this method to get information about whether the upper bound
* of the support is inclusive or not.
*
* @return whether the upper bound of the support is inclusive or not
*/
isSupportUpperBoundInclusive();
/**
* Use this method to get information about whether the support is connected,
* i.e. whether all values between the lower and upper bound of the support
* are included in the support.
*
* @return whether the support is connected or not
*/
boolean isSupportConnected();
Love some clarification (links would be appreciated) and I can try and continue on a bit further. Otherwise, I have some work that I can attach to share with the future. :)
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13262144#comment-13262144 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
Thanks Sébastien, I've taken a look, integrated the code and the previous unit test, but it looks like there's still more work to be done:
Failed tests: testQuantiles(org.apache.commons.math3.distribution.LevyDistributionTest): Incorrect cumulative probability value returned for -2.0 expected:<0.02275013194817921> but was:<NaN>
testGetMean(org.apache.commons.math3.distribution.LevyDistributionTest): expected:<2.1> but was:<0.0>
testGetStandardDeviation(org.apache.commons.math3.distribution.LevyDistributionTest): expected:<1.4> but was:<1.0>
testCumulativeProbabilities(org.apache.commons.math3.distribution.LevyDistributionTest): expected:<0.0015654023> but was:<0.02298919298507429>
testConsistency(org.apache.commons.math3.distribution.LevyDistributionTest): Inconsistent cumulative probabilities for (1.0,1.1) expected:<0.0015654022580025018> but was:<0.02298919298507429>
Tests in error:
testInverseCumulativeProbabilities(org.apache.commons.math3.distribution.LevyDistributionTest): endpoints do not specify an interval: [1, 0.52]
testSampling(org.apache.commons.math3.distribution.LevyDistributionTest): endpoints do not specify an interval: [1, 0.789]
Tests run: 3560, Failures: 5, Errors: 2, Skipped: 1
I wonder if the tests ran successfully before? I'm not sure if they were ever executed in the previous patch, due to the missing @Test annotations.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261706#comment-13261706 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
This would be a very useful addition to the math libraries; especially for ecological modeling. If the issue with the patch is just ensuring that the getSupportUpperBound() abstract method is implemented, I'm willing to volunteer some time to take a look.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Phil Steitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464189#comment-13464189 ]
Phil Steitz commented on MATH-460:
----------------------------------
The test is doing a crude verification that the distribution of values generated by the sample() method is consistent with the distribution function. If the quartiles are correct and the sample() method is functioning correctly, generated values should be evenly divided among quartile ranges. The Chisquare test verifies that. When it fails, it should show you what the bin counts are and you can see where too many or too few values are being generated. You can also check the quartiles themselves using R's qlevy(p,m,s) function. Compare what R gives for p=0.25, 0.5, 0.75 to what your code gives for these quartiles.
And yes, R quantile tests would be helpful. Patches welcome :)
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Mikkel Meyer Andersen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291724#comment-13291724 ]
Mikkel Meyer Andersen commented on MATH-460:
--------------------------------------------
Okay, thanks for clarifying, Sébastien. After reading the original issue description by Pavel again, I agree that we should simply implement the classical Levy distribution. If anybody needs stable distributions implemented, they must open a new issue.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Assignee: Mikkel Meyer Andersen
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266859#comment-13266859 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
Hi Sébastien,
Things are moving along. I just noticed that there is already a T-Distribution available in the Math Commons, which we are currently using in R for both x and y coordinates in a plot that looks very similar to what I expect from a Lévy distribution. This may just suffice for our work. I can pass on the modifications that I made to reach this current point if you think that would be worthwhile. Love to hear your opinion on using the T-Distribution in order to extract an RPM comparable to Lévy (e.g., rt (10,df=2) versus a 10 length vector directly from Lévy).
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Phil Steitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13083505#comment-13083505 ]
Phil Steitz commented on MATH-460:
----------------------------------
Any comments on this? Anyone else want to claim it? ;)
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.0
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13261766#comment-13261766 ]
Sébastien Brisard commented on MATH-460:
----------------------------------------
Andrew, if you want to have a go, please do so! We will then gladly review your code once you feel.
Please note that real distributions have significantly been changed in December last year, so the work required for the patch to compile against the current version of CM might or might not reduce to the implementation of one single method. Anyway, implementation from scratch (if you find it more appropriate) would be fairly easy within the current framework. Please also note that we got rid of {{xxxDistribution}}/{{xxxDistributionImpl}} pairs by removing all interfaces, and renaming {{xxxDistributionImpl}} to {{xxxDistribution}}.
Good luck, and thanks in advance.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MATH-460) Levy Distribution
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gilles updated MATH-460:
------------------------
Fix Version/s: (was: 3.1)
3.2
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MATH-460) Levy Distribution
Posted by "Pavel Ryzhov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pavel Ryzhov updated MATH-460:
------------------------------
Attachment: (was: levy_math_460.patch)
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.0
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267207#comment-13267207 ]
Sébastien Brisard commented on MATH-460:
----------------------------------------
Hi Andrew,
sorry I haven't been very helpful up to now. I'm not sure I really understand your problem. Are you stuck with the sampling of the Lévy distribution? There is a default implementation of the {{sample()}} method which _should_ do the work, provided {{double inverseCumulativeProbability(double)}} is correctly implemented.
By all means, if you think that your work is already in a good shape (if not complete), do post your implementation. Do not forget to check the box which says that your attachment is fit for inclusion in CM3.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13088026#comment-13088026 ]
Gilles commented on MATH-460:
-----------------------------
* In "LevyDistribution.java":
** Javadoc typo: "mean" instead of "median".
* In "LevyDistributionImpl.java":
** The license header is missing.
** Instance fields should be declared "final".
** "erfConst" should have a more explict name like "INVERSE_ERF_HALF_SQUARED" (?)
** Couldn't "erfConst" be computed? Couldn't there be more than 10 significant digits?
** Shouldn't "getDomainUpperBound" return "POSITIVE_INFINITY" instead of "MAX_VALUE"?
* In "LevyDistributionTest.java":
** JUnit4 "import" statements are missing.
** The constructor is unnecessary and should be removed.
** The additional test cases are not marked with "@Test".
The patch is old; it does not compile on the current trunk:
{noformat}
[...]/LevyDistributionImpl.java:[18,7] org.apache.commons.math.distribution.LevyDistributionImpl is not abstract and does not override abstract method getSupportUpperBound() in org.apache.commons.math.distribution.AbstractContinuousDistribution
{noformat}
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.0
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Phil Steitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448513#comment-13448513 ]
Phil Steitz commented on MATH-460:
----------------------------------
Thanks, Andrew!
What we are trying to capture in "support" is the smallest closed set whose complement has probability zero. The definitions of getSupportUpperBound and getSupportLowerBound in the RealDistribution javadoc roughly correspond to this in the case where the distribution is continuous with connected support (every interval of positive length between lower and upper support bounds has positive mass). The isSupportConnected property says that there are no such gaps. This is used by the default inverse cumulative probability method. The isSupportXxxBoundInclusive methods are no longer used; but should be interpreted to mean that these values are within the domain of the density function (i.e. density does not return NaN, and infinity or throw).
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MATH-460) Levy Distribution
Posted by "Gilles (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463636#comment-13463636 ]
Gilles edited comment on MATH-460 at 9/27/12 1:59 AM:
------------------------------------------------------
Could you please edit your last comment, as the output is garbled (see [wiki markup help|https://issues.apache.org/jira/secure/WikiRendererHelpAction.jspa?section=advanced])?
bq. It seems like it should be overridden in my own test case, [...]
No, the test is based on the definition of "quartiles".
was (Author: erans):
Could you please edit your last comment, as the output is garbled (see [wiki markup help|https://issues.apache.org/jira/secure/WikiRendererHelpAction.jspa?section=advanced])?
bq. It seems like it should be overridden in my own test case, [...]
No, it is the test is based on the definition of "quartiles".
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463470#comment-13463470 ]
Andrew Waterman edited comment on MATH-460 at 9/27/12 1:43 AM:
---------------------------------------------------------------
Thanks for the help Phil! Do you think you could explain the following test? It seems like it should be overridden in my own test case, as I'm a bit confused about the set [250,250,250,250] for the variable "expected" below:
Link: https://github.com/apache/commons-math/blob/trunk/src/test/java/org/apache/commons/math3/distribution/RealDistributionAbstractTest.java#L290
was (Author: awaterma):
Thanks for the help Phil! Do you think you could explain the following test? It seems like it should be overridden in my own test case, as I'm a bit confused about the set [250,250,250,250] for the variable "expected" below:
Link: https://github.com/apache/commons-math/blob/trunk/src/test/java/org/apache/commons/math3/distribution/RealDistributionAbstractTest.java#L290
public abstract class RealDistributionAbstractTest {
....
/**
* Test sampling
*/
@Test
public void testSampling() {
final int sampleSize = 1000;
distribution.reseedRandomGenerator(1000); // Use fixed seed
double[] sample = distribution.sample(sampleSize);
double[] quartiles = TestUtils.getDistributionQuartiles(distribution);
double[] expected = {250, 250, 250, 250};
long[] counts = new long[4];
for (int i = 0; i < sampleSize; i++) {
TestUtils.updateCounts(sample[i], counts, quartiles);
}
TestUtils.assertChiSquareAccept(expected, counts, 0.001);
}
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263984#comment-13263984 ]
Andrew Waterman edited comment on MATH-460 at 4/30/12 12:21 AM:
----------------------------------------------------------------
Hi Sebastian,
I've been working with a local mathematician, and we've been able to resolve a few of the tests. It looks like the mean and standard deviation for Levy should return +∞. From my friend (in response to a few questions):
(1) According to the summary of properties of the Lévy distrib given in the right-hand margin of the Wikipedia article (https://en.wikipedia.org/wiki/L%C3%A9vy_distribution), the mean for the Lévy distrib is indeed +∞.
(2) If you search the article https://en.wikipedia.org/wiki/Standard_deviation for the word "variance", you'll find several places that say that the standard deviation is the square root of the variance. Since the summary mentioned in (1) says that the variance of the Lévy distrib is +∞, its standard deviation should be +∞ as well. (just as Mathematica says). From what I recall of statistics and probability, that conclusion is consistent with the statement in the article mentioned in (1), that the formula for the moments of a Levy distrib "diverges for all n > 0 so that the moments of the Lévy distribution do not exist." (The variance is the second moment [n=2].)
I'll continue to discuss with my friend, but I'm still not quite sure how to implement the failed tests for:
testQuantiles
testCumulativeProbabilities
testInverseCumulativeProbabilities
testConsistency
testSampling
Any feedback and insight/suggestions you might want to pass on would be very helpful.
was (Author: awaterma):
Hi Sebastian,
I've been working with a local mathematician, and we've been able to resolve a few of the tests. It looks like the mean and standard deviation for Levy should return +∞. From my friend (in response to a few questions):
(1) According to the summary of properties of the Lévy distrib given in the right-hand margin of the Wikipedia article (https://en.wikipedia.org/wiki/L%C3%A9vy_distribution), the mean for the Lévy distrib is indeed +∞.
(2) If you search the article https://en.wikipedia.org/wiki/Standard_deviation for the word "variance", you'll find several places that say that the standard deviation is the square root of the variance. Since the summary mentioned in (1) says that the variance of the Lévy distrib is +∞, its standard deviation should be +∞ as well. (just as Mathematica says). From what I recall of statistics and probability, that conclusion is consistent with the statement in the article mentioned in (1), that the formula for the moments of a Levy distrib "diverges for all n > 0 so that the moments of the Lévy distribution do not exist." (The variance is the second moment [n=2].)
The GNU Scientific Library has an implementation of Levy for "alpha-stable distributions" and for "skew alpha-stable Distribution" which might provide some insight and provides something to test against and compare to: [http://bzr.savannah.gnu.org/lh/gsl/trunk/annotate/head:/randist/levy.c]
I'll continue to discuss with my friend, but I'm still not quite sure how to implement the failed tests for:
testQuantiles
testCumulativeProbabilities
testInverseCumulativeProbabilities
testConsistency
testSampling
Any feedback and insight/suggestions you might want to pass on would be very helpful.
Of course, if Pavel wanted to take this up again, that would work as well.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Sébastien Brisard (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291723#comment-13291723 ]
Sébastien Brisard commented on MATH-460:
----------------------------------------
As far as I understood from offline exchanges, there was a slight confusion between
* classical Levy distributions (which I believe would be fairly easy to implement),
* Levy alpha-stable distributions, which are more involved.
My understanding is that Andrew wanted to have a go with the latter, but gave up. He also told me that he would probably not have time to implemented the former. So feel free to do it!
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Assignee: Mikkel Meyer Andersen
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13263984#comment-13263984 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
Hi Sebastian,
I've been working with a local mathematician, and we've been able to resolve a few of the tests. It looks like the mean and standard deviation for Levy should return +∞. From my friend (in response to a few questions):
(1) According to the summary of properties of the Lévy distrib given in the right-hand margin of the Wikipedia article (https://en.wikipedia.org/wiki/L%C3%A9vy_distribution), the mean for the Lévy distrib is indeed +∞.
(2) If you search the article https://en.wikipedia.org/wiki/Standard_deviation for the word "variance", you'll find several places that say that the standard deviation is the square root of the variance. Since the summary mentioned in (1) says that the variance of the Lévy distrib is +∞, its standard deviation should be +∞ as well. (just as Mathematica says). From what I recall of statistics and probability, that conclusion is consistent with the statement in the article mentioned in (1), that the formula for the moments of a Levy distrib "diverges for all n > 0 so that the moments of the Lévy distribution do not exist." (The variance is the second moment [n=2].)
The GNU Scientific Library has an implementation of Levy for "alpha-stable distributions" and for "skew alpha-stable Distribution" which might provide some insight and provides something to test against and compare to: [http://bzr.savannah.gnu.org/lh/gsl/trunk/annotate/head:/randist/levy.c]
I'll continue to discuss with my friend, but I'm still not quite sure how to implement the failed tests for:
testQuantiles
testCumulativeProbabilities
testInverseCumulativeProbabilities
testConsistency
testSampling
Any feedback and insight/suggestions you might want to pass on would be very helpful.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463470#comment-13463470 ]
Andrew Waterman edited comment on MATH-460 at 9/26/12 1:18 PM:
---------------------------------------------------------------
Thanks for the help Phil! Do you think you could explain the following test? It seems like it should be overridden in my own test case, as I'm a bit confused about the set [250,250,250,250] for the variable "expected" below:
Link: https://github.com/apache/commons-math/blob/trunk/src/test/java/org/apache/commons/math3/distribution/RealDistributionAbstractTest.java#L290
public abstract class RealDistributionAbstractTest {
....
/**
* Test sampling
*/
@Test
public void testSampling() {
final int sampleSize = 1000;
distribution.reseedRandomGenerator(1000); // Use fixed seed
double[] sample = distribution.sample(sampleSize);
double[] quartiles = TestUtils.getDistributionQuartiles(distribution);
double[] expected = {250, 250, 250, 250};
long[] counts = new long[4];
for (int i = 0; i < sampleSize; i++) {
TestUtils.updateCounts(sample[i], counts, quartiles);
}
TestUtils.assertChiSquareAccept(expected, counts, 0.001);
}
was (Author: awaterma):
Thanks for the help Phil! Do you think you could explain the following test? It seems like it should be overridden in my own test case, as I'm a bit confused about the set [250,250,250,250] for the variable "expected" below:
public abstract class RealDistributionAbstractTest {
....
/**
* Test sampling
*/
@Test
public void testSampling() {
final int sampleSize = 1000;
distribution.reseedRandomGenerator(1000); // Use fixed seed
double[] sample = distribution.sample(sampleSize);
double[] quartiles = TestUtils.getDistributionQuartiles(distribution);
double[] expected = {250, 250, 250, 250};
long[] counts = new long[4];
for (int i = 0; i < sampleSize; i++) {
TestUtils.updateCounts(sample[i], counts, quartiles);
}
TestUtils.assertChiSquareAccept(expected, counts, 0.001);
}
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13463903#comment-13463903 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
Hi Giles, I've updated the comment to just point to the code for the test. For some reason, I had trouble with the monospace indicator for the quoted code.
I see now that the array, "{{expected { 250, 250, 250, 250}}} refers to the quantiles to be tested in the ChiSquare test: https://github.com/apache/commons-math/blob/trunk/src/test/java/org/apache/commons/math3/TestUtils.java#L442.
This test fails in my implementation.
I still need to independently confirm my results for the other tests via R. I'm looking to use the continuous Levy implementation to verify:
http://en.wikibooks.org/wiki/R_Programming/Probability_Distributions#Levy
Although I'm having some issues getting the rmutil package to install on OSX for R 2.15 (64). Once I have this working, I will also create a Levy distribution testcase for your test package. By the way, an example testcase for the quantiles of a levy distribution in R would be really helpful. :)
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.2
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MATH-460) Levy Distribution
Posted by "Andrew Waterman (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13291829#comment-13291829 ]
Andrew Waterman commented on MATH-460:
--------------------------------------
Yes, please feel free to go ahead. As Sébastien noted, we were able to get what we needed from what's currently available in the Math library. My apologies for not updating the issue with this information earlier.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Assignee: Mikkel Meyer Andersen
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MATH-460) Levy Distribution
Posted by "Pavel Ryzhov (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pavel Ryzhov updated MATH-460:
------------------------------
Attachment: levy_math_460.patch
Levy distribution patch without tabs.
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.0
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (MATH-460) Levy Distribution
Posted by "Gilles (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MATH-460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gilles updated MATH-460:
------------------------
Fix Version/s: (was: 3.0)
3.1
> Levy Distribution
> -----------------
>
> Key: MATH-460
> URL: https://issues.apache.org/jira/browse/MATH-460
> Project: Commons Math
> Issue Type: New Feature
> Reporter: Pavel Ryzhov
> Priority: Minor
> Fix For: 3.1
>
> Attachments: levy_math_460.patch
>
>
> Pretty straightforward implementation of Levy Distribution (not Levy alpha-stable) according to http://en.wikipedia.org/wiki/Lévy_distribution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira