You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Mikkel Meyer Andersen <mi...@mikl.dk> on 2010/12/30 11:00:47 UTC

Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Hi,

This commit consists of new files, so I put
@version $Revision: $ $Date: $
in the top. I thought SVN would then auto update these, but after
checking out again to see if it worked, I see that it didn't. What
should I do to make it work? Fire a svn propXXX, use other variables,
or?

Cheers, Mikkel.

2010/12/30  <mi...@apache.org>:
> Author: mikl
> Date: Thu Dec 30 09:52:00 2010
> New Revision: 1053836
>
> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
> Log:
> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>
> Added:
>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>
> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
> ==============================================================================
> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
> @@ -0,0 +1,96 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements.  See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License.  You may obtain a copy of the License at
> + *
> + *      http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +package org.apache.commons.math.stat.inference;
> +
> +import org.apache.commons.math.MathException;
> +
> +/**
> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
> + *
> + * @version $Revision: $ $Date: $
> + */
> +public interface MannWhitneyUTest {
> +
> +    /**
> +     * Computes the <a
> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
> +     * U statistic</a> comparing mean for two independent samples possibly of
> +     * different length.
> +     * <p>
> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
> +     * the null hypothesis that the two independent samples has equal mean.
> +     * </p>
> +     * <p>
> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
> +     * samples would often have different length.
> +     * </p>
> +     * <p>
> +     * <strong>Preconditions</strong>:
> +     * <ul>
> +     * <li>All observations in the two samples are independent.</li>
> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
> +     * </ul>
> +     * </p>
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @return mannWhitneyU statistic
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     */
> +    double mannWhitneyU(final double[] x, final double[] y)
> +            throws IllegalArgumentException;
> +
> +    /**
> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
> +     * p-value</a>, associated with a <a
> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
> +     * U statistic</a> comparing mean for two independent samples.
> +     * <p>
> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
> +     * samples would often have different length.
> +     * </p>
> +     * <p>
> +     * <strong>Preconditions</strong>:
> +     * <ul>
> +     * <li>All observations in the two samples are independent.</li>
> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
> +     * </ul>
> +     * </p>
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @param exactPValue
> +     *            if the exact p-value is wanted (only works for x.length <= 30,
> +     *            if true and x.length > 30, this is ignored because
> +     *            calculations may take too long)
> +     * @return asymptotic p-value
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     * @throws MathException
> +     *             if an error occurs computing the p-value
> +     */
> +    double mannWhitneyUTest(final double[] x, final double[] y)
> +            throws IllegalArgumentException, MathException;
> +}
>
> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
> ==============================================================================
> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
> @@ -0,0 +1,199 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements.  See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License.  You may obtain a copy of the License at
> + *
> + *      http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +package org.apache.commons.math.stat.inference;
> +
> +import org.apache.commons.math.MathException;
> +import org.apache.commons.math.distribution.NormalDistributionImpl;
> +import org.apache.commons.math.stat.ranking.NaNStrategy;
> +import org.apache.commons.math.stat.ranking.NaturalRanking;
> +import org.apache.commons.math.stat.ranking.TiesStrategy;
> +import org.apache.commons.math.util.FastMath;
> +
> +/**
> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
> + * test).
> + *
> + * @version $Revision: $ $Date: $
> + */
> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
> +    private NaturalRanking naturalRanking;
> +
> +    /**
> +     * Create a test instance using where NaN's are left in place and ties get
> +     * the average of applicable ranks. Use this unless you are very sure of
> +     * what you are doing.
> +     */
> +    public MannWhitneyUTestImpl() {
> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
> +                TiesStrategy.AVERAGE);
> +    }
> +
> +    /**
> +     * Create a test instance using the given strategies for NaN's and ties.
> +     * Only use this if you are sure of what you are doing.
> +     *
> +     * @param nanStrategy
> +     *            specifies the strategy that should be used for Double.NaN's
> +     * @param tiesStrategy
> +     *            specifies the strategy that should be used for ties
> +     */
> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
> +            TiesStrategy tiesStrategy) {
> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
> +    }
> +
> +    /**
> +     * Ensures that the provided arrays fulfills the assumptions.
> +     *
> +     * @param x
> +     * @param y
> +     * @throws IllegalArgumentException
> +     *             if assumptions are not met
> +     */
> +    private void ensureDataConformance(final double[] x, final double[] y)
> +            throws IllegalArgumentException {
> +        if (x == null) {
> +            throw new IllegalArgumentException("x must not be null");
> +        }
> +
> +        if (y == null) {
> +            throw new IllegalArgumentException("y must not be null");
> +        }
> +
> +        if (x.length == 0) {
> +            throw new IllegalArgumentException(
> +                    "x must contain at least one element");
> +        }
> +
> +        if (y.length == 0) {
> +            throw new IllegalArgumentException(
> +                    "y must contain at least one element");
> +        }
> +    }
> +
> +    private double[] concatinateSamples(final double[] x, final double[] y) {
> +        final double[] z = new double[x.length + y.length];
> +
> +        System.arraycopy(x, 0, z, 0, x.length);
> +        System.arraycopy(y, 0, z, x.length, y.length);
> +
> +        return z;
> +    }
> +
> +    /**
> +     * {@inheritDoc}
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     */
> +    public double mannWhitneyU(final double[] x, final double[] y)
> +            throws IllegalArgumentException {
> +
> +        ensureDataConformance(x, y);
> +
> +        final double[] z = concatinateSamples(x, y);
> +        final double[] ranks = naturalRanking.rank(z);
> +
> +        double sumRankX = 0;
> +
> +        /*
> +         * The ranks for x is in the first x.length entries in ranks because x
> +         * is in the first x.length entries in z
> +         */
> +        for (int i = 0; i < x.length; ++i) {
> +            sumRankX += ranks[i];
> +        }
> +
> +        /*
> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
> +         * e.g. x, n1 is the number of observations in sample 1.
> +         */
> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
> +
> +        /*
> +         * It can be shown that U1 + U2 = n1 * n2
> +         */
> +        final double U2 = x.length * y.length - U1;
> +
> +        return FastMath.max(U1, U2);
> +    }
> +
> +    /**
> +     * @param Umin
> +     *            smallest Mann-Whitney U value
> +     * @param N
> +     *            number of subjects (corresponding to x.length)
> +     * @return two-sided asymptotic p-value
> +     * @throws MathException
> +     *             if an error occurs computing the p-value
> +     */
> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
> +            final int n2) throws MathException {
> +
> +        final int n1n2prod = n1 * n2;
> +
> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
> +        final double EU = (double) n1n2prod / 2.0;
> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
> +
> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
> +
> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
> +                0, 1);
> +
> +        return 2 * standardNormal.cumulativeProbability(z);
> +    }
> +
> +    /**
> +     * Ties give rise to biased variance at the moment. See e.g. <a
> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
> +     *
> +     * {@inheritDoc}
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @param exactPValue
> +     *            if the exact p-value is wanted (only for x.length <= 50)
> +     * @return asymptotic p-value (biased for samples with ties)
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     * @throws MathException
> +     *             if an error occurs computing the p-value
> +     */
> +    public double mannWhitneyUTest(final double[] x, final double[] y)
> +            throws IllegalArgumentException, MathException {
> +
> +        ensureDataConformance(x, y);
> +
> +        final double Umax = mannWhitneyU(x, y);
> +
> +        /*
> +         * It can be shown that U1 + U2 = n1 * n2
> +         */
> +        final double Umin = x.length * y.length - Umax;
> +
> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
> +    }
> +}
>
> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
> ==============================================================================
> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
> @@ -0,0 +1,107 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements.  See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License.  You may obtain a copy of the License at
> + *
> + *      http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +package org.apache.commons.math.stat.inference;
> +
> +import org.apache.commons.math.MathException;
> +
> +/**
> + * An interface for Wilcoxon signed-rank test.
> + *
> + * @version $Revision: $ $Date: $
> + */
> +public interface WilcoxonSignedRankTest {
> +
> +    /**
> +     * Computes the <a
> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
> +     * samples or repeated measurements on a single sample.
> +     * <p>
> +     * This statistic can be used to perform a Wilcoxon signed ranked test
> +     * evaluating the null hypothesis that the two related samples or repeated
> +     * measurements on a single sample has equal mean.
> +     * </p>
> +     * <p>
> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
> +     * </p>
> +     * <p>
> +     * <strong>Preconditions</strong>:
> +     * <ul>
> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
> +     * identical) and is symmetric about a common median.</li>
> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
> +     * ordered, so the comparisons greater than, less than, and equal to are
> +     * meaningful.</li>
> +     * </ul>
> +     * </p>
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @return wilcoxonSignedRank statistic
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     */
> +    double wilcoxonSignedRank(final double[] x, final double[] y)
> +            throws IllegalArgumentException;
> +
> +    /**
> +     * Returns the <i>observed significance level</i>, or <a href=
> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
> +     * p-value</a>, associated with a <a
> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
> +     * samples or repeated measurements on a single sample.
> +     * <p>
> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
> +     * </p>
> +     * <p>
> +     * <strong>Preconditions</strong>:
> +     * <ul>
> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
> +     * identical) and is symmetric about a common median.</li>
> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
> +     * ordered, so the comparisons greater than, less than, and equal to are
> +     * meaningful.</li>
> +     * </ul>
> +     * </p>
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @param exactPValue
> +     *            if the exact p-value is wanted (only works for x.length <= 30,
> +     *            if true and x.length > 30, this is ignored because
> +     *            calculations may take too long)
> +     * @return p-value
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     * @throws MathException
> +     *             if an error occurs computing the p-value
> +     */
> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
> +            boolean exactPValue) throws IllegalArgumentException,
> +            MathException;
> +}
>
> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
> ==============================================================================
> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
> @@ -0,0 +1,271 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements.  See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License.  You may obtain a copy of the License at
> + *
> + *      http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +package org.apache.commons.math.stat.inference;
> +
> +import org.apache.commons.math.MathException;
> +import org.apache.commons.math.distribution.NormalDistributionImpl;
> +import org.apache.commons.math.stat.ranking.NaNStrategy;
> +import org.apache.commons.math.stat.ranking.NaturalRanking;
> +import org.apache.commons.math.stat.ranking.TiesStrategy;
> +import org.apache.commons.math.util.FastMath;
> +
> +/**
> + * An implementation of the Wilcoxon signed-rank test.
> + *
> + * @version $Revision: $ $Date: $
> + */
> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
> +    private NaturalRanking naturalRanking;
> +
> +    /**
> +     * Create a test instance where NaN's are left in place and ties get
> +     * the average of applicable ranks. Use this unless you are very sure
> +     * of what you are doing.
> +     */
> +    public WilcoxonSignedRankTestImpl() {
> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
> +                TiesStrategy.AVERAGE);
> +    }
> +
> +    /**
> +     * Create a test instance using the given strategies for NaN's and ties.
> +     * Only use this if you are sure of what you are doing.
> +     *
> +     * @param nanStrategy
> +     *            specifies the strategy that should be used for Double.NaN's
> +     * @param tiesStrategy
> +     *            specifies the strategy that should be used for ties
> +     */
> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
> +            TiesStrategy tiesStrategy) {
> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
> +    }
> +
> +    /**
> +     * Ensures that the provided arrays fulfills the assumptions.
> +     *
> +     * @param x
> +     * @param y
> +     * @throws IllegalArgumentException
> +     *             if assumptions are not met
> +     */
> +    private void ensureDataConformance(final double[] x, final double[] y)
> +            throws IllegalArgumentException {
> +        if (x == null) {
> +            throw new IllegalArgumentException("x must not be null");
> +        }
> +
> +        if (y == null) {
> +            throw new IllegalArgumentException("y must not be null");
> +        }
> +
> +        if (x.length != y.length) {
> +            throw new IllegalArgumentException(
> +                    "x and y must contain the same number of elements");
> +        }
> +
> +        if (x.length == 0) {
> +            throw new IllegalArgumentException(
> +                    "x and y must contain at least one element");
> +        }
> +    }
> +
> +    /**
> +     * Calculates y[i] - x[i] for all i
> +     *
> +     * @param x
> +     * @param y
> +     * @throws IllegalArgumentException
> +     *             if assumptions are not met
> +     */
> +    private double[] calculateDifferences(final double[] x, final double[] y)
> +            throws IllegalArgumentException {
> +
> +        final double[] z = new double[x.length];
> +
> +        for (int i = 0; i < x.length; ++i) {
> +            z[i] = y[i] - x[i];
> +        }
> +
> +        return z;
> +    }
> +
> +    /**
> +     * Calculates |z[i]| for all i
> +     *
> +     * @param z
> +     * @throws IllegalArgumentException
> +     *             if assumptions are not met
> +     */
> +    private double[] calculateAbsoluteDifferences(final double[] z)
> +            throws IllegalArgumentException {
> +        if (z == null) {
> +            throw new IllegalArgumentException("z must not be null");
> +        }
> +
> +        if (z.length == 0) {
> +            throw new IllegalArgumentException(
> +                    "z must contain at least one element");
> +        }
> +
> +        final double[] zAbs = new double[z.length];
> +
> +        for (int i = 0; i < z.length; ++i) {
> +            zAbs[i] = FastMath.abs(z[i]);
> +        }
> +
> +        return zAbs;
> +    }
> +
> +    /**
> +     * {@inheritDoc}
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met
> +     */
> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
> +            throws IllegalArgumentException {
> +
> +        ensureDataConformance(x, y);
> +
> +        // throws IllegalArgumentException if x and y are not correctly
> +        // specified
> +        final double[] z = calculateDifferences(x, y);
> +        final double[] zAbs = calculateAbsoluteDifferences(z);
> +
> +        final double[] ranks = naturalRanking.rank(zAbs);
> +
> +        double Wplus = 0;
> +
> +        for (int i = 0; i < z.length; ++i) {
> +            if (z[i] > 0) {
> +                Wplus += ranks[i];
> +            }
> +        }
> +
> +        final int N = x.length;
> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
> +
> +        return FastMath.max(Wplus, Wminus);
> +    }
> +
> +    /**
> +     * Algorithm inspired by
> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
> +     * University of Amsterdam
> +     *
> +     * @param Wmax largest Wilcoxon signed rank value
> +     * @param N number of subjects (corresponding to x.length)
> +     * @return two-sided exact p-value
> +     */
> +    private double calculateExactPValue(final double Wmax, final int N) {
> +
> +        // Total number of outcomes (equal to 2^N but a lot faster)
> +        final int m = 1 << N;
> +
> +        int largerRankSums = 0;
> +
> +        for (int i = 0; i < m; ++i) {
> +            int rankSum = 0;
> +
> +            // Generate all possible rank sums
> +            for (int j = 0; j < N; ++j) {
> +
> +                // (i >> j) & 1 extract i's j-th bit from the right
> +                if (((i >> j) & 1) == 1) {
> +                    rankSum += j + 1;
> +                }
> +            }
> +
> +            if (rankSum >= Wmax) {
> +                ++largerRankSums;
> +            }
> +        }
> +
> +        /*
> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
> +         * with 2 to get the two-sided p-value
> +         */
> +        return 2 * ((double) largerRankSums) / ((double) m);
> +    }
> +
> +    /**
> +     * @param Wmin smallest Wilcoxon signed rank value
> +     * @param N number of subjects (corresponding to x.length)
> +     * @return two-sided asymptotic p-value
> +     * @throws MathException if an error occurs computing the p-value
> +     */
> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
> +
> +        final double ES = (double) (N * (N + 1)) / 4.0;
> +
> +        /* Same as (but saves computations):
> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
> +         */
> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
> +
> +        // - 0.5 is a continuity correction
> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
> +
> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
> +
> +        return 2*standardNormal.cumulativeProbability(z);
> +    }
> +
> +    /**
> +     * {@inheritDoc}
> +     *
> +     * @param x
> +     *            the first sample
> +     * @param y
> +     *            the second sample
> +     * @param exactPValue
> +     *            if the exact p-value is wanted (only for x.length <= 30)
> +     * @return p-value
> +     * @throws IllegalArgumentException
> +     *             if preconditions are not met or exact p-value is wanted for
> +     *             when x.length > 30
> +     * @throws MathException
> +     *             if an error occurs computing the p-value
> +     */
> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
> +            boolean exactPValue) throws IllegalArgumentException,
> +            MathException {
> +
> +        ensureDataConformance(x, y);
> +
> +        final int N = x.length;
> +        final double Wmax = wilcoxonSignedRank(x, y);
> +
> +        if (exactPValue && N > 30) {
> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
> +        }
> +
> +        if (exactPValue) {
> +            return calculateExactPValue(Wmax, N);
> +        } else {
> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
> +            return calculateAsymptoticPValue(Wmin, N);
> +        }
> +    }
> +}
>
> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
> ==============================================================================
> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
> @@ -0,0 +1,101 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements.  See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License.  You may obtain a copy of the License at
> + *
> + *      http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +package org.apache.commons.math.stat.inference;
> +
> +import junit.framework.TestCase;
> +
> +/**
> + * Test cases for the ChiSquareTestImpl class.
> + *
> + * @version $Revision: $ $Date: $
> + */
> +
> +public class MannWhitneyUTestTest extends TestCase {
> +
> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
> +
> +    public MannWhitneyUTestTest(String name) {
> +        super(name);
> +    }
> +
> +    public void testMannWhitneyUSimple() throws Exception {
> +        /* Target values computed using R version 2.11.1
> +         * x <- c(19, 22, 16, 29, 24)
> +         * y <- c(20, 11, 17, 12)
> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
> +         * W = 17, p-value = 0.08641
> +         */
> +        final double x[] = {19, 22, 16, 29, 24};
> +        final double y[] = {20, 11, 17, 12};
> +
> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
> +    }
> +
> +
> +    public void testMannWhitneyUInputValidation() throws Exception {
> +        /* Samples must be present, i.e. length > 0
> +         */
> +        try {
> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        /*
> +         * x and y is null
> +         */
> +        try {
> +            testStatistic.mannWhitneyUTest(null, null);
> +            fail("x and y is null (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.mannWhitneyUTest(null, null);
> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        /*
> +         * x or y is null
> +         */
> +        try {
> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
> +            fail("x is null (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
> +            fail("y is null (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +    }
> +}
>
> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
> ==============================================================================
> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
> @@ -0,0 +1,180 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements.  See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License.  You may obtain a copy of the License at
> + *
> + *      http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + */
> +package org.apache.commons.math.stat.inference;
> +
> +import junit.framework.TestCase;
> +
> +/**
> + * Test cases for the ChiSquareTestImpl class.
> + *
> + * @version $Revision: $ $Date: $
> + */
> +
> +public class WilcoxonSignedRankTestTest extends TestCase {
> +
> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
> +
> +    public WilcoxonSignedRankTestTest(String name) {
> +        super(name);
> +    }
> +
> +    public void testWilcoxonSignedRankSimple() throws Exception {
> +        /* Target values computed using R version 2.11.1
> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
> +         */
> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
> +
> +        /* EXACT:
> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
> +         * V = 40, p-value = 0.03906
> +         *
> +         * Corresponds to the value obtained in R.
> +         */
> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
> +
> +        /* ASYMPTOTIC:
> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
> +         * V = 40, p-value = 0.03815
> +         *
> +         * This is not entirely the same due to different corrects,
> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
> +         * and src/library/stats/R/wilcox.test.R in the R source
> +         */
> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
> +    }
> +
> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
> +        /*
> +         * Exact only for sample size <= 30
> +         */
> +        final double[] x1 = new double[30];
> +        final double[] x2 = new double[31];
> +        final double[] y1 = new double[30];
> +        final double[] y2 = new double[31];
> +        for (int i = 0; i < 30; ++i) {
> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
> +        }
> +
> +        // Exactly 30 is okay
> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        /* Samples must be present, i.e. length > 0
> +         */
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        /* Samples not same size, i.e. cannot be pairred
> +         */
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
> +            fail("x and y not same size (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        /*
> +         * x and y is null
> +         */
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
> +            fail("x and y is null (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        /*
> +         * x or y is null
> +         */
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
> +            fail("x is null (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
> +            fail("x is null (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
> +            fail("y is null (exact), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +
> +        try {
> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
> +            fail("y is null (asymptotic), IllegalArgumentException expected");
> +        } catch (IllegalArgumentException ex) {
> +            // expected
> +        }
> +    }
> +}
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Mikkel Meyer Andersen <mi...@mikl.dk>.
2010/12/30 Luc Maisonobe <Lu...@free.fr>:
> Le 30/12/2010 16:44, Mikkel Meyer Andersen a écrit :
>> 2010/12/30 Luc Maisonobe <Lu...@free.fr>:
>>> Le 30/12/2010 16:18, Luc Maisonobe a écrit :
>>>> Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
>>>>> Hi,
>>>>
>>>> Hi Mikkel,
>>>>
>>>>>
>>>>> This commit consists of new files, so I put
>>>>> @version $Revision: $ $Date: $
>>>>> in the top. I thought SVN would then auto update these, but after
>>>>> checking out again to see if it worked, I see that it didn't. What
>>>>> should I do to make it work? Fire a svn propXXX, use other variables,
>>>>> or?
>>>>
>>>> You should set some properties. The ones we traditionally use are:
>>>>  svn:eol-style=native
>>>>  svn:keywords="Author Date Id Revision"
>>>
>>> Well, in fact we have a policy not to use Author, but rather to put the
>>> author names in the contibutor and developpers sections of the pom.xml file.
>>>
>>> Luc
>>>
>>>>
>>>> One way to have these properties automatically set is to use a "config"
>>>> file in subversion configuration folder. I have put the file I use here:
>>>> <http://people.apache.org/~luc/config>.
>>>>
>>>> I think the file is used only for newly created files, so now that the
>>>> files are already in the repository, you should probably set the
>>>> properties manually.
>>>>
>>>> best regards,
>>>> Luc
>>>>
>>>>>
>>>>> Cheers, Mikkel.
>>>>>
>>>>> 2010/12/30  <mi...@apache.org>:
>>>>>> Author: mikl
>>>>>> Date: Thu Dec 30 09:52:00 2010
>>>>>> New Revision: 1053836
>>>>>>
>>>>>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>>>>>> Log:
>>>>>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>>>>>
>>>>>> Added:
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,96 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +
>>>>>> +/**
>>>>>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public interface MannWhitneyUTest {
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Computes the <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>>> +     * U statistic</a> comparing mean for two independent samples possibly of
>>>>>> +     * different length.
>>>>>> +     * <p>
>>>>>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>>>>>> +     * the null hypothesis that the two independent samples has equal mean.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>>> +     * samples would often have different length.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return mannWhitneyU statistic
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    double mannWhitneyU(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>>> +     * p-value</a>, associated with a <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>>> +     * U statistic</a> comparing mean for two independent samples.
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>>> +     * samples would often have different length.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>>> +     *            calculations may take too long)
>>>>>> +     * @return asymptotic p-value
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException, MathException;
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,199 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>>> +import org.apache.commons.math.util.FastMath;
>>>>>> +
>>>>>> +/**
>>>>>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>>>>>> + * test).
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>>>>>> +    private NaturalRanking naturalRanking;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance using where NaN's are left in place and ties get
>>>>>> +     * the average of applicable ranks. Use this unless you are very sure of
>>>>>> +     * what you are doing.
>>>>>> +     */
>>>>>> +    public MannWhitneyUTestImpl() {
>>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>>> +                TiesStrategy.AVERAGE);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>>> +     * Only use this if you are sure of what you are doing.
>>>>>> +     *
>>>>>> +     * @param nanStrategy
>>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>>> +     * @param tiesStrategy
>>>>>> +     *            specifies the strategy that should be used for ties
>>>>>> +     */
>>>>>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>>>>>> +            TiesStrategy tiesStrategy) {
>>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     * @param y
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +        if (x == null) {
>>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (y == null) {
>>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (x.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "x must contain at least one element");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (y.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "y must contain at least one element");
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>>>>>> +        final double[] z = new double[x.length + y.length];
>>>>>> +
>>>>>> +        System.arraycopy(x, 0, z, 0, x.length);
>>>>>> +        System.arraycopy(y, 0, z, x.length, y.length);
>>>>>> +
>>>>>> +        return z;
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    public double mannWhitneyU(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        final double[] z = concatinateSamples(x, y);
>>>>>> +        final double[] ranks = naturalRanking.rank(z);
>>>>>> +
>>>>>> +        double sumRankX = 0;
>>>>>> +
>>>>>> +        /*
>>>>>> +         * The ranks for x is in the first x.length entries in ranks because x
>>>>>> +         * is in the first x.length entries in z
>>>>>> +         */
>>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>>> +            sumRankX += ranks[i];
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>>>>>> +         * e.g. x, n1 is the number of observations in sample 1.
>>>>>> +         */
>>>>>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>>>>>> +
>>>>>> +        /*
>>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>>> +         */
>>>>>> +        final double U2 = x.length * y.length - U1;
>>>>>> +
>>>>>> +        return FastMath.max(U1, U2);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * @param Umin
>>>>>> +     *            smallest Mann-Whitney U value
>>>>>> +     * @param N
>>>>>> +     *            number of subjects (corresponding to x.length)
>>>>>> +     * @return two-sided asymptotic p-value
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>>>>>> +            final int n2) throws MathException {
>>>>>> +
>>>>>> +        final int n1n2prod = n1 * n2;
>>>>>> +
>>>>>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>>>>>> +        final double EU = (double) n1n2prod / 2.0;
>>>>>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>>>>>> +
>>>>>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>>>>>> +
>>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>>>>>> +                0, 1);
>>>>>> +
>>>>>> +        return 2 * standardNormal.cumulativeProbability(z);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>>>>>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>>>>>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>>>>>> +     *
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>>>>>> +     * @return asymptotic p-value (biased for samples with ties)
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException, MathException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        final double Umax = mannWhitneyU(x, y);
>>>>>> +
>>>>>> +        /*
>>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>>> +         */
>>>>>> +        final double Umin = x.length * y.length - Umax;
>>>>>> +
>>>>>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,107 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +
>>>>>> +/**
>>>>>> + * An interface for Wilcoxon signed-rank test.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public interface WilcoxonSignedRankTest {
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Computes the <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>>> +     * samples or repeated measurements on a single sample.
>>>>>> +     * <p>
>>>>>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>>>>>> +     * evaluating the null hypothesis that the two related samples or repeated
>>>>>> +     * measurements on a single sample has equal mean.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>>> +     * meaningful.</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return wilcoxonSignedRank statistic
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Returns the <i>observed significance level</i>, or <a href=
>>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>>> +     * p-value</a>, associated with a <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>>> +     * samples or repeated measurements on a single sample.
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>>> +     * meaningful.</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>>> +     *            calculations may take too long)
>>>>>> +     * @return p-value
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>>> +            MathException;
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,271 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>>> +import org.apache.commons.math.util.FastMath;
>>>>>> +
>>>>>> +/**
>>>>>> + * An implementation of the Wilcoxon signed-rank test.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>>>>>> +    private NaturalRanking naturalRanking;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance where NaN's are left in place and ties get
>>>>>> +     * the average of applicable ranks. Use this unless you are very sure
>>>>>> +     * of what you are doing.
>>>>>> +     */
>>>>>> +    public WilcoxonSignedRankTestImpl() {
>>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>>> +                TiesStrategy.AVERAGE);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>>> +     * Only use this if you are sure of what you are doing.
>>>>>> +     *
>>>>>> +     * @param nanStrategy
>>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>>> +     * @param tiesStrategy
>>>>>> +     *            specifies the strategy that should be used for ties
>>>>>> +     */
>>>>>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>>>>>> +            TiesStrategy tiesStrategy) {
>>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     * @param y
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +        if (x == null) {
>>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (y == null) {
>>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (x.length != y.length) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "x and y must contain the same number of elements");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (x.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "x and y must contain at least one element");
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Calculates y[i] - x[i] for all i
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     * @param y
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +
>>>>>> +        final double[] z = new double[x.length];
>>>>>> +
>>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>>> +            z[i] = y[i] - x[i];
>>>>>> +        }
>>>>>> +
>>>>>> +        return z;
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Calculates |z[i]| for all i
>>>>>> +     *
>>>>>> +     * @param z
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +        if (z == null) {
>>>>>> +            throw new IllegalArgumentException("z must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (z.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "z must contain at least one element");
>>>>>> +        }
>>>>>> +
>>>>>> +        final double[] zAbs = new double[z.length];
>>>>>> +
>>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>>> +            zAbs[i] = FastMath.abs(z[i]);
>>>>>> +        }
>>>>>> +
>>>>>> +        return zAbs;
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        // throws IllegalArgumentException if x and y are not correctly
>>>>>> +        // specified
>>>>>> +        final double[] z = calculateDifferences(x, y);
>>>>>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>>>>>> +
>>>>>> +        final double[] ranks = naturalRanking.rank(zAbs);
>>>>>> +
>>>>>> +        double Wplus = 0;
>>>>>> +
>>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>>> +            if (z[i] > 0) {
>>>>>> +                Wplus += ranks[i];
>>>>>> +            }
>>>>>> +        }
>>>>>> +
>>>>>> +        final int N = x.length;
>>>>>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>>>>>> +
>>>>>> +        return FastMath.max(Wplus, Wminus);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Algorithm inspired by
>>>>>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>>>>>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>>>>>> +     * University of Amsterdam
>>>>>> +     *
>>>>>> +     * @param Wmax largest Wilcoxon signed rank value
>>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>>> +     * @return two-sided exact p-value
>>>>>> +     */
>>>>>> +    private double calculateExactPValue(final double Wmax, final int N) {
>>>>>> +
>>>>>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>>>>>> +        final int m = 1 << N;
>>>>>> +
>>>>>> +        int largerRankSums = 0;
>>>>>> +
>>>>>> +        for (int i = 0; i < m; ++i) {
>>>>>> +            int rankSum = 0;
>>>>>> +
>>>>>> +            // Generate all possible rank sums
>>>>>> +            for (int j = 0; j < N; ++j) {
>>>>>> +
>>>>>> +                // (i >> j) & 1 extract i's j-th bit from the right
>>>>>> +                if (((i >> j) & 1) == 1) {
>>>>>> +                    rankSum += j + 1;
>>>>>> +                }
>>>>>> +            }
>>>>>> +
>>>>>> +            if (rankSum >= Wmax) {
>>>>>> +                ++largerRankSums;
>>>>>> +            }
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>>>>>> +         * with 2 to get the two-sided p-value
>>>>>> +         */
>>>>>> +        return 2 * ((double) largerRankSums) / ((double) m);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * @param Wmin smallest Wilcoxon signed rank value
>>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>>> +     * @return two-sided asymptotic p-value
>>>>>> +     * @throws MathException if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>>>>>> +
>>>>>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>>>>>> +
>>>>>> +        /* Same as (but saves computations):
>>>>>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>>>>>> +         */
>>>>>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>>>>>> +
>>>>>> +        // - 0.5 is a continuity correction
>>>>>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>>>>>> +
>>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>>>>>> +
>>>>>> +        return 2*standardNormal.cumulativeProbability(z);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>>>>>> +     * @return p-value
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met or exact p-value is wanted for
>>>>>> +     *             when x.length > 30
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>>> +            MathException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        final int N = x.length;
>>>>>> +        final double Wmax = wilcoxonSignedRank(x, y);
>>>>>> +
>>>>>> +        if (exactPValue && N > 30) {
>>>>>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (exactPValue) {
>>>>>> +            return calculateExactPValue(Wmax, N);
>>>>>> +        } else {
>>>>>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>>>>>> +            return calculateAsymptoticPValue(Wmin, N);
>>>>>> +        }
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,101 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import junit.framework.TestCase;
>>>>>> +
>>>>>> +/**
>>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +
>>>>>> +public class MannWhitneyUTestTest extends TestCase {
>>>>>> +
>>>>>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>>>>>> +
>>>>>> +    public MannWhitneyUTestTest(String name) {
>>>>>> +        super(name);
>>>>>> +    }
>>>>>> +
>>>>>> +    public void testMannWhitneyUSimple() throws Exception {
>>>>>> +        /* Target values computed using R version 2.11.1
>>>>>> +         * x <- c(19, 22, 16, 29, 24)
>>>>>> +         * y <- c(20, 11, 17, 12)
>>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>>>>>> +         * W = 17, p-value = 0.08641
>>>>>> +         */
>>>>>> +        final double x[] = {19, 22, 16, 29, 24};
>>>>>> +        final double y[] = {20, 11, 17, 12};
>>>>>> +
>>>>>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>>>>>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>>>>>> +    }
>>>>>> +
>>>>>> +
>>>>>> +    public void testMannWhitneyUInputValidation() throws Exception {
>>>>>> +        /* Samples must be present, i.e. length > 0
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x and y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x or y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,180 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import junit.framework.TestCase;
>>>>>> +
>>>>>> +/**
>>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +
>>>>>> +public class WilcoxonSignedRankTestTest extends TestCase {
>>>>>> +
>>>>>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>>>>>> +
>>>>>> +    public WilcoxonSignedRankTestTest(String name) {
>>>>>> +        super(name);
>>>>>> +    }
>>>>>> +
>>>>>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>>>>>> +        /* Target values computed using R version 2.11.1
>>>>>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>>>>>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>>>>>> +         */
>>>>>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>>>>>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>>>>>> +
>>>>>> +        /* EXACT:
>>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>>>>>> +         * V = 40, p-value = 0.03906
>>>>>> +         *
>>>>>> +         * Corresponds to the value obtained in R.
>>>>>> +         */
>>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>>>>>> +
>>>>>> +        /* ASYMPTOTIC:
>>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>>>>>> +         * V = 40, p-value = 0.03815
>>>>>> +         *
>>>>>> +         * This is not entirely the same due to different corrects,
>>>>>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>>>>>> +         * and src/library/stats/R/wilcox.test.R in the R source
>>>>>> +         */
>>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>>>>>> +    }
>>>>>> +
>>>>>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>>>>>> +        /*
>>>>>> +         * Exact only for sample size <= 30
>>>>>> +         */
>>>>>> +        final double[] x1 = new double[30];
>>>>>> +        final double[] x2 = new double[31];
>>>>>> +        final double[] y1 = new double[30];
>>>>>> +        final double[] y2 = new double[31];
>>>>>> +        for (int i = 0; i < 30; ++i) {
>>>>>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>>>>>> +        }
>>>>>> +
>>>>>> +        // Exactly 30 is okay
>>>>>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>>>>>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /* Samples must be present, i.e. length > 0
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>>>>>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>>>>>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /* Samples not same size, i.e. cannot be pairred
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>>>>>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>>>>>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x and y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x or y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>>>>>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>>>>>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>> Hi Luc,
>>
>> Thanks for the help, including the config-file. I've manually changed
>> the properties for the newly created files in revision 1053918 using
>> svn propset svn:eol-style native
>> src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest*
>> etc.
>>
>> And now that I'm using a config-file similar to yours, new files
>> should be dealt with automatically.
>>
>> In regards to the pom.xml, then I should just make an entry for myself
>> in the developers-section, right?
>
> Yes, of course.
Thanks for the help with this practical stuff :-).

Cheers, Mikkel.
>
> Luc
>
>>
>> Cheers, Mikkel.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Luc Maisonobe <Lu...@free.fr>.
Le 30/12/2010 16:44, Mikkel Meyer Andersen a écrit :
> 2010/12/30 Luc Maisonobe <Lu...@free.fr>:
>> Le 30/12/2010 16:18, Luc Maisonobe a écrit :
>>> Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
>>>> Hi,
>>>
>>> Hi Mikkel,
>>>
>>>>
>>>> This commit consists of new files, so I put
>>>> @version $Revision: $ $Date: $
>>>> in the top. I thought SVN would then auto update these, but after
>>>> checking out again to see if it worked, I see that it didn't. What
>>>> should I do to make it work? Fire a svn propXXX, use other variables,
>>>> or?
>>>
>>> You should set some properties. The ones we traditionally use are:
>>>  svn:eol-style=native
>>>  svn:keywords="Author Date Id Revision"
>>
>> Well, in fact we have a policy not to use Author, but rather to put the
>> author names in the contibutor and developpers sections of the pom.xml file.
>>
>> Luc
>>
>>>
>>> One way to have these properties automatically set is to use a "config"
>>> file in subversion configuration folder. I have put the file I use here:
>>> <http://people.apache.org/~luc/config>.
>>>
>>> I think the file is used only for newly created files, so now that the
>>> files are already in the repository, you should probably set the
>>> properties manually.
>>>
>>> best regards,
>>> Luc
>>>
>>>>
>>>> Cheers, Mikkel.
>>>>
>>>> 2010/12/30  <mi...@apache.org>:
>>>>> Author: mikl
>>>>> Date: Thu Dec 30 09:52:00 2010
>>>>> New Revision: 1053836
>>>>>
>>>>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>>>>> Log:
>>>>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>>>>
>>>>> Added:
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,96 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +
>>>>> +/**
>>>>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public interface MannWhitneyUTest {
>>>>> +
>>>>> +    /**
>>>>> +     * Computes the <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>> +     * U statistic</a> comparing mean for two independent samples possibly of
>>>>> +     * different length.
>>>>> +     * <p>
>>>>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>>>>> +     * the null hypothesis that the two independent samples has equal mean.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>> +     * samples would often have different length.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return mannWhitneyU statistic
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    double mannWhitneyU(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException;
>>>>> +
>>>>> +    /**
>>>>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>> +     * p-value</a>, associated with a <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>> +     * U statistic</a> comparing mean for two independent samples.
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>> +     * samples would often have different length.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>> +     *            calculations may take too long)
>>>>> +     * @return asymptotic p-value
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException, MathException;
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,199 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>> +import org.apache.commons.math.util.FastMath;
>>>>> +
>>>>> +/**
>>>>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>>>>> + * test).
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>>>>> +    private NaturalRanking naturalRanking;
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance using where NaN's are left in place and ties get
>>>>> +     * the average of applicable ranks. Use this unless you are very sure of
>>>>> +     * what you are doing.
>>>>> +     */
>>>>> +    public MannWhitneyUTestImpl() {
>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>> +                TiesStrategy.AVERAGE);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>> +     * Only use this if you are sure of what you are doing.
>>>>> +     *
>>>>> +     * @param nanStrategy
>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>> +     * @param tiesStrategy
>>>>> +     *            specifies the strategy that should be used for ties
>>>>> +     */
>>>>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>>>>> +            TiesStrategy tiesStrategy) {
>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>> +     *
>>>>> +     * @param x
>>>>> +     * @param y
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +        if (x == null) {
>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (y == null) {
>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (x.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "x must contain at least one element");
>>>>> +        }
>>>>> +
>>>>> +        if (y.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "y must contain at least one element");
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>>>>> +        final double[] z = new double[x.length + y.length];
>>>>> +
>>>>> +        System.arraycopy(x, 0, z, 0, x.length);
>>>>> +        System.arraycopy(y, 0, z, x.length, y.length);
>>>>> +
>>>>> +        return z;
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    public double mannWhitneyU(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        final double[] z = concatinateSamples(x, y);
>>>>> +        final double[] ranks = naturalRanking.rank(z);
>>>>> +
>>>>> +        double sumRankX = 0;
>>>>> +
>>>>> +        /*
>>>>> +         * The ranks for x is in the first x.length entries in ranks because x
>>>>> +         * is in the first x.length entries in z
>>>>> +         */
>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>> +            sumRankX += ranks[i];
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>>>>> +         * e.g. x, n1 is the number of observations in sample 1.
>>>>> +         */
>>>>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>>>>> +
>>>>> +        /*
>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>> +         */
>>>>> +        final double U2 = x.length * y.length - U1;
>>>>> +
>>>>> +        return FastMath.max(U1, U2);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * @param Umin
>>>>> +     *            smallest Mann-Whitney U value
>>>>> +     * @param N
>>>>> +     *            number of subjects (corresponding to x.length)
>>>>> +     * @return two-sided asymptotic p-value
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>>>>> +            final int n2) throws MathException {
>>>>> +
>>>>> +        final int n1n2prod = n1 * n2;
>>>>> +
>>>>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>>>>> +        final double EU = (double) n1n2prod / 2.0;
>>>>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>>>>> +
>>>>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>>>>> +
>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>>>>> +                0, 1);
>>>>> +
>>>>> +        return 2 * standardNormal.cumulativeProbability(z);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>>>>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>>>>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>>>>> +     *
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>>>>> +     * @return asymptotic p-value (biased for samples with ties)
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException, MathException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        final double Umax = mannWhitneyU(x, y);
>>>>> +
>>>>> +        /*
>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>> +         */
>>>>> +        final double Umin = x.length * y.length - Umax;
>>>>> +
>>>>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>>>>> +    }
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,107 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +
>>>>> +/**
>>>>> + * An interface for Wilcoxon signed-rank test.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public interface WilcoxonSignedRankTest {
>>>>> +
>>>>> +    /**
>>>>> +     * Computes the <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>> +     * samples or repeated measurements on a single sample.
>>>>> +     * <p>
>>>>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>>>>> +     * evaluating the null hypothesis that the two related samples or repeated
>>>>> +     * measurements on a single sample has equal mean.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>> +     * meaningful.</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return wilcoxonSignedRank statistic
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException;
>>>>> +
>>>>> +    /**
>>>>> +     * Returns the <i>observed significance level</i>, or <a href=
>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>> +     * p-value</a>, associated with a <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>> +     * samples or repeated measurements on a single sample.
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>> +     * meaningful.</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>> +     *            calculations may take too long)
>>>>> +     * @return p-value
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>> +            MathException;
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,271 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>> +import org.apache.commons.math.util.FastMath;
>>>>> +
>>>>> +/**
>>>>> + * An implementation of the Wilcoxon signed-rank test.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>>>>> +    private NaturalRanking naturalRanking;
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance where NaN's are left in place and ties get
>>>>> +     * the average of applicable ranks. Use this unless you are very sure
>>>>> +     * of what you are doing.
>>>>> +     */
>>>>> +    public WilcoxonSignedRankTestImpl() {
>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>> +                TiesStrategy.AVERAGE);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>> +     * Only use this if you are sure of what you are doing.
>>>>> +     *
>>>>> +     * @param nanStrategy
>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>> +     * @param tiesStrategy
>>>>> +     *            specifies the strategy that should be used for ties
>>>>> +     */
>>>>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>>>>> +            TiesStrategy tiesStrategy) {
>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>> +     *
>>>>> +     * @param x
>>>>> +     * @param y
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +        if (x == null) {
>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (y == null) {
>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (x.length != y.length) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "x and y must contain the same number of elements");
>>>>> +        }
>>>>> +
>>>>> +        if (x.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "x and y must contain at least one element");
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Calculates y[i] - x[i] for all i
>>>>> +     *
>>>>> +     * @param x
>>>>> +     * @param y
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +
>>>>> +        final double[] z = new double[x.length];
>>>>> +
>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>> +            z[i] = y[i] - x[i];
>>>>> +        }
>>>>> +
>>>>> +        return z;
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Calculates |z[i]| for all i
>>>>> +     *
>>>>> +     * @param z
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>>>>> +            throws IllegalArgumentException {
>>>>> +        if (z == null) {
>>>>> +            throw new IllegalArgumentException("z must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (z.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "z must contain at least one element");
>>>>> +        }
>>>>> +
>>>>> +        final double[] zAbs = new double[z.length];
>>>>> +
>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>> +            zAbs[i] = FastMath.abs(z[i]);
>>>>> +        }
>>>>> +
>>>>> +        return zAbs;
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        // throws IllegalArgumentException if x and y are not correctly
>>>>> +        // specified
>>>>> +        final double[] z = calculateDifferences(x, y);
>>>>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>>>>> +
>>>>> +        final double[] ranks = naturalRanking.rank(zAbs);
>>>>> +
>>>>> +        double Wplus = 0;
>>>>> +
>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>> +            if (z[i] > 0) {
>>>>> +                Wplus += ranks[i];
>>>>> +            }
>>>>> +        }
>>>>> +
>>>>> +        final int N = x.length;
>>>>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>>>>> +
>>>>> +        return FastMath.max(Wplus, Wminus);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Algorithm inspired by
>>>>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>>>>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>>>>> +     * University of Amsterdam
>>>>> +     *
>>>>> +     * @param Wmax largest Wilcoxon signed rank value
>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>> +     * @return two-sided exact p-value
>>>>> +     */
>>>>> +    private double calculateExactPValue(final double Wmax, final int N) {
>>>>> +
>>>>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>>>>> +        final int m = 1 << N;
>>>>> +
>>>>> +        int largerRankSums = 0;
>>>>> +
>>>>> +        for (int i = 0; i < m; ++i) {
>>>>> +            int rankSum = 0;
>>>>> +
>>>>> +            // Generate all possible rank sums
>>>>> +            for (int j = 0; j < N; ++j) {
>>>>> +
>>>>> +                // (i >> j) & 1 extract i's j-th bit from the right
>>>>> +                if (((i >> j) & 1) == 1) {
>>>>> +                    rankSum += j + 1;
>>>>> +                }
>>>>> +            }
>>>>> +
>>>>> +            if (rankSum >= Wmax) {
>>>>> +                ++largerRankSums;
>>>>> +            }
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>>>>> +         * with 2 to get the two-sided p-value
>>>>> +         */
>>>>> +        return 2 * ((double) largerRankSums) / ((double) m);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * @param Wmin smallest Wilcoxon signed rank value
>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>> +     * @return two-sided asymptotic p-value
>>>>> +     * @throws MathException if an error occurs computing the p-value
>>>>> +     */
>>>>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>>>>> +
>>>>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>>>>> +
>>>>> +        /* Same as (but saves computations):
>>>>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>>>>> +         */
>>>>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>>>>> +
>>>>> +        // - 0.5 is a continuity correction
>>>>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>>>>> +
>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>>>>> +
>>>>> +        return 2*standardNormal.cumulativeProbability(z);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>>>>> +     * @return p-value
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met or exact p-value is wanted for
>>>>> +     *             when x.length > 30
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>> +            MathException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        final int N = x.length;
>>>>> +        final double Wmax = wilcoxonSignedRank(x, y);
>>>>> +
>>>>> +        if (exactPValue && N > 30) {
>>>>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>>>>> +        }
>>>>> +
>>>>> +        if (exactPValue) {
>>>>> +            return calculateExactPValue(Wmax, N);
>>>>> +        } else {
>>>>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>>>>> +            return calculateAsymptoticPValue(Wmin, N);
>>>>> +        }
>>>>> +    }
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,101 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import junit.framework.TestCase;
>>>>> +
>>>>> +/**
>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +
>>>>> +public class MannWhitneyUTestTest extends TestCase {
>>>>> +
>>>>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>>>>> +
>>>>> +    public MannWhitneyUTestTest(String name) {
>>>>> +        super(name);
>>>>> +    }
>>>>> +
>>>>> +    public void testMannWhitneyUSimple() throws Exception {
>>>>> +        /* Target values computed using R version 2.11.1
>>>>> +         * x <- c(19, 22, 16, 29, 24)
>>>>> +         * y <- c(20, 11, 17, 12)
>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>>>>> +         * W = 17, p-value = 0.08641
>>>>> +         */
>>>>> +        final double x[] = {19, 22, 16, 29, 24};
>>>>> +        final double y[] = {20, 11, 17, 12};
>>>>> +
>>>>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>>>>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>>>>> +    }
>>>>> +
>>>>> +
>>>>> +    public void testMannWhitneyUInputValidation() throws Exception {
>>>>> +        /* Samples must be present, i.e. length > 0
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x and y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x or y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +    }
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,180 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import junit.framework.TestCase;
>>>>> +
>>>>> +/**
>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +
>>>>> +public class WilcoxonSignedRankTestTest extends TestCase {
>>>>> +
>>>>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>>>>> +
>>>>> +    public WilcoxonSignedRankTestTest(String name) {
>>>>> +        super(name);
>>>>> +    }
>>>>> +
>>>>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>>>>> +        /* Target values computed using R version 2.11.1
>>>>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>>>>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>>>>> +         */
>>>>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>>>>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>>>>> +
>>>>> +        /* EXACT:
>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>>>>> +         * V = 40, p-value = 0.03906
>>>>> +         *
>>>>> +         * Corresponds to the value obtained in R.
>>>>> +         */
>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>>>>> +
>>>>> +        /* ASYMPTOTIC:
>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>>>>> +         * V = 40, p-value = 0.03815
>>>>> +         *
>>>>> +         * This is not entirely the same due to different corrects,
>>>>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>>>>> +         * and src/library/stats/R/wilcox.test.R in the R source
>>>>> +         */
>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>>>>> +    }
>>>>> +
>>>>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>>>>> +        /*
>>>>> +         * Exact only for sample size <= 30
>>>>> +         */
>>>>> +        final double[] x1 = new double[30];
>>>>> +        final double[] x2 = new double[31];
>>>>> +        final double[] y1 = new double[30];
>>>>> +        final double[] y2 = new double[31];
>>>>> +        for (int i = 0; i < 30; ++i) {
>>>>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>>>>> +        }
>>>>> +
>>>>> +        // Exactly 30 is okay
>>>>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>>>>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /* Samples must be present, i.e. length > 0
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>>>>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>>>>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /* Samples not same size, i.e. cannot be pairred
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>>>>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>>>>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x and y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x or y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>>>>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>>>>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +    }
>>>>> +}
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
> 
> Hi Luc,
> 
> Thanks for the help, including the config-file. I've manually changed
> the properties for the newly created files in revision 1053918 using
> svn propset svn:eol-style native
> src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest*
> etc.
> 
> And now that I'm using a config-file similar to yours, new files
> should be dealt with automatically.
> 
> In regards to the pom.xml, then I should just make an entry for myself
> in the developers-section, right?

Yes, of course.

Luc

> 
> Cheers, Mikkel.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Luc Maisonobe <Lu...@free.fr>.
Le 30/12/2010 16:47, Mikkel Meyer Andersen a écrit :
> 2010/12/30 Mikkel Meyer Andersen <mi...@mikl.dk>:
>> 2010/12/30 Luc Maisonobe <Lu...@free.fr>:
>>> Le 30/12/2010 16:18, Luc Maisonobe a écrit :
>>>> Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
>>>>> Hi,
>>>>
>>>> Hi Mikkel,
>>>>
>>>>>
>>>>> This commit consists of new files, so I put
>>>>> @version $Revision: $ $Date: $
>>>>> in the top. I thought SVN would then auto update these, but after
>>>>> checking out again to see if it worked, I see that it didn't. What
>>>>> should I do to make it work? Fire a svn propXXX, use other variables,
>>>>> or?
>>>>
>>>> You should set some properties. The ones we traditionally use are:
>>>>  svn:eol-style=native
>>>>  svn:keywords="Author Date Id Revision"
>>>
>>> Well, in fact we have a policy not to use Author, but rather to put the
>>> author names in the contibutor and developpers sections of the pom.xml file.
>>>
>>> Luc
>>>
>>>>
>>>> One way to have these properties automatically set is to use a "config"
>>>> file in subversion configuration folder. I have put the file I use here:
>>>> <http://people.apache.org/~luc/config>.
>>>>
>>>> I think the file is used only for newly created files, so now that the
>>>> files are already in the repository, you should probably set the
>>>> properties manually.
>>>>
>>>> best regards,
>>>> Luc
>>>>
>>>>>
>>>>> Cheers, Mikkel.
>>>>>
>>>>> 2010/12/30  <mi...@apache.org>:
>>>>>> Author: mikl
>>>>>> Date: Thu Dec 30 09:52:00 2010
>>>>>> New Revision: 1053836
>>>>>>
>>>>>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>>>>>> Log:
>>>>>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>>>>>
>>>>>> Added:
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,96 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +
>>>>>> +/**
>>>>>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public interface MannWhitneyUTest {
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Computes the <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>>> +     * U statistic</a> comparing mean for two independent samples possibly of
>>>>>> +     * different length.
>>>>>> +     * <p>
>>>>>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>>>>>> +     * the null hypothesis that the two independent samples has equal mean.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>>> +     * samples would often have different length.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return mannWhitneyU statistic
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    double mannWhitneyU(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>>> +     * p-value</a>, associated with a <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>>> +     * U statistic</a> comparing mean for two independent samples.
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>>> +     * samples would often have different length.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>>> +     *            calculations may take too long)
>>>>>> +     * @return asymptotic p-value
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException, MathException;
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,199 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>>> +import org.apache.commons.math.util.FastMath;
>>>>>> +
>>>>>> +/**
>>>>>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>>>>>> + * test).
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>>>>>> +    private NaturalRanking naturalRanking;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance using where NaN's are left in place and ties get
>>>>>> +     * the average of applicable ranks. Use this unless you are very sure of
>>>>>> +     * what you are doing.
>>>>>> +     */
>>>>>> +    public MannWhitneyUTestImpl() {
>>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>>> +                TiesStrategy.AVERAGE);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>>> +     * Only use this if you are sure of what you are doing.
>>>>>> +     *
>>>>>> +     * @param nanStrategy
>>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>>> +     * @param tiesStrategy
>>>>>> +     *            specifies the strategy that should be used for ties
>>>>>> +     */
>>>>>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>>>>>> +            TiesStrategy tiesStrategy) {
>>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     * @param y
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +        if (x == null) {
>>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (y == null) {
>>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (x.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "x must contain at least one element");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (y.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "y must contain at least one element");
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>>>>>> +        final double[] z = new double[x.length + y.length];
>>>>>> +
>>>>>> +        System.arraycopy(x, 0, z, 0, x.length);
>>>>>> +        System.arraycopy(y, 0, z, x.length, y.length);
>>>>>> +
>>>>>> +        return z;
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    public double mannWhitneyU(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        final double[] z = concatinateSamples(x, y);
>>>>>> +        final double[] ranks = naturalRanking.rank(z);
>>>>>> +
>>>>>> +        double sumRankX = 0;
>>>>>> +
>>>>>> +        /*
>>>>>> +         * The ranks for x is in the first x.length entries in ranks because x
>>>>>> +         * is in the first x.length entries in z
>>>>>> +         */
>>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>>> +            sumRankX += ranks[i];
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>>>>>> +         * e.g. x, n1 is the number of observations in sample 1.
>>>>>> +         */
>>>>>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>>>>>> +
>>>>>> +        /*
>>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>>> +         */
>>>>>> +        final double U2 = x.length * y.length - U1;
>>>>>> +
>>>>>> +        return FastMath.max(U1, U2);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * @param Umin
>>>>>> +     *            smallest Mann-Whitney U value
>>>>>> +     * @param N
>>>>>> +     *            number of subjects (corresponding to x.length)
>>>>>> +     * @return two-sided asymptotic p-value
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>>>>>> +            final int n2) throws MathException {
>>>>>> +
>>>>>> +        final int n1n2prod = n1 * n2;
>>>>>> +
>>>>>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>>>>>> +        final double EU = (double) n1n2prod / 2.0;
>>>>>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>>>>>> +
>>>>>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>>>>>> +
>>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>>>>>> +                0, 1);
>>>>>> +
>>>>>> +        return 2 * standardNormal.cumulativeProbability(z);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>>>>>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>>>>>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>>>>>> +     *
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>>>>>> +     * @return asymptotic p-value (biased for samples with ties)
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException, MathException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        final double Umax = mannWhitneyU(x, y);
>>>>>> +
>>>>>> +        /*
>>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>>> +         */
>>>>>> +        final double Umin = x.length * y.length - Umax;
>>>>>> +
>>>>>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,107 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +
>>>>>> +/**
>>>>>> + * An interface for Wilcoxon signed-rank test.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public interface WilcoxonSignedRankTest {
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Computes the <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>>> +     * samples or repeated measurements on a single sample.
>>>>>> +     * <p>
>>>>>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>>>>>> +     * evaluating the null hypothesis that the two related samples or repeated
>>>>>> +     * measurements on a single sample has equal mean.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>>> +     * meaningful.</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return wilcoxonSignedRank statistic
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Returns the <i>observed significance level</i>, or <a href=
>>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>>> +     * p-value</a>, associated with a <a
>>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>>> +     * samples or repeated measurements on a single sample.
>>>>>> +     * <p>
>>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>>> +     * </p>
>>>>>> +     * <p>
>>>>>> +     * <strong>Preconditions</strong>:
>>>>>> +     * <ul>
>>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>>> +     * meaningful.</li>
>>>>>> +     * </ul>
>>>>>> +     * </p>
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>>> +     *            calculations may take too long)
>>>>>> +     * @return p-value
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>>> +            MathException;
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,271 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import org.apache.commons.math.MathException;
>>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>>> +import org.apache.commons.math.util.FastMath;
>>>>>> +
>>>>>> +/**
>>>>>> + * An implementation of the Wilcoxon signed-rank test.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>>>>>> +    private NaturalRanking naturalRanking;
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance where NaN's are left in place and ties get
>>>>>> +     * the average of applicable ranks. Use this unless you are very sure
>>>>>> +     * of what you are doing.
>>>>>> +     */
>>>>>> +    public WilcoxonSignedRankTestImpl() {
>>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>>> +                TiesStrategy.AVERAGE);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>>> +     * Only use this if you are sure of what you are doing.
>>>>>> +     *
>>>>>> +     * @param nanStrategy
>>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>>> +     * @param tiesStrategy
>>>>>> +     *            specifies the strategy that should be used for ties
>>>>>> +     */
>>>>>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>>>>>> +            TiesStrategy tiesStrategy) {
>>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     * @param y
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +        if (x == null) {
>>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (y == null) {
>>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (x.length != y.length) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "x and y must contain the same number of elements");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (x.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "x and y must contain at least one element");
>>>>>> +        }
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Calculates y[i] - x[i] for all i
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     * @param y
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +
>>>>>> +        final double[] z = new double[x.length];
>>>>>> +
>>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>>> +            z[i] = y[i] - x[i];
>>>>>> +        }
>>>>>> +
>>>>>> +        return z;
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Calculates |z[i]| for all i
>>>>>> +     *
>>>>>> +     * @param z
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if assumptions are not met
>>>>>> +     */
>>>>>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +        if (z == null) {
>>>>>> +            throw new IllegalArgumentException("z must not be null");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (z.length == 0) {
>>>>>> +            throw new IllegalArgumentException(
>>>>>> +                    "z must contain at least one element");
>>>>>> +        }
>>>>>> +
>>>>>> +        final double[] zAbs = new double[z.length];
>>>>>> +
>>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>>> +            zAbs[i] = FastMath.abs(z[i]);
>>>>>> +        }
>>>>>> +
>>>>>> +        return zAbs;
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met
>>>>>> +     */
>>>>>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>>> +            throws IllegalArgumentException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        // throws IllegalArgumentException if x and y are not correctly
>>>>>> +        // specified
>>>>>> +        final double[] z = calculateDifferences(x, y);
>>>>>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>>>>>> +
>>>>>> +        final double[] ranks = naturalRanking.rank(zAbs);
>>>>>> +
>>>>>> +        double Wplus = 0;
>>>>>> +
>>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>>> +            if (z[i] > 0) {
>>>>>> +                Wplus += ranks[i];
>>>>>> +            }
>>>>>> +        }
>>>>>> +
>>>>>> +        final int N = x.length;
>>>>>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>>>>>> +
>>>>>> +        return FastMath.max(Wplus, Wminus);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * Algorithm inspired by
>>>>>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>>>>>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>>>>>> +     * University of Amsterdam
>>>>>> +     *
>>>>>> +     * @param Wmax largest Wilcoxon signed rank value
>>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>>> +     * @return two-sided exact p-value
>>>>>> +     */
>>>>>> +    private double calculateExactPValue(final double Wmax, final int N) {
>>>>>> +
>>>>>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>>>>>> +        final int m = 1 << N;
>>>>>> +
>>>>>> +        int largerRankSums = 0;
>>>>>> +
>>>>>> +        for (int i = 0; i < m; ++i) {
>>>>>> +            int rankSum = 0;
>>>>>> +
>>>>>> +            // Generate all possible rank sums
>>>>>> +            for (int j = 0; j < N; ++j) {
>>>>>> +
>>>>>> +                // (i >> j) & 1 extract i's j-th bit from the right
>>>>>> +                if (((i >> j) & 1) == 1) {
>>>>>> +                    rankSum += j + 1;
>>>>>> +                }
>>>>>> +            }
>>>>>> +
>>>>>> +            if (rankSum >= Wmax) {
>>>>>> +                ++largerRankSums;
>>>>>> +            }
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>>>>>> +         * with 2 to get the two-sided p-value
>>>>>> +         */
>>>>>> +        return 2 * ((double) largerRankSums) / ((double) m);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * @param Wmin smallest Wilcoxon signed rank value
>>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>>> +     * @return two-sided asymptotic p-value
>>>>>> +     * @throws MathException if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>>>>>> +
>>>>>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>>>>>> +
>>>>>> +        /* Same as (but saves computations):
>>>>>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>>>>>> +         */
>>>>>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>>>>>> +
>>>>>> +        // - 0.5 is a continuity correction
>>>>>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>>>>>> +
>>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>>>>>> +
>>>>>> +        return 2*standardNormal.cumulativeProbability(z);
>>>>>> +    }
>>>>>> +
>>>>>> +    /**
>>>>>> +     * {@inheritDoc}
>>>>>> +     *
>>>>>> +     * @param x
>>>>>> +     *            the first sample
>>>>>> +     * @param y
>>>>>> +     *            the second sample
>>>>>> +     * @param exactPValue
>>>>>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>>>>>> +     * @return p-value
>>>>>> +     * @throws IllegalArgumentException
>>>>>> +     *             if preconditions are not met or exact p-value is wanted for
>>>>>> +     *             when x.length > 30
>>>>>> +     * @throws MathException
>>>>>> +     *             if an error occurs computing the p-value
>>>>>> +     */
>>>>>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>>> +            MathException {
>>>>>> +
>>>>>> +        ensureDataConformance(x, y);
>>>>>> +
>>>>>> +        final int N = x.length;
>>>>>> +        final double Wmax = wilcoxonSignedRank(x, y);
>>>>>> +
>>>>>> +        if (exactPValue && N > 30) {
>>>>>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>>>>>> +        }
>>>>>> +
>>>>>> +        if (exactPValue) {
>>>>>> +            return calculateExactPValue(Wmax, N);
>>>>>> +        } else {
>>>>>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>>>>>> +            return calculateAsymptoticPValue(Wmin, N);
>>>>>> +        }
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,101 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import junit.framework.TestCase;
>>>>>> +
>>>>>> +/**
>>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +
>>>>>> +public class MannWhitneyUTestTest extends TestCase {
>>>>>> +
>>>>>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>>>>>> +
>>>>>> +    public MannWhitneyUTestTest(String name) {
>>>>>> +        super(name);
>>>>>> +    }
>>>>>> +
>>>>>> +    public void testMannWhitneyUSimple() throws Exception {
>>>>>> +        /* Target values computed using R version 2.11.1
>>>>>> +         * x <- c(19, 22, 16, 29, 24)
>>>>>> +         * y <- c(20, 11, 17, 12)
>>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>>>>>> +         * W = 17, p-value = 0.08641
>>>>>> +         */
>>>>>> +        final double x[] = {19, 22, 16, 29, 24};
>>>>>> +        final double y[] = {20, 11, 17, 12};
>>>>>> +
>>>>>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>>>>>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>>>>>> +    }
>>>>>> +
>>>>>> +
>>>>>> +    public void testMannWhitneyUInputValidation() throws Exception {
>>>>>> +        /* Samples must be present, i.e. length > 0
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x and y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x or y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>>>>>> ==============================================================================
>>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>>>>>> @@ -0,0 +1,180 @@
>>>>>> +/*
>>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>>> + * this work for additional information regarding copyright ownership.
>>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>>> + * the License.  You may obtain a copy of the License at
>>>>>> + *
>>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>>> + *
>>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>>> + * See the License for the specific language governing permissions and
>>>>>> + * limitations under the License.
>>>>>> + */
>>>>>> +package org.apache.commons.math.stat.inference;
>>>>>> +
>>>>>> +import junit.framework.TestCase;
>>>>>> +
>>>>>> +/**
>>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>>> + *
>>>>>> + * @version $Revision: $ $Date: $
>>>>>> + */
>>>>>> +
>>>>>> +public class WilcoxonSignedRankTestTest extends TestCase {
>>>>>> +
>>>>>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>>>>>> +
>>>>>> +    public WilcoxonSignedRankTestTest(String name) {
>>>>>> +        super(name);
>>>>>> +    }
>>>>>> +
>>>>>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>>>>>> +        /* Target values computed using R version 2.11.1
>>>>>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>>>>>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>>>>>> +         */
>>>>>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>>>>>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>>>>>> +
>>>>>> +        /* EXACT:
>>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>>>>>> +         * V = 40, p-value = 0.03906
>>>>>> +         *
>>>>>> +         * Corresponds to the value obtained in R.
>>>>>> +         */
>>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>>>>>> +
>>>>>> +        /* ASYMPTOTIC:
>>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>>>>>> +         * V = 40, p-value = 0.03815
>>>>>> +         *
>>>>>> +         * This is not entirely the same due to different corrects,
>>>>>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>>>>>> +         * and src/library/stats/R/wilcox.test.R in the R source
>>>>>> +         */
>>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>>>>>> +    }
>>>>>> +
>>>>>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>>>>>> +        /*
>>>>>> +         * Exact only for sample size <= 30
>>>>>> +         */
>>>>>> +        final double[] x1 = new double[30];
>>>>>> +        final double[] x2 = new double[31];
>>>>>> +        final double[] y1 = new double[30];
>>>>>> +        final double[] y2 = new double[31];
>>>>>> +        for (int i = 0; i < 30; ++i) {
>>>>>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>>>>>> +        }
>>>>>> +
>>>>>> +        // Exactly 30 is okay
>>>>>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>>>>>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /* Samples must be present, i.e. length > 0
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>>>>>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>>>>>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /* Samples not same size, i.e. cannot be pairred
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>>>>>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>>>>>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x and y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        /*
>>>>>> +         * x or y is null
>>>>>> +         */
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>>>>>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +
>>>>>> +        try {
>>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>>>>>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>>>>>> +        } catch (IllegalArgumentException ex) {
>>>>>> +            // expected
>>>>>> +        }
>>>>>> +    }
>>>>>> +}
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>> Hi Luc,
>>
>> Thanks for the help, including the config-file. I've manually changed
>> the properties for the newly created files in revision 1053918 using
>> svn propset svn:eol-style native
>> src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest*
>> etc.
>>
>> And now that I'm using a config-file similar to yours, new files
>> should be dealt with automatically.
>>
>> In regards to the pom.xml, then I should just make an entry for myself
>> in the developers-section, right?
> Both in the trunk and in the 2.2-branch?

Yes, you are here from some times and you contributed to both.

Luc

>>
>> Cheers, Mikkel.
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Mikkel Meyer Andersen <mi...@mikl.dk>.
2010/12/30 Mikkel Meyer Andersen <mi...@mikl.dk>:
> 2010/12/30 Luc Maisonobe <Lu...@free.fr>:
>> Le 30/12/2010 16:18, Luc Maisonobe a écrit :
>>> Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
>>>> Hi,
>>>
>>> Hi Mikkel,
>>>
>>>>
>>>> This commit consists of new files, so I put
>>>> @version $Revision: $ $Date: $
>>>> in the top. I thought SVN would then auto update these, but after
>>>> checking out again to see if it worked, I see that it didn't. What
>>>> should I do to make it work? Fire a svn propXXX, use other variables,
>>>> or?
>>>
>>> You should set some properties. The ones we traditionally use are:
>>>  svn:eol-style=native
>>>  svn:keywords="Author Date Id Revision"
>>
>> Well, in fact we have a policy not to use Author, but rather to put the
>> author names in the contibutor and developpers sections of the pom.xml file.
>>
>> Luc
>>
>>>
>>> One way to have these properties automatically set is to use a "config"
>>> file in subversion configuration folder. I have put the file I use here:
>>> <http://people.apache.org/~luc/config>.
>>>
>>> I think the file is used only for newly created files, so now that the
>>> files are already in the repository, you should probably set the
>>> properties manually.
>>>
>>> best regards,
>>> Luc
>>>
>>>>
>>>> Cheers, Mikkel.
>>>>
>>>> 2010/12/30  <mi...@apache.org>:
>>>>> Author: mikl
>>>>> Date: Thu Dec 30 09:52:00 2010
>>>>> New Revision: 1053836
>>>>>
>>>>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>>>>> Log:
>>>>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>>>>
>>>>> Added:
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,96 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +
>>>>> +/**
>>>>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public interface MannWhitneyUTest {
>>>>> +
>>>>> +    /**
>>>>> +     * Computes the <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>> +     * U statistic</a> comparing mean for two independent samples possibly of
>>>>> +     * different length.
>>>>> +     * <p>
>>>>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>>>>> +     * the null hypothesis that the two independent samples has equal mean.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>> +     * samples would often have different length.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return mannWhitneyU statistic
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    double mannWhitneyU(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException;
>>>>> +
>>>>> +    /**
>>>>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>> +     * p-value</a>, associated with a <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>>> +     * U statistic</a> comparing mean for two independent samples.
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>>> +     * samples would often have different length.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>All observations in the two samples are independent.</li>
>>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>> +     *            calculations may take too long)
>>>>> +     * @return asymptotic p-value
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException, MathException;
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,199 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>> +import org.apache.commons.math.util.FastMath;
>>>>> +
>>>>> +/**
>>>>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>>>>> + * test).
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>>>>> +    private NaturalRanking naturalRanking;
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance using where NaN's are left in place and ties get
>>>>> +     * the average of applicable ranks. Use this unless you are very sure of
>>>>> +     * what you are doing.
>>>>> +     */
>>>>> +    public MannWhitneyUTestImpl() {
>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>> +                TiesStrategy.AVERAGE);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>> +     * Only use this if you are sure of what you are doing.
>>>>> +     *
>>>>> +     * @param nanStrategy
>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>> +     * @param tiesStrategy
>>>>> +     *            specifies the strategy that should be used for ties
>>>>> +     */
>>>>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>>>>> +            TiesStrategy tiesStrategy) {
>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>> +     *
>>>>> +     * @param x
>>>>> +     * @param y
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +        if (x == null) {
>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (y == null) {
>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (x.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "x must contain at least one element");
>>>>> +        }
>>>>> +
>>>>> +        if (y.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "y must contain at least one element");
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>>>>> +        final double[] z = new double[x.length + y.length];
>>>>> +
>>>>> +        System.arraycopy(x, 0, z, 0, x.length);
>>>>> +        System.arraycopy(y, 0, z, x.length, y.length);
>>>>> +
>>>>> +        return z;
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    public double mannWhitneyU(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        final double[] z = concatinateSamples(x, y);
>>>>> +        final double[] ranks = naturalRanking.rank(z);
>>>>> +
>>>>> +        double sumRankX = 0;
>>>>> +
>>>>> +        /*
>>>>> +         * The ranks for x is in the first x.length entries in ranks because x
>>>>> +         * is in the first x.length entries in z
>>>>> +         */
>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>> +            sumRankX += ranks[i];
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>>>>> +         * e.g. x, n1 is the number of observations in sample 1.
>>>>> +         */
>>>>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>>>>> +
>>>>> +        /*
>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>> +         */
>>>>> +        final double U2 = x.length * y.length - U1;
>>>>> +
>>>>> +        return FastMath.max(U1, U2);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * @param Umin
>>>>> +     *            smallest Mann-Whitney U value
>>>>> +     * @param N
>>>>> +     *            number of subjects (corresponding to x.length)
>>>>> +     * @return two-sided asymptotic p-value
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>>>>> +            final int n2) throws MathException {
>>>>> +
>>>>> +        final int n1n2prod = n1 * n2;
>>>>> +
>>>>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>>>>> +        final double EU = (double) n1n2prod / 2.0;
>>>>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>>>>> +
>>>>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>>>>> +
>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>>>>> +                0, 1);
>>>>> +
>>>>> +        return 2 * standardNormal.cumulativeProbability(z);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>>>>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>>>>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>>>>> +     *
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>>>>> +     * @return asymptotic p-value (biased for samples with ties)
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException, MathException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        final double Umax = mannWhitneyU(x, y);
>>>>> +
>>>>> +        /*
>>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>>> +         */
>>>>> +        final double Umin = x.length * y.length - Umax;
>>>>> +
>>>>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>>>>> +    }
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,107 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +
>>>>> +/**
>>>>> + * An interface for Wilcoxon signed-rank test.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public interface WilcoxonSignedRankTest {
>>>>> +
>>>>> +    /**
>>>>> +     * Computes the <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>> +     * samples or repeated measurements on a single sample.
>>>>> +     * <p>
>>>>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>>>>> +     * evaluating the null hypothesis that the two related samples or repeated
>>>>> +     * measurements on a single sample has equal mean.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>> +     * meaningful.</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return wilcoxonSignedRank statistic
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException;
>>>>> +
>>>>> +    /**
>>>>> +     * Returns the <i>observed significance level</i>, or <a href=
>>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>>> +     * p-value</a>, associated with a <a
>>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>>> +     * samples or repeated measurements on a single sample.
>>>>> +     * <p>
>>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>>> +     * </p>
>>>>> +     * <p>
>>>>> +     * <strong>Preconditions</strong>:
>>>>> +     * <ul>
>>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>>> +     * identical) and is symmetric about a common median.</li>
>>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>>> +     * meaningful.</li>
>>>>> +     * </ul>
>>>>> +     * </p>
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>>> +     *            if true and x.length > 30, this is ignored because
>>>>> +     *            calculations may take too long)
>>>>> +     * @return p-value
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>> +            MathException;
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,271 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import org.apache.commons.math.MathException;
>>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>>> +import org.apache.commons.math.util.FastMath;
>>>>> +
>>>>> +/**
>>>>> + * An implementation of the Wilcoxon signed-rank test.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>>>>> +    private NaturalRanking naturalRanking;
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance where NaN's are left in place and ties get
>>>>> +     * the average of applicable ranks. Use this unless you are very sure
>>>>> +     * of what you are doing.
>>>>> +     */
>>>>> +    public WilcoxonSignedRankTestImpl() {
>>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>>> +                TiesStrategy.AVERAGE);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>>> +     * Only use this if you are sure of what you are doing.
>>>>> +     *
>>>>> +     * @param nanStrategy
>>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>>> +     * @param tiesStrategy
>>>>> +     *            specifies the strategy that should be used for ties
>>>>> +     */
>>>>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>>>>> +            TiesStrategy tiesStrategy) {
>>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>>> +     *
>>>>> +     * @param x
>>>>> +     * @param y
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +        if (x == null) {
>>>>> +            throw new IllegalArgumentException("x must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (y == null) {
>>>>> +            throw new IllegalArgumentException("y must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (x.length != y.length) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "x and y must contain the same number of elements");
>>>>> +        }
>>>>> +
>>>>> +        if (x.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "x and y must contain at least one element");
>>>>> +        }
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Calculates y[i] - x[i] for all i
>>>>> +     *
>>>>> +     * @param x
>>>>> +     * @param y
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +
>>>>> +        final double[] z = new double[x.length];
>>>>> +
>>>>> +        for (int i = 0; i < x.length; ++i) {
>>>>> +            z[i] = y[i] - x[i];
>>>>> +        }
>>>>> +
>>>>> +        return z;
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Calculates |z[i]| for all i
>>>>> +     *
>>>>> +     * @param z
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if assumptions are not met
>>>>> +     */
>>>>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>>>>> +            throws IllegalArgumentException {
>>>>> +        if (z == null) {
>>>>> +            throw new IllegalArgumentException("z must not be null");
>>>>> +        }
>>>>> +
>>>>> +        if (z.length == 0) {
>>>>> +            throw new IllegalArgumentException(
>>>>> +                    "z must contain at least one element");
>>>>> +        }
>>>>> +
>>>>> +        final double[] zAbs = new double[z.length];
>>>>> +
>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>> +            zAbs[i] = FastMath.abs(z[i]);
>>>>> +        }
>>>>> +
>>>>> +        return zAbs;
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met
>>>>> +     */
>>>>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>>>>> +            throws IllegalArgumentException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        // throws IllegalArgumentException if x and y are not correctly
>>>>> +        // specified
>>>>> +        final double[] z = calculateDifferences(x, y);
>>>>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>>>>> +
>>>>> +        final double[] ranks = naturalRanking.rank(zAbs);
>>>>> +
>>>>> +        double Wplus = 0;
>>>>> +
>>>>> +        for (int i = 0; i < z.length; ++i) {
>>>>> +            if (z[i] > 0) {
>>>>> +                Wplus += ranks[i];
>>>>> +            }
>>>>> +        }
>>>>> +
>>>>> +        final int N = x.length;
>>>>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>>>>> +
>>>>> +        return FastMath.max(Wplus, Wminus);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * Algorithm inspired by
>>>>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>>>>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>>>>> +     * University of Amsterdam
>>>>> +     *
>>>>> +     * @param Wmax largest Wilcoxon signed rank value
>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>> +     * @return two-sided exact p-value
>>>>> +     */
>>>>> +    private double calculateExactPValue(final double Wmax, final int N) {
>>>>> +
>>>>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>>>>> +        final int m = 1 << N;
>>>>> +
>>>>> +        int largerRankSums = 0;
>>>>> +
>>>>> +        for (int i = 0; i < m; ++i) {
>>>>> +            int rankSum = 0;
>>>>> +
>>>>> +            // Generate all possible rank sums
>>>>> +            for (int j = 0; j < N; ++j) {
>>>>> +
>>>>> +                // (i >> j) & 1 extract i's j-th bit from the right
>>>>> +                if (((i >> j) & 1) == 1) {
>>>>> +                    rankSum += j + 1;
>>>>> +                }
>>>>> +            }
>>>>> +
>>>>> +            if (rankSum >= Wmax) {
>>>>> +                ++largerRankSums;
>>>>> +            }
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>>>>> +         * with 2 to get the two-sided p-value
>>>>> +         */
>>>>> +        return 2 * ((double) largerRankSums) / ((double) m);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * @param Wmin smallest Wilcoxon signed rank value
>>>>> +     * @param N number of subjects (corresponding to x.length)
>>>>> +     * @return two-sided asymptotic p-value
>>>>> +     * @throws MathException if an error occurs computing the p-value
>>>>> +     */
>>>>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>>>>> +
>>>>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>>>>> +
>>>>> +        /* Same as (but saves computations):
>>>>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>>>>> +         */
>>>>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>>>>> +
>>>>> +        // - 0.5 is a continuity correction
>>>>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>>>>> +
>>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>>>>> +
>>>>> +        return 2*standardNormal.cumulativeProbability(z);
>>>>> +    }
>>>>> +
>>>>> +    /**
>>>>> +     * {@inheritDoc}
>>>>> +     *
>>>>> +     * @param x
>>>>> +     *            the first sample
>>>>> +     * @param y
>>>>> +     *            the second sample
>>>>> +     * @param exactPValue
>>>>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>>>>> +     * @return p-value
>>>>> +     * @throws IllegalArgumentException
>>>>> +     *             if preconditions are not met or exact p-value is wanted for
>>>>> +     *             when x.length > 30
>>>>> +     * @throws MathException
>>>>> +     *             if an error occurs computing the p-value
>>>>> +     */
>>>>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>>> +            MathException {
>>>>> +
>>>>> +        ensureDataConformance(x, y);
>>>>> +
>>>>> +        final int N = x.length;
>>>>> +        final double Wmax = wilcoxonSignedRank(x, y);
>>>>> +
>>>>> +        if (exactPValue && N > 30) {
>>>>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>>>>> +        }
>>>>> +
>>>>> +        if (exactPValue) {
>>>>> +            return calculateExactPValue(Wmax, N);
>>>>> +        } else {
>>>>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>>>>> +            return calculateAsymptoticPValue(Wmin, N);
>>>>> +        }
>>>>> +    }
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,101 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import junit.framework.TestCase;
>>>>> +
>>>>> +/**
>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +
>>>>> +public class MannWhitneyUTestTest extends TestCase {
>>>>> +
>>>>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>>>>> +
>>>>> +    public MannWhitneyUTestTest(String name) {
>>>>> +        super(name);
>>>>> +    }
>>>>> +
>>>>> +    public void testMannWhitneyUSimple() throws Exception {
>>>>> +        /* Target values computed using R version 2.11.1
>>>>> +         * x <- c(19, 22, 16, 29, 24)
>>>>> +         * y <- c(20, 11, 17, 12)
>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>>>>> +         * W = 17, p-value = 0.08641
>>>>> +         */
>>>>> +        final double x[] = {19, 22, 16, 29, 24};
>>>>> +        final double y[] = {20, 11, 17, 12};
>>>>> +
>>>>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>>>>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>>>>> +    }
>>>>> +
>>>>> +
>>>>> +    public void testMannWhitneyUInputValidation() throws Exception {
>>>>> +        /* Samples must be present, i.e. length > 0
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x and y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x or y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +    }
>>>>> +}
>>>>>
>>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>>>>> ==============================================================================
>>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>>>>> @@ -0,0 +1,180 @@
>>>>> +/*
>>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>>> + * this work for additional information regarding copyright ownership.
>>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>>> + * (the "License"); you may not use this file except in compliance with
>>>>> + * the License.  You may obtain a copy of the License at
>>>>> + *
>>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>>> + *
>>>>> + * Unless required by applicable law or agreed to in writing, software
>>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>>> + * See the License for the specific language governing permissions and
>>>>> + * limitations under the License.
>>>>> + */
>>>>> +package org.apache.commons.math.stat.inference;
>>>>> +
>>>>> +import junit.framework.TestCase;
>>>>> +
>>>>> +/**
>>>>> + * Test cases for the ChiSquareTestImpl class.
>>>>> + *
>>>>> + * @version $Revision: $ $Date: $
>>>>> + */
>>>>> +
>>>>> +public class WilcoxonSignedRankTestTest extends TestCase {
>>>>> +
>>>>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>>>>> +
>>>>> +    public WilcoxonSignedRankTestTest(String name) {
>>>>> +        super(name);
>>>>> +    }
>>>>> +
>>>>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>>>>> +        /* Target values computed using R version 2.11.1
>>>>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>>>>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>>>>> +         */
>>>>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>>>>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>>>>> +
>>>>> +        /* EXACT:
>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>>>>> +         * V = 40, p-value = 0.03906
>>>>> +         *
>>>>> +         * Corresponds to the value obtained in R.
>>>>> +         */
>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>>>>> +
>>>>> +        /* ASYMPTOTIC:
>>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>>>>> +         * V = 40, p-value = 0.03815
>>>>> +         *
>>>>> +         * This is not entirely the same due to different corrects,
>>>>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>>>>> +         * and src/library/stats/R/wilcox.test.R in the R source
>>>>> +         */
>>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>>>>> +    }
>>>>> +
>>>>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>>>>> +        /*
>>>>> +         * Exact only for sample size <= 30
>>>>> +         */
>>>>> +        final double[] x1 = new double[30];
>>>>> +        final double[] x2 = new double[31];
>>>>> +        final double[] y1 = new double[30];
>>>>> +        final double[] y2 = new double[31];
>>>>> +        for (int i = 0; i < 30; ++i) {
>>>>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>>>>> +        }
>>>>> +
>>>>> +        // Exactly 30 is okay
>>>>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>>>>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /* Samples must be present, i.e. length > 0
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>>>>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>>>>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /* Samples not same size, i.e. cannot be pairred
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>>>>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>>>>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x and y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        /*
>>>>> +         * x or y is null
>>>>> +         */
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>>>>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +
>>>>> +        try {
>>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>>>>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>>>>> +        } catch (IllegalArgumentException ex) {
>>>>> +            // expected
>>>>> +        }
>>>>> +    }
>>>>> +}
>>>>>
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> Hi Luc,
>
> Thanks for the help, including the config-file. I've manually changed
> the properties for the newly created files in revision 1053918 using
> svn propset svn:eol-style native
> src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest*
> etc.
>
> And now that I'm using a config-file similar to yours, new files
> should be dealt with automatically.
>
> In regards to the pom.xml, then I should just make an entry for myself
> in the developers-section, right?
Both in the trunk and in the 2.2-branch?
>
> Cheers, Mikkel.
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Mikkel Meyer Andersen <mi...@mikl.dk>.
2010/12/30 Luc Maisonobe <Lu...@free.fr>:
> Le 30/12/2010 16:18, Luc Maisonobe a écrit :
>> Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
>>> Hi,
>>
>> Hi Mikkel,
>>
>>>
>>> This commit consists of new files, so I put
>>> @version $Revision: $ $Date: $
>>> in the top. I thought SVN would then auto update these, but after
>>> checking out again to see if it worked, I see that it didn't. What
>>> should I do to make it work? Fire a svn propXXX, use other variables,
>>> or?
>>
>> You should set some properties. The ones we traditionally use are:
>>  svn:eol-style=native
>>  svn:keywords="Author Date Id Revision"
>
> Well, in fact we have a policy not to use Author, but rather to put the
> author names in the contibutor and developpers sections of the pom.xml file.
>
> Luc
>
>>
>> One way to have these properties automatically set is to use a "config"
>> file in subversion configuration folder. I have put the file I use here:
>> <http://people.apache.org/~luc/config>.
>>
>> I think the file is used only for newly created files, so now that the
>> files are already in the repository, you should probably set the
>> properties manually.
>>
>> best regards,
>> Luc
>>
>>>
>>> Cheers, Mikkel.
>>>
>>> 2010/12/30  <mi...@apache.org>:
>>>> Author: mikl
>>>> Date: Thu Dec 30 09:52:00 2010
>>>> New Revision: 1053836
>>>>
>>>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>>>> Log:
>>>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>>>
>>>> Added:
>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>>
>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>>>> ==============================================================================
>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>>>> @@ -0,0 +1,96 @@
>>>> +/*
>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>> + * this work for additional information regarding copyright ownership.
>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>> + * (the "License"); you may not use this file except in compliance with
>>>> + * the License.  You may obtain a copy of the License at
>>>> + *
>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>> + *
>>>> + * Unless required by applicable law or agreed to in writing, software
>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> + * See the License for the specific language governing permissions and
>>>> + * limitations under the License.
>>>> + */
>>>> +package org.apache.commons.math.stat.inference;
>>>> +
>>>> +import org.apache.commons.math.MathException;
>>>> +
>>>> +/**
>>>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>>>> + *
>>>> + * @version $Revision: $ $Date: $
>>>> + */
>>>> +public interface MannWhitneyUTest {
>>>> +
>>>> +    /**
>>>> +     * Computes the <a
>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>> +     * U statistic</a> comparing mean for two independent samples possibly of
>>>> +     * different length.
>>>> +     * <p>
>>>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>>>> +     * the null hypothesis that the two independent samples has equal mean.
>>>> +     * </p>
>>>> +     * <p>
>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>> +     * samples would often have different length.
>>>> +     * </p>
>>>> +     * <p>
>>>> +     * <strong>Preconditions</strong>:
>>>> +     * <ul>
>>>> +     * <li>All observations in the two samples are independent.</li>
>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>> +     * </ul>
>>>> +     * </p>
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @return mannWhitneyU statistic
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     */
>>>> +    double mannWhitneyU(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException;
>>>> +
>>>> +    /**
>>>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>> +     * p-value</a>, associated with a <a
>>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>>> +     * U statistic</a> comparing mean for two independent samples.
>>>> +     * <p>
>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>>> +     * samples would often have different length.
>>>> +     * </p>
>>>> +     * <p>
>>>> +     * <strong>Preconditions</strong>:
>>>> +     * <ul>
>>>> +     * <li>All observations in the two samples are independent.</li>
>>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>>> +     * </ul>
>>>> +     * </p>
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @param exactPValue
>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>> +     *            if true and x.length > 30, this is ignored because
>>>> +     *            calculations may take too long)
>>>> +     * @return asymptotic p-value
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     * @throws MathException
>>>> +     *             if an error occurs computing the p-value
>>>> +     */
>>>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException, MathException;
>>>> +}
>>>>
>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>>>> ==============================================================================
>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>>>> @@ -0,0 +1,199 @@
>>>> +/*
>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>> + * this work for additional information regarding copyright ownership.
>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>> + * (the "License"); you may not use this file except in compliance with
>>>> + * the License.  You may obtain a copy of the License at
>>>> + *
>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>> + *
>>>> + * Unless required by applicable law or agreed to in writing, software
>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> + * See the License for the specific language governing permissions and
>>>> + * limitations under the License.
>>>> + */
>>>> +package org.apache.commons.math.stat.inference;
>>>> +
>>>> +import org.apache.commons.math.MathException;
>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>> +import org.apache.commons.math.util.FastMath;
>>>> +
>>>> +/**
>>>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>>>> + * test).
>>>> + *
>>>> + * @version $Revision: $ $Date: $
>>>> + */
>>>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>>>> +    private NaturalRanking naturalRanking;
>>>> +
>>>> +    /**
>>>> +     * Create a test instance using where NaN's are left in place and ties get
>>>> +     * the average of applicable ranks. Use this unless you are very sure of
>>>> +     * what you are doing.
>>>> +     */
>>>> +    public MannWhitneyUTestImpl() {
>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>> +                TiesStrategy.AVERAGE);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>> +     * Only use this if you are sure of what you are doing.
>>>> +     *
>>>> +     * @param nanStrategy
>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>> +     * @param tiesStrategy
>>>> +     *            specifies the strategy that should be used for ties
>>>> +     */
>>>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>>>> +            TiesStrategy tiesStrategy) {
>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>> +     *
>>>> +     * @param x
>>>> +     * @param y
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if assumptions are not met
>>>> +     */
>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException {
>>>> +        if (x == null) {
>>>> +            throw new IllegalArgumentException("x must not be null");
>>>> +        }
>>>> +
>>>> +        if (y == null) {
>>>> +            throw new IllegalArgumentException("y must not be null");
>>>> +        }
>>>> +
>>>> +        if (x.length == 0) {
>>>> +            throw new IllegalArgumentException(
>>>> +                    "x must contain at least one element");
>>>> +        }
>>>> +
>>>> +        if (y.length == 0) {
>>>> +            throw new IllegalArgumentException(
>>>> +                    "y must contain at least one element");
>>>> +        }
>>>> +    }
>>>> +
>>>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>>>> +        final double[] z = new double[x.length + y.length];
>>>> +
>>>> +        System.arraycopy(x, 0, z, 0, x.length);
>>>> +        System.arraycopy(y, 0, z, x.length, y.length);
>>>> +
>>>> +        return z;
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * {@inheritDoc}
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     */
>>>> +    public double mannWhitneyU(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException {
>>>> +
>>>> +        ensureDataConformance(x, y);
>>>> +
>>>> +        final double[] z = concatinateSamples(x, y);
>>>> +        final double[] ranks = naturalRanking.rank(z);
>>>> +
>>>> +        double sumRankX = 0;
>>>> +
>>>> +        /*
>>>> +         * The ranks for x is in the first x.length entries in ranks because x
>>>> +         * is in the first x.length entries in z
>>>> +         */
>>>> +        for (int i = 0; i < x.length; ++i) {
>>>> +            sumRankX += ranks[i];
>>>> +        }
>>>> +
>>>> +        /*
>>>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>>>> +         * e.g. x, n1 is the number of observations in sample 1.
>>>> +         */
>>>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>>>> +
>>>> +        /*
>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>> +         */
>>>> +        final double U2 = x.length * y.length - U1;
>>>> +
>>>> +        return FastMath.max(U1, U2);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * @param Umin
>>>> +     *            smallest Mann-Whitney U value
>>>> +     * @param N
>>>> +     *            number of subjects (corresponding to x.length)
>>>> +     * @return two-sided asymptotic p-value
>>>> +     * @throws MathException
>>>> +     *             if an error occurs computing the p-value
>>>> +     */
>>>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>>>> +            final int n2) throws MathException {
>>>> +
>>>> +        final int n1n2prod = n1 * n2;
>>>> +
>>>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>>>> +        final double EU = (double) n1n2prod / 2.0;
>>>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>>>> +
>>>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>>>> +
>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>>>> +                0, 1);
>>>> +
>>>> +        return 2 * standardNormal.cumulativeProbability(z);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>>>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>>>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>>>> +     *
>>>> +     * {@inheritDoc}
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @param exactPValue
>>>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>>>> +     * @return asymptotic p-value (biased for samples with ties)
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     * @throws MathException
>>>> +     *             if an error occurs computing the p-value
>>>> +     */
>>>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException, MathException {
>>>> +
>>>> +        ensureDataConformance(x, y);
>>>> +
>>>> +        final double Umax = mannWhitneyU(x, y);
>>>> +
>>>> +        /*
>>>> +         * It can be shown that U1 + U2 = n1 * n2
>>>> +         */
>>>> +        final double Umin = x.length * y.length - Umax;
>>>> +
>>>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>>>> +    }
>>>> +}
>>>>
>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>>>> ==============================================================================
>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>>>> @@ -0,0 +1,107 @@
>>>> +/*
>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>> + * this work for additional information regarding copyright ownership.
>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>> + * (the "License"); you may not use this file except in compliance with
>>>> + * the License.  You may obtain a copy of the License at
>>>> + *
>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>> + *
>>>> + * Unless required by applicable law or agreed to in writing, software
>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> + * See the License for the specific language governing permissions and
>>>> + * limitations under the License.
>>>> + */
>>>> +package org.apache.commons.math.stat.inference;
>>>> +
>>>> +import org.apache.commons.math.MathException;
>>>> +
>>>> +/**
>>>> + * An interface for Wilcoxon signed-rank test.
>>>> + *
>>>> + * @version $Revision: $ $Date: $
>>>> + */
>>>> +public interface WilcoxonSignedRankTest {
>>>> +
>>>> +    /**
>>>> +     * Computes the <a
>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>> +     * samples or repeated measurements on a single sample.
>>>> +     * <p>
>>>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>>>> +     * evaluating the null hypothesis that the two related samples or repeated
>>>> +     * measurements on a single sample has equal mean.
>>>> +     * </p>
>>>> +     * <p>
>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>> +     * </p>
>>>> +     * <p>
>>>> +     * <strong>Preconditions</strong>:
>>>> +     * <ul>
>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>> +     * identical) and is symmetric about a common median.</li>
>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>> +     * meaningful.</li>
>>>> +     * </ul>
>>>> +     * </p>
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @return wilcoxonSignedRank statistic
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     */
>>>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException;
>>>> +
>>>> +    /**
>>>> +     * Returns the <i>observed significance level</i>, or <a href=
>>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>>> +     * p-value</a>, associated with a <a
>>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>>> +     * samples or repeated measurements on a single sample.
>>>> +     * <p>
>>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>>> +     * </p>
>>>> +     * <p>
>>>> +     * <strong>Preconditions</strong>:
>>>> +     * <ul>
>>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>>> +     * identical) and is symmetric about a common median.</li>
>>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>>> +     * meaningful.</li>
>>>> +     * </ul>
>>>> +     * </p>
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @param exactPValue
>>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>>> +     *            if true and x.length > 30, this is ignored because
>>>> +     *            calculations may take too long)
>>>> +     * @return p-value
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     * @throws MathException
>>>> +     *             if an error occurs computing the p-value
>>>> +     */
>>>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>> +            MathException;
>>>> +}
>>>>
>>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>>>> ==============================================================================
>>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>>>> @@ -0,0 +1,271 @@
>>>> +/*
>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>> + * this work for additional information regarding copyright ownership.
>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>> + * (the "License"); you may not use this file except in compliance with
>>>> + * the License.  You may obtain a copy of the License at
>>>> + *
>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>> + *
>>>> + * Unless required by applicable law or agreed to in writing, software
>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> + * See the License for the specific language governing permissions and
>>>> + * limitations under the License.
>>>> + */
>>>> +package org.apache.commons.math.stat.inference;
>>>> +
>>>> +import org.apache.commons.math.MathException;
>>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>>> +import org.apache.commons.math.util.FastMath;
>>>> +
>>>> +/**
>>>> + * An implementation of the Wilcoxon signed-rank test.
>>>> + *
>>>> + * @version $Revision: $ $Date: $
>>>> + */
>>>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>>>> +    private NaturalRanking naturalRanking;
>>>> +
>>>> +    /**
>>>> +     * Create a test instance where NaN's are left in place and ties get
>>>> +     * the average of applicable ranks. Use this unless you are very sure
>>>> +     * of what you are doing.
>>>> +     */
>>>> +    public WilcoxonSignedRankTestImpl() {
>>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>>> +                TiesStrategy.AVERAGE);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>>> +     * Only use this if you are sure of what you are doing.
>>>> +     *
>>>> +     * @param nanStrategy
>>>> +     *            specifies the strategy that should be used for Double.NaN's
>>>> +     * @param tiesStrategy
>>>> +     *            specifies the strategy that should be used for ties
>>>> +     */
>>>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>>>> +            TiesStrategy tiesStrategy) {
>>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>>> +     *
>>>> +     * @param x
>>>> +     * @param y
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if assumptions are not met
>>>> +     */
>>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException {
>>>> +        if (x == null) {
>>>> +            throw new IllegalArgumentException("x must not be null");
>>>> +        }
>>>> +
>>>> +        if (y == null) {
>>>> +            throw new IllegalArgumentException("y must not be null");
>>>> +        }
>>>> +
>>>> +        if (x.length != y.length) {
>>>> +            throw new IllegalArgumentException(
>>>> +                    "x and y must contain the same number of elements");
>>>> +        }
>>>> +
>>>> +        if (x.length == 0) {
>>>> +            throw new IllegalArgumentException(
>>>> +                    "x and y must contain at least one element");
>>>> +        }
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Calculates y[i] - x[i] for all i
>>>> +     *
>>>> +     * @param x
>>>> +     * @param y
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if assumptions are not met
>>>> +     */
>>>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException {
>>>> +
>>>> +        final double[] z = new double[x.length];
>>>> +
>>>> +        for (int i = 0; i < x.length; ++i) {
>>>> +            z[i] = y[i] - x[i];
>>>> +        }
>>>> +
>>>> +        return z;
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Calculates |z[i]| for all i
>>>> +     *
>>>> +     * @param z
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if assumptions are not met
>>>> +     */
>>>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>>>> +            throws IllegalArgumentException {
>>>> +        if (z == null) {
>>>> +            throw new IllegalArgumentException("z must not be null");
>>>> +        }
>>>> +
>>>> +        if (z.length == 0) {
>>>> +            throw new IllegalArgumentException(
>>>> +                    "z must contain at least one element");
>>>> +        }
>>>> +
>>>> +        final double[] zAbs = new double[z.length];
>>>> +
>>>> +        for (int i = 0; i < z.length; ++i) {
>>>> +            zAbs[i] = FastMath.abs(z[i]);
>>>> +        }
>>>> +
>>>> +        return zAbs;
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * {@inheritDoc}
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met
>>>> +     */
>>>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>>>> +            throws IllegalArgumentException {
>>>> +
>>>> +        ensureDataConformance(x, y);
>>>> +
>>>> +        // throws IllegalArgumentException if x and y are not correctly
>>>> +        // specified
>>>> +        final double[] z = calculateDifferences(x, y);
>>>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>>>> +
>>>> +        final double[] ranks = naturalRanking.rank(zAbs);
>>>> +
>>>> +        double Wplus = 0;
>>>> +
>>>> +        for (int i = 0; i < z.length; ++i) {
>>>> +            if (z[i] > 0) {
>>>> +                Wplus += ranks[i];
>>>> +            }
>>>> +        }
>>>> +
>>>> +        final int N = x.length;
>>>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>>>> +
>>>> +        return FastMath.max(Wplus, Wminus);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * Algorithm inspired by
>>>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>>>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>>>> +     * University of Amsterdam
>>>> +     *
>>>> +     * @param Wmax largest Wilcoxon signed rank value
>>>> +     * @param N number of subjects (corresponding to x.length)
>>>> +     * @return two-sided exact p-value
>>>> +     */
>>>> +    private double calculateExactPValue(final double Wmax, final int N) {
>>>> +
>>>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>>>> +        final int m = 1 << N;
>>>> +
>>>> +        int largerRankSums = 0;
>>>> +
>>>> +        for (int i = 0; i < m; ++i) {
>>>> +            int rankSum = 0;
>>>> +
>>>> +            // Generate all possible rank sums
>>>> +            for (int j = 0; j < N; ++j) {
>>>> +
>>>> +                // (i >> j) & 1 extract i's j-th bit from the right
>>>> +                if (((i >> j) & 1) == 1) {
>>>> +                    rankSum += j + 1;
>>>> +                }
>>>> +            }
>>>> +
>>>> +            if (rankSum >= Wmax) {
>>>> +                ++largerRankSums;
>>>> +            }
>>>> +        }
>>>> +
>>>> +        /*
>>>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>>>> +         * with 2 to get the two-sided p-value
>>>> +         */
>>>> +        return 2 * ((double) largerRankSums) / ((double) m);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * @param Wmin smallest Wilcoxon signed rank value
>>>> +     * @param N number of subjects (corresponding to x.length)
>>>> +     * @return two-sided asymptotic p-value
>>>> +     * @throws MathException if an error occurs computing the p-value
>>>> +     */
>>>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>>>> +
>>>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>>>> +
>>>> +        /* Same as (but saves computations):
>>>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>>>> +         */
>>>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>>>> +
>>>> +        // - 0.5 is a continuity correction
>>>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>>>> +
>>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>>>> +
>>>> +        return 2*standardNormal.cumulativeProbability(z);
>>>> +    }
>>>> +
>>>> +    /**
>>>> +     * {@inheritDoc}
>>>> +     *
>>>> +     * @param x
>>>> +     *            the first sample
>>>> +     * @param y
>>>> +     *            the second sample
>>>> +     * @param exactPValue
>>>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>>>> +     * @return p-value
>>>> +     * @throws IllegalArgumentException
>>>> +     *             if preconditions are not met or exact p-value is wanted for
>>>> +     *             when x.length > 30
>>>> +     * @throws MathException
>>>> +     *             if an error occurs computing the p-value
>>>> +     */
>>>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>>> +            boolean exactPValue) throws IllegalArgumentException,
>>>> +            MathException {
>>>> +
>>>> +        ensureDataConformance(x, y);
>>>> +
>>>> +        final int N = x.length;
>>>> +        final double Wmax = wilcoxonSignedRank(x, y);
>>>> +
>>>> +        if (exactPValue && N > 30) {
>>>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>>>> +        }
>>>> +
>>>> +        if (exactPValue) {
>>>> +            return calculateExactPValue(Wmax, N);
>>>> +        } else {
>>>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>>>> +            return calculateAsymptoticPValue(Wmin, N);
>>>> +        }
>>>> +    }
>>>> +}
>>>>
>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>>>> ==============================================================================
>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>>>> @@ -0,0 +1,101 @@
>>>> +/*
>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>> + * this work for additional information regarding copyright ownership.
>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>> + * (the "License"); you may not use this file except in compliance with
>>>> + * the License.  You may obtain a copy of the License at
>>>> + *
>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>> + *
>>>> + * Unless required by applicable law or agreed to in writing, software
>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> + * See the License for the specific language governing permissions and
>>>> + * limitations under the License.
>>>> + */
>>>> +package org.apache.commons.math.stat.inference;
>>>> +
>>>> +import junit.framework.TestCase;
>>>> +
>>>> +/**
>>>> + * Test cases for the ChiSquareTestImpl class.
>>>> + *
>>>> + * @version $Revision: $ $Date: $
>>>> + */
>>>> +
>>>> +public class MannWhitneyUTestTest extends TestCase {
>>>> +
>>>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>>>> +
>>>> +    public MannWhitneyUTestTest(String name) {
>>>> +        super(name);
>>>> +    }
>>>> +
>>>> +    public void testMannWhitneyUSimple() throws Exception {
>>>> +        /* Target values computed using R version 2.11.1
>>>> +         * x <- c(19, 22, 16, 29, 24)
>>>> +         * y <- c(20, 11, 17, 12)
>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>>>> +         * W = 17, p-value = 0.08641
>>>> +         */
>>>> +        final double x[] = {19, 22, 16, 29, 24};
>>>> +        final double y[] = {20, 11, 17, 12};
>>>> +
>>>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>>>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>>>> +    }
>>>> +
>>>> +
>>>> +    public void testMannWhitneyUInputValidation() throws Exception {
>>>> +        /* Samples must be present, i.e. length > 0
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        /*
>>>> +         * x and y is null
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.mannWhitneyUTest(null, null);
>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        /*
>>>> +         * x or y is null
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +    }
>>>> +}
>>>>
>>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>>>> ==============================================================================
>>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>>>> @@ -0,0 +1,180 @@
>>>> +/*
>>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>>> + * contributor license agreements.  See the NOTICE file distributed with
>>>> + * this work for additional information regarding copyright ownership.
>>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>>> + * (the "License"); you may not use this file except in compliance with
>>>> + * the License.  You may obtain a copy of the License at
>>>> + *
>>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>>> + *
>>>> + * Unless required by applicable law or agreed to in writing, software
>>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>>> + * See the License for the specific language governing permissions and
>>>> + * limitations under the License.
>>>> + */
>>>> +package org.apache.commons.math.stat.inference;
>>>> +
>>>> +import junit.framework.TestCase;
>>>> +
>>>> +/**
>>>> + * Test cases for the ChiSquareTestImpl class.
>>>> + *
>>>> + * @version $Revision: $ $Date: $
>>>> + */
>>>> +
>>>> +public class WilcoxonSignedRankTestTest extends TestCase {
>>>> +
>>>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>>>> +
>>>> +    public WilcoxonSignedRankTestTest(String name) {
>>>> +        super(name);
>>>> +    }
>>>> +
>>>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>>>> +        /* Target values computed using R version 2.11.1
>>>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>>>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>>>> +         */
>>>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>>>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>>>> +
>>>> +        /* EXACT:
>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>>>> +         * V = 40, p-value = 0.03906
>>>> +         *
>>>> +         * Corresponds to the value obtained in R.
>>>> +         */
>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>>>> +
>>>> +        /* ASYMPTOTIC:
>>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>>>> +         * V = 40, p-value = 0.03815
>>>> +         *
>>>> +         * This is not entirely the same due to different corrects,
>>>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>>>> +         * and src/library/stats/R/wilcox.test.R in the R source
>>>> +         */
>>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>>>> +    }
>>>> +
>>>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>>>> +        /*
>>>> +         * Exact only for sample size <= 30
>>>> +         */
>>>> +        final double[] x1 = new double[30];
>>>> +        final double[] x2 = new double[31];
>>>> +        final double[] y1 = new double[30];
>>>> +        final double[] y2 = new double[31];
>>>> +        for (int i = 0; i < 30; ++i) {
>>>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>>>> +        }
>>>> +
>>>> +        // Exactly 30 is okay
>>>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>>>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        /* Samples must be present, i.e. length > 0
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>>>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>>>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        /* Samples not same size, i.e. cannot be pairred
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>>>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>>>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        /*
>>>> +         * x and y is null
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        /*
>>>> +         * x or y is null
>>>> +         */
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>>>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +
>>>> +        try {
>>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>>>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>>>> +        } catch (IllegalArgumentException ex) {
>>>> +            // expected
>>>> +        }
>>>> +    }
>>>> +}
>>>>
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>

Hi Luc,

Thanks for the help, including the config-file. I've manually changed
the properties for the newly created files in revision 1053918 using
svn propset svn:eol-style native
src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest*
etc.

And now that I'm using a config-file similar to yours, new files
should be dealt with automatically.

In regards to the pom.xml, then I should just make an entry for myself
in the developers-section, right?

Cheers, Mikkel.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Luc Maisonobe <Lu...@free.fr>.
Le 30/12/2010 16:18, Luc Maisonobe a écrit :
> Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
>> Hi,
> 
> Hi Mikkel,
> 
>>
>> This commit consists of new files, so I put
>> @version $Revision: $ $Date: $
>> in the top. I thought SVN would then auto update these, but after
>> checking out again to see if it worked, I see that it didn't. What
>> should I do to make it work? Fire a svn propXXX, use other variables,
>> or?
> 
> You should set some properties. The ones we traditionally use are:
>  svn:eol-style=native
>  svn:keywords="Author Date Id Revision"

Well, in fact we have a policy not to use Author, but rather to put the
author names in the contibutor and developpers sections of the pom.xml file.

Luc

> 
> One way to have these properties automatically set is to use a "config"
> file in subversion configuration folder. I have put the file I use here:
> <http://people.apache.org/~luc/config>.
> 
> I think the file is used only for newly created files, so now that the
> files are already in the repository, you should probably set the
> properties manually.
> 
> best regards,
> Luc
> 
>>
>> Cheers, Mikkel.
>>
>> 2010/12/30  <mi...@apache.org>:
>>> Author: mikl
>>> Date: Thu Dec 30 09:52:00 2010
>>> New Revision: 1053836
>>>
>>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>>> Log:
>>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>>
>>> Added:
>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>>
>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>>> ==============================================================================
>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>>> @@ -0,0 +1,96 @@
>>> +/*
>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>> + * contributor license agreements.  See the NOTICE file distributed with
>>> + * this work for additional information regarding copyright ownership.
>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>> + * (the "License"); you may not use this file except in compliance with
>>> + * the License.  You may obtain a copy of the License at
>>> + *
>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>> + *
>>> + * Unless required by applicable law or agreed to in writing, software
>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> + * See the License for the specific language governing permissions and
>>> + * limitations under the License.
>>> + */
>>> +package org.apache.commons.math.stat.inference;
>>> +
>>> +import org.apache.commons.math.MathException;
>>> +
>>> +/**
>>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>>> + *
>>> + * @version $Revision: $ $Date: $
>>> + */
>>> +public interface MannWhitneyUTest {
>>> +
>>> +    /**
>>> +     * Computes the <a
>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>> +     * U statistic</a> comparing mean for two independent samples possibly of
>>> +     * different length.
>>> +     * <p>
>>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>>> +     * the null hypothesis that the two independent samples has equal mean.
>>> +     * </p>
>>> +     * <p>
>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>> +     * samples would often have different length.
>>> +     * </p>
>>> +     * <p>
>>> +     * <strong>Preconditions</strong>:
>>> +     * <ul>
>>> +     * <li>All observations in the two samples are independent.</li>
>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>> +     * </ul>
>>> +     * </p>
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @return mannWhitneyU statistic
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     */
>>> +    double mannWhitneyU(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException;
>>> +
>>> +    /**
>>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>> +     * p-value</a>, associated with a <a
>>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>>> +     * U statistic</a> comparing mean for two independent samples.
>>> +     * <p>
>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>>> +     * samples would often have different length.
>>> +     * </p>
>>> +     * <p>
>>> +     * <strong>Preconditions</strong>:
>>> +     * <ul>
>>> +     * <li>All observations in the two samples are independent.</li>
>>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>>> +     * </ul>
>>> +     * </p>
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @param exactPValue
>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>> +     *            if true and x.length > 30, this is ignored because
>>> +     *            calculations may take too long)
>>> +     * @return asymptotic p-value
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     * @throws MathException
>>> +     *             if an error occurs computing the p-value
>>> +     */
>>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException, MathException;
>>> +}
>>>
>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>>> ==============================================================================
>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>>> @@ -0,0 +1,199 @@
>>> +/*
>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>> + * contributor license agreements.  See the NOTICE file distributed with
>>> + * this work for additional information regarding copyright ownership.
>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>> + * (the "License"); you may not use this file except in compliance with
>>> + * the License.  You may obtain a copy of the License at
>>> + *
>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>> + *
>>> + * Unless required by applicable law or agreed to in writing, software
>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> + * See the License for the specific language governing permissions and
>>> + * limitations under the License.
>>> + */
>>> +package org.apache.commons.math.stat.inference;
>>> +
>>> +import org.apache.commons.math.MathException;
>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>> +import org.apache.commons.math.util.FastMath;
>>> +
>>> +/**
>>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>>> + * test).
>>> + *
>>> + * @version $Revision: $ $Date: $
>>> + */
>>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>>> +    private NaturalRanking naturalRanking;
>>> +
>>> +    /**
>>> +     * Create a test instance using where NaN's are left in place and ties get
>>> +     * the average of applicable ranks. Use this unless you are very sure of
>>> +     * what you are doing.
>>> +     */
>>> +    public MannWhitneyUTestImpl() {
>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>> +                TiesStrategy.AVERAGE);
>>> +    }
>>> +
>>> +    /**
>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>> +     * Only use this if you are sure of what you are doing.
>>> +     *
>>> +     * @param nanStrategy
>>> +     *            specifies the strategy that should be used for Double.NaN's
>>> +     * @param tiesStrategy
>>> +     *            specifies the strategy that should be used for ties
>>> +     */
>>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>>> +            TiesStrategy tiesStrategy) {
>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>> +    }
>>> +
>>> +    /**
>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>> +     *
>>> +     * @param x
>>> +     * @param y
>>> +     * @throws IllegalArgumentException
>>> +     *             if assumptions are not met
>>> +     */
>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException {
>>> +        if (x == null) {
>>> +            throw new IllegalArgumentException("x must not be null");
>>> +        }
>>> +
>>> +        if (y == null) {
>>> +            throw new IllegalArgumentException("y must not be null");
>>> +        }
>>> +
>>> +        if (x.length == 0) {
>>> +            throw new IllegalArgumentException(
>>> +                    "x must contain at least one element");
>>> +        }
>>> +
>>> +        if (y.length == 0) {
>>> +            throw new IllegalArgumentException(
>>> +                    "y must contain at least one element");
>>> +        }
>>> +    }
>>> +
>>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>>> +        final double[] z = new double[x.length + y.length];
>>> +
>>> +        System.arraycopy(x, 0, z, 0, x.length);
>>> +        System.arraycopy(y, 0, z, x.length, y.length);
>>> +
>>> +        return z;
>>> +    }
>>> +
>>> +    /**
>>> +     * {@inheritDoc}
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     */
>>> +    public double mannWhitneyU(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException {
>>> +
>>> +        ensureDataConformance(x, y);
>>> +
>>> +        final double[] z = concatinateSamples(x, y);
>>> +        final double[] ranks = naturalRanking.rank(z);
>>> +
>>> +        double sumRankX = 0;
>>> +
>>> +        /*
>>> +         * The ranks for x is in the first x.length entries in ranks because x
>>> +         * is in the first x.length entries in z
>>> +         */
>>> +        for (int i = 0; i < x.length; ++i) {
>>> +            sumRankX += ranks[i];
>>> +        }
>>> +
>>> +        /*
>>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>>> +         * e.g. x, n1 is the number of observations in sample 1.
>>> +         */
>>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>>> +
>>> +        /*
>>> +         * It can be shown that U1 + U2 = n1 * n2
>>> +         */
>>> +        final double U2 = x.length * y.length - U1;
>>> +
>>> +        return FastMath.max(U1, U2);
>>> +    }
>>> +
>>> +    /**
>>> +     * @param Umin
>>> +     *            smallest Mann-Whitney U value
>>> +     * @param N
>>> +     *            number of subjects (corresponding to x.length)
>>> +     * @return two-sided asymptotic p-value
>>> +     * @throws MathException
>>> +     *             if an error occurs computing the p-value
>>> +     */
>>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>>> +            final int n2) throws MathException {
>>> +
>>> +        final int n1n2prod = n1 * n2;
>>> +
>>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>>> +        final double EU = (double) n1n2prod / 2.0;
>>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>>> +
>>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>>> +
>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>>> +                0, 1);
>>> +
>>> +        return 2 * standardNormal.cumulativeProbability(z);
>>> +    }
>>> +
>>> +    /**
>>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>>> +     *
>>> +     * {@inheritDoc}
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @param exactPValue
>>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>>> +     * @return asymptotic p-value (biased for samples with ties)
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     * @throws MathException
>>> +     *             if an error occurs computing the p-value
>>> +     */
>>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException, MathException {
>>> +
>>> +        ensureDataConformance(x, y);
>>> +
>>> +        final double Umax = mannWhitneyU(x, y);
>>> +
>>> +        /*
>>> +         * It can be shown that U1 + U2 = n1 * n2
>>> +         */
>>> +        final double Umin = x.length * y.length - Umax;
>>> +
>>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>>> +    }
>>> +}
>>>
>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>>> ==============================================================================
>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>>> @@ -0,0 +1,107 @@
>>> +/*
>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>> + * contributor license agreements.  See the NOTICE file distributed with
>>> + * this work for additional information regarding copyright ownership.
>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>> + * (the "License"); you may not use this file except in compliance with
>>> + * the License.  You may obtain a copy of the License at
>>> + *
>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>> + *
>>> + * Unless required by applicable law or agreed to in writing, software
>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> + * See the License for the specific language governing permissions and
>>> + * limitations under the License.
>>> + */
>>> +package org.apache.commons.math.stat.inference;
>>> +
>>> +import org.apache.commons.math.MathException;
>>> +
>>> +/**
>>> + * An interface for Wilcoxon signed-rank test.
>>> + *
>>> + * @version $Revision: $ $Date: $
>>> + */
>>> +public interface WilcoxonSignedRankTest {
>>> +
>>> +    /**
>>> +     * Computes the <a
>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>> +     * samples or repeated measurements on a single sample.
>>> +     * <p>
>>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>>> +     * evaluating the null hypothesis that the two related samples or repeated
>>> +     * measurements on a single sample has equal mean.
>>> +     * </p>
>>> +     * <p>
>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>> +     * </p>
>>> +     * <p>
>>> +     * <strong>Preconditions</strong>:
>>> +     * <ul>
>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>> +     * identical) and is symmetric about a common median.</li>
>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>> +     * meaningful.</li>
>>> +     * </ul>
>>> +     * </p>
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @return wilcoxonSignedRank statistic
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     */
>>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException;
>>> +
>>> +    /**
>>> +     * Returns the <i>observed significance level</i>, or <a href=
>>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>>> +     * p-value</a>, associated with a <a
>>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>>> +     * samples or repeated measurements on a single sample.
>>> +     * <p>
>>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>>> +     * </p>
>>> +     * <p>
>>> +     * <strong>Preconditions</strong>:
>>> +     * <ul>
>>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>>> +     * identical) and is symmetric about a common median.</li>
>>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>>> +     * ordered, so the comparisons greater than, less than, and equal to are
>>> +     * meaningful.</li>
>>> +     * </ul>
>>> +     * </p>
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @param exactPValue
>>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>>> +     *            if true and x.length > 30, this is ignored because
>>> +     *            calculations may take too long)
>>> +     * @return p-value
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     * @throws MathException
>>> +     *             if an error occurs computing the p-value
>>> +     */
>>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>> +            boolean exactPValue) throws IllegalArgumentException,
>>> +            MathException;
>>> +}
>>>
>>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>>> ==============================================================================
>>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>>> @@ -0,0 +1,271 @@
>>> +/*
>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>> + * contributor license agreements.  See the NOTICE file distributed with
>>> + * this work for additional information regarding copyright ownership.
>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>> + * (the "License"); you may not use this file except in compliance with
>>> + * the License.  You may obtain a copy of the License at
>>> + *
>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>> + *
>>> + * Unless required by applicable law or agreed to in writing, software
>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> + * See the License for the specific language governing permissions and
>>> + * limitations under the License.
>>> + */
>>> +package org.apache.commons.math.stat.inference;
>>> +
>>> +import org.apache.commons.math.MathException;
>>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>>> +import org.apache.commons.math.util.FastMath;
>>> +
>>> +/**
>>> + * An implementation of the Wilcoxon signed-rank test.
>>> + *
>>> + * @version $Revision: $ $Date: $
>>> + */
>>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>>> +    private NaturalRanking naturalRanking;
>>> +
>>> +    /**
>>> +     * Create a test instance where NaN's are left in place and ties get
>>> +     * the average of applicable ranks. Use this unless you are very sure
>>> +     * of what you are doing.
>>> +     */
>>> +    public WilcoxonSignedRankTestImpl() {
>>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>>> +                TiesStrategy.AVERAGE);
>>> +    }
>>> +
>>> +    /**
>>> +     * Create a test instance using the given strategies for NaN's and ties.
>>> +     * Only use this if you are sure of what you are doing.
>>> +     *
>>> +     * @param nanStrategy
>>> +     *            specifies the strategy that should be used for Double.NaN's
>>> +     * @param tiesStrategy
>>> +     *            specifies the strategy that should be used for ties
>>> +     */
>>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>>> +            TiesStrategy tiesStrategy) {
>>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>>> +    }
>>> +
>>> +    /**
>>> +     * Ensures that the provided arrays fulfills the assumptions.
>>> +     *
>>> +     * @param x
>>> +     * @param y
>>> +     * @throws IllegalArgumentException
>>> +     *             if assumptions are not met
>>> +     */
>>> +    private void ensureDataConformance(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException {
>>> +        if (x == null) {
>>> +            throw new IllegalArgumentException("x must not be null");
>>> +        }
>>> +
>>> +        if (y == null) {
>>> +            throw new IllegalArgumentException("y must not be null");
>>> +        }
>>> +
>>> +        if (x.length != y.length) {
>>> +            throw new IllegalArgumentException(
>>> +                    "x and y must contain the same number of elements");
>>> +        }
>>> +
>>> +        if (x.length == 0) {
>>> +            throw new IllegalArgumentException(
>>> +                    "x and y must contain at least one element");
>>> +        }
>>> +    }
>>> +
>>> +    /**
>>> +     * Calculates y[i] - x[i] for all i
>>> +     *
>>> +     * @param x
>>> +     * @param y
>>> +     * @throws IllegalArgumentException
>>> +     *             if assumptions are not met
>>> +     */
>>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException {
>>> +
>>> +        final double[] z = new double[x.length];
>>> +
>>> +        for (int i = 0; i < x.length; ++i) {
>>> +            z[i] = y[i] - x[i];
>>> +        }
>>> +
>>> +        return z;
>>> +    }
>>> +
>>> +    /**
>>> +     * Calculates |z[i]| for all i
>>> +     *
>>> +     * @param z
>>> +     * @throws IllegalArgumentException
>>> +     *             if assumptions are not met
>>> +     */
>>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>>> +            throws IllegalArgumentException {
>>> +        if (z == null) {
>>> +            throw new IllegalArgumentException("z must not be null");
>>> +        }
>>> +
>>> +        if (z.length == 0) {
>>> +            throw new IllegalArgumentException(
>>> +                    "z must contain at least one element");
>>> +        }
>>> +
>>> +        final double[] zAbs = new double[z.length];
>>> +
>>> +        for (int i = 0; i < z.length; ++i) {
>>> +            zAbs[i] = FastMath.abs(z[i]);
>>> +        }
>>> +
>>> +        return zAbs;
>>> +    }
>>> +
>>> +    /**
>>> +     * {@inheritDoc}
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met
>>> +     */
>>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>>> +            throws IllegalArgumentException {
>>> +
>>> +        ensureDataConformance(x, y);
>>> +
>>> +        // throws IllegalArgumentException if x and y are not correctly
>>> +        // specified
>>> +        final double[] z = calculateDifferences(x, y);
>>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>>> +
>>> +        final double[] ranks = naturalRanking.rank(zAbs);
>>> +
>>> +        double Wplus = 0;
>>> +
>>> +        for (int i = 0; i < z.length; ++i) {
>>> +            if (z[i] > 0) {
>>> +                Wplus += ranks[i];
>>> +            }
>>> +        }
>>> +
>>> +        final int N = x.length;
>>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>>> +
>>> +        return FastMath.max(Wplus, Wminus);
>>> +    }
>>> +
>>> +    /**
>>> +     * Algorithm inspired by
>>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>>> +     * University of Amsterdam
>>> +     *
>>> +     * @param Wmax largest Wilcoxon signed rank value
>>> +     * @param N number of subjects (corresponding to x.length)
>>> +     * @return two-sided exact p-value
>>> +     */
>>> +    private double calculateExactPValue(final double Wmax, final int N) {
>>> +
>>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>>> +        final int m = 1 << N;
>>> +
>>> +        int largerRankSums = 0;
>>> +
>>> +        for (int i = 0; i < m; ++i) {
>>> +            int rankSum = 0;
>>> +
>>> +            // Generate all possible rank sums
>>> +            for (int j = 0; j < N; ++j) {
>>> +
>>> +                // (i >> j) & 1 extract i's j-th bit from the right
>>> +                if (((i >> j) & 1) == 1) {
>>> +                    rankSum += j + 1;
>>> +                }
>>> +            }
>>> +
>>> +            if (rankSum >= Wmax) {
>>> +                ++largerRankSums;
>>> +            }
>>> +        }
>>> +
>>> +        /*
>>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>>> +         * with 2 to get the two-sided p-value
>>> +         */
>>> +        return 2 * ((double) largerRankSums) / ((double) m);
>>> +    }
>>> +
>>> +    /**
>>> +     * @param Wmin smallest Wilcoxon signed rank value
>>> +     * @param N number of subjects (corresponding to x.length)
>>> +     * @return two-sided asymptotic p-value
>>> +     * @throws MathException if an error occurs computing the p-value
>>> +     */
>>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>>> +
>>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>>> +
>>> +        /* Same as (but saves computations):
>>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>>> +         */
>>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>>> +
>>> +        // - 0.5 is a continuity correction
>>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>>> +
>>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>>> +
>>> +        return 2*standardNormal.cumulativeProbability(z);
>>> +    }
>>> +
>>> +    /**
>>> +     * {@inheritDoc}
>>> +     *
>>> +     * @param x
>>> +     *            the first sample
>>> +     * @param y
>>> +     *            the second sample
>>> +     * @param exactPValue
>>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>>> +     * @return p-value
>>> +     * @throws IllegalArgumentException
>>> +     *             if preconditions are not met or exact p-value is wanted for
>>> +     *             when x.length > 30
>>> +     * @throws MathException
>>> +     *             if an error occurs computing the p-value
>>> +     */
>>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>>> +            boolean exactPValue) throws IllegalArgumentException,
>>> +            MathException {
>>> +
>>> +        ensureDataConformance(x, y);
>>> +
>>> +        final int N = x.length;
>>> +        final double Wmax = wilcoxonSignedRank(x, y);
>>> +
>>> +        if (exactPValue && N > 30) {
>>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>>> +        }
>>> +
>>> +        if (exactPValue) {
>>> +            return calculateExactPValue(Wmax, N);
>>> +        } else {
>>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>>> +            return calculateAsymptoticPValue(Wmin, N);
>>> +        }
>>> +    }
>>> +}
>>>
>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>>> ==============================================================================
>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>>> @@ -0,0 +1,101 @@
>>> +/*
>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>> + * contributor license agreements.  See the NOTICE file distributed with
>>> + * this work for additional information regarding copyright ownership.
>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>> + * (the "License"); you may not use this file except in compliance with
>>> + * the License.  You may obtain a copy of the License at
>>> + *
>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>> + *
>>> + * Unless required by applicable law or agreed to in writing, software
>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> + * See the License for the specific language governing permissions and
>>> + * limitations under the License.
>>> + */
>>> +package org.apache.commons.math.stat.inference;
>>> +
>>> +import junit.framework.TestCase;
>>> +
>>> +/**
>>> + * Test cases for the ChiSquareTestImpl class.
>>> + *
>>> + * @version $Revision: $ $Date: $
>>> + */
>>> +
>>> +public class MannWhitneyUTestTest extends TestCase {
>>> +
>>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>>> +
>>> +    public MannWhitneyUTestTest(String name) {
>>> +        super(name);
>>> +    }
>>> +
>>> +    public void testMannWhitneyUSimple() throws Exception {
>>> +        /* Target values computed using R version 2.11.1
>>> +         * x <- c(19, 22, 16, 29, 24)
>>> +         * y <- c(20, 11, 17, 12)
>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>>> +         * W = 17, p-value = 0.08641
>>> +         */
>>> +        final double x[] = {19, 22, 16, 29, 24};
>>> +        final double y[] = {20, 11, 17, 12};
>>> +
>>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>>> +    }
>>> +
>>> +
>>> +    public void testMannWhitneyUInputValidation() throws Exception {
>>> +        /* Samples must be present, i.e. length > 0
>>> +         */
>>> +        try {
>>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        /*
>>> +         * x and y is null
>>> +         */
>>> +        try {
>>> +            testStatistic.mannWhitneyUTest(null, null);
>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.mannWhitneyUTest(null, null);
>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        /*
>>> +         * x or y is null
>>> +         */
>>> +        try {
>>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +    }
>>> +}
>>>
>>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>>> ==============================================================================
>>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>>> @@ -0,0 +1,180 @@
>>> +/*
>>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>>> + * contributor license agreements.  See the NOTICE file distributed with
>>> + * this work for additional information regarding copyright ownership.
>>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>>> + * (the "License"); you may not use this file except in compliance with
>>> + * the License.  You may obtain a copy of the License at
>>> + *
>>> + *      http://www.apache.org/licenses/LICENSE-2.0
>>> + *
>>> + * Unless required by applicable law or agreed to in writing, software
>>> + * distributed under the License is distributed on an "AS IS" BASIS,
>>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>>> + * See the License for the specific language governing permissions and
>>> + * limitations under the License.
>>> + */
>>> +package org.apache.commons.math.stat.inference;
>>> +
>>> +import junit.framework.TestCase;
>>> +
>>> +/**
>>> + * Test cases for the ChiSquareTestImpl class.
>>> + *
>>> + * @version $Revision: $ $Date: $
>>> + */
>>> +
>>> +public class WilcoxonSignedRankTestTest extends TestCase {
>>> +
>>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>>> +
>>> +    public WilcoxonSignedRankTestTest(String name) {
>>> +        super(name);
>>> +    }
>>> +
>>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>>> +        /* Target values computed using R version 2.11.1
>>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>>> +         */
>>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>>> +
>>> +        /* EXACT:
>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>>> +         * V = 40, p-value = 0.03906
>>> +         *
>>> +         * Corresponds to the value obtained in R.
>>> +         */
>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>>> +
>>> +        /* ASYMPTOTIC:
>>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>>> +         * V = 40, p-value = 0.03815
>>> +         *
>>> +         * This is not entirely the same due to different corrects,
>>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>>> +         * and src/library/stats/R/wilcox.test.R in the R source
>>> +         */
>>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>>> +    }
>>> +
>>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>>> +        /*
>>> +         * Exact only for sample size <= 30
>>> +         */
>>> +        final double[] x1 = new double[30];
>>> +        final double[] x2 = new double[31];
>>> +        final double[] y1 = new double[30];
>>> +        final double[] y2 = new double[31];
>>> +        for (int i = 0; i < 30; ++i) {
>>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>>> +        }
>>> +
>>> +        // Exactly 30 is okay
>>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        /* Samples must be present, i.e. length > 0
>>> +         */
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        /* Samples not same size, i.e. cannot be pairred
>>> +         */
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        /*
>>> +         * x and y is null
>>> +         */
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        /*
>>> +         * x or y is null
>>> +         */
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>>> +            fail("x is null (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>>> +            fail("y is null (exact), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +
>>> +        try {
>>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>>> +        } catch (IllegalArgumentException ex) {
>>> +            // expected
>>> +        }
>>> +    }
>>> +}
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Re: svn commit: r1053836 - in /commons/proper/math/trunk/src: main/java/org/apache/commons/math/stat/inference/ test/java/org/apache/commons/math/stat/inference/

Posted by Luc Maisonobe <Lu...@free.fr>.
Le 30/12/2010 11:00, Mikkel Meyer Andersen a écrit :
> Hi,

Hi Mikkel,

> 
> This commit consists of new files, so I put
> @version $Revision: $ $Date: $
> in the top. I thought SVN would then auto update these, but after
> checking out again to see if it worked, I see that it didn't. What
> should I do to make it work? Fire a svn propXXX, use other variables,
> or?

You should set some properties. The ones we traditionally use are:
 svn:eol-style=native
 svn:keywords="Author Date Id Revision"

One way to have these properties automatically set is to use a "config"
file in subversion configuration folder. I have put the file I use here:
<http://people.apache.org/~luc/config>.

I think the file is used only for newly created files, so now that the
files are already in the repository, you should probably set the
properties manually.

best regards,
Luc

> 
> Cheers, Mikkel.
> 
> 2010/12/30  <mi...@apache.org>:
>> Author: mikl
>> Date: Thu Dec 30 09:52:00 2010
>> New Revision: 1053836
>>
>> URL: http://svn.apache.org/viewvc?rev=1053836&view=rev
>> Log:
>> Starting point for fixing MATH-431. Still some validation tests and documentation are missing.
>>
>> Added:
>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>>    commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>>    commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>>
>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java?rev=1053836&view=auto
>> ==============================================================================
>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java (added)
>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTest.java Thu Dec 30 09:52:00 2010
>> @@ -0,0 +1,96 @@
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>> + * contributor license agreements.  See the NOTICE file distributed with
>> + * this work for additional information regarding copyright ownership.
>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>> + * (the "License"); you may not use this file except in compliance with
>> + * the License.  You may obtain a copy of the License at
>> + *
>> + *      http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +package org.apache.commons.math.stat.inference;
>> +
>> +import org.apache.commons.math.MathException;
>> +
>> +/**
>> + * An interface for Mann-Whitney U test (also called Wilcoxon rank-sum test).
>> + *
>> + * @version $Revision: $ $Date: $
>> + */
>> +public interface MannWhitneyUTest {
>> +
>> +    /**
>> +     * Computes the <a
>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>> +     * U statistic</a> comparing mean for two independent samples possibly of
>> +     * different length.
>> +     * <p>
>> +     * This statistic can be used to perform a Mann-Whitney U test evaluating
>> +     * the null hypothesis that the two independent samples has equal mean.
>> +     * </p>
>> +     * <p>
>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>> +     * samples would often have different length.
>> +     * </p>
>> +     * <p>
>> +     * <strong>Preconditions</strong>:
>> +     * <ul>
>> +     * <li>All observations in the two samples are independent.</li>
>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>> +     * </ul>
>> +     * </p>
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @return mannWhitneyU statistic
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     */
>> +    double mannWhitneyU(final double[] x, final double[] y)
>> +            throws IllegalArgumentException;
>> +
>> +    /**
>> +     * Returns the asymptotic <i>observed significance level</i>, or <a href=
>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>> +     * p-value</a>, associated with a <a
>> +     * href="http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U"> Mann-Whitney
>> +     * U statistic</a> comparing mean for two independent samples.
>> +     * <p>
>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>> +     * Y<sub>j</sub> the j'th individual in the second sample. Note that the
>> +     * samples would often have different length.
>> +     * </p>
>> +     * <p>
>> +     * <strong>Preconditions</strong>:
>> +     * <ul>
>> +     * <li>All observations in the two samples are independent.</li>
>> +     * <li>The observations are at least ordinal (continuous are also ordinal).</li>
>> +     * </ul>
>> +     * </p>
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @param exactPValue
>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>> +     *            if true and x.length > 30, this is ignored because
>> +     *            calculations may take too long)
>> +     * @return asymptotic p-value
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     * @throws MathException
>> +     *             if an error occurs computing the p-value
>> +     */
>> +    double mannWhitneyUTest(final double[] x, final double[] y)
>> +            throws IllegalArgumentException, MathException;
>> +}
>>
>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java?rev=1053836&view=auto
>> ==============================================================================
>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java (added)
>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/MannWhitneyUTestImpl.java Thu Dec 30 09:52:00 2010
>> @@ -0,0 +1,199 @@
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>> + * contributor license agreements.  See the NOTICE file distributed with
>> + * this work for additional information regarding copyright ownership.
>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>> + * (the "License"); you may not use this file except in compliance with
>> + * the License.  You may obtain a copy of the License at
>> + *
>> + *      http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +package org.apache.commons.math.stat.inference;
>> +
>> +import org.apache.commons.math.MathException;
>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>> +import org.apache.commons.math.util.FastMath;
>> +
>> +/**
>> + * An implementation of the Mann-Whitney U test (also called Wilcoxon rank-sum
>> + * test).
>> + *
>> + * @version $Revision: $ $Date: $
>> + */
>> +public class MannWhitneyUTestImpl implements MannWhitneyUTest {
>> +    private NaturalRanking naturalRanking;
>> +
>> +    /**
>> +     * Create a test instance using where NaN's are left in place and ties get
>> +     * the average of applicable ranks. Use this unless you are very sure of
>> +     * what you are doing.
>> +     */
>> +    public MannWhitneyUTestImpl() {
>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>> +                TiesStrategy.AVERAGE);
>> +    }
>> +
>> +    /**
>> +     * Create a test instance using the given strategies for NaN's and ties.
>> +     * Only use this if you are sure of what you are doing.
>> +     *
>> +     * @param nanStrategy
>> +     *            specifies the strategy that should be used for Double.NaN's
>> +     * @param tiesStrategy
>> +     *            specifies the strategy that should be used for ties
>> +     */
>> +    public MannWhitneyUTestImpl(NaNStrategy nanStrategy,
>> +            TiesStrategy tiesStrategy) {
>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>> +    }
>> +
>> +    /**
>> +     * Ensures that the provided arrays fulfills the assumptions.
>> +     *
>> +     * @param x
>> +     * @param y
>> +     * @throws IllegalArgumentException
>> +     *             if assumptions are not met
>> +     */
>> +    private void ensureDataConformance(final double[] x, final double[] y)
>> +            throws IllegalArgumentException {
>> +        if (x == null) {
>> +            throw new IllegalArgumentException("x must not be null");
>> +        }
>> +
>> +        if (y == null) {
>> +            throw new IllegalArgumentException("y must not be null");
>> +        }
>> +
>> +        if (x.length == 0) {
>> +            throw new IllegalArgumentException(
>> +                    "x must contain at least one element");
>> +        }
>> +
>> +        if (y.length == 0) {
>> +            throw new IllegalArgumentException(
>> +                    "y must contain at least one element");
>> +        }
>> +    }
>> +
>> +    private double[] concatinateSamples(final double[] x, final double[] y) {
>> +        final double[] z = new double[x.length + y.length];
>> +
>> +        System.arraycopy(x, 0, z, 0, x.length);
>> +        System.arraycopy(y, 0, z, x.length, y.length);
>> +
>> +        return z;
>> +    }
>> +
>> +    /**
>> +     * {@inheritDoc}
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @return mannWhitneyU statistic U (maximum of U<sup>x</sup> and U<sup>y</sup>)
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     */
>> +    public double mannWhitneyU(final double[] x, final double[] y)
>> +            throws IllegalArgumentException {
>> +
>> +        ensureDataConformance(x, y);
>> +
>> +        final double[] z = concatinateSamples(x, y);
>> +        final double[] ranks = naturalRanking.rank(z);
>> +
>> +        double sumRankX = 0;
>> +
>> +        /*
>> +         * The ranks for x is in the first x.length entries in ranks because x
>> +         * is in the first x.length entries in z
>> +         */
>> +        for (int i = 0; i < x.length; ++i) {
>> +            sumRankX += ranks[i];
>> +        }
>> +
>> +        /*
>> +         * U1 = R1 - (n1 * (n1 + 1)) / 2 where R1 is sum of ranks for sample 1,
>> +         * e.g. x, n1 is the number of observations in sample 1.
>> +         */
>> +        final double U1 = sumRankX - (x.length * (x.length + 1)) / 2;
>> +
>> +        /*
>> +         * It can be shown that U1 + U2 = n1 * n2
>> +         */
>> +        final double U2 = x.length * y.length - U1;
>> +
>> +        return FastMath.max(U1, U2);
>> +    }
>> +
>> +    /**
>> +     * @param Umin
>> +     *            smallest Mann-Whitney U value
>> +     * @param N
>> +     *            number of subjects (corresponding to x.length)
>> +     * @return two-sided asymptotic p-value
>> +     * @throws MathException
>> +     *             if an error occurs computing the p-value
>> +     */
>> +    private double calculateAsymptoticPValue(final double Umin, final int n1,
>> +            final int n2) throws MathException {
>> +
>> +        final int n1n2prod = n1 * n2;
>> +
>> +        // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
>> +        final double EU = (double) n1n2prod / 2.0;
>> +        final double VarU = (double) (n1n2prod * (n1 + n2 + 1)) / 12.0;
>> +
>> +        final double z = (Umin - EU) / FastMath.sqrt(VarU);
>> +
>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(
>> +                0, 1);
>> +
>> +        return 2 * standardNormal.cumulativeProbability(z);
>> +    }
>> +
>> +    /**
>> +     * Ties give rise to biased variance at the moment. See e.g. <a
>> +     * href="http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf"
>> +     * >http://mlsc.lboro.ac.uk/resources/statistics/Mannwhitney.pdf</a>.
>> +     *
>> +     * {@inheritDoc}
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @param exactPValue
>> +     *            if the exact p-value is wanted (only for x.length <= 50)
>> +     * @return asymptotic p-value (biased for samples with ties)
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     * @throws MathException
>> +     *             if an error occurs computing the p-value
>> +     */
>> +    public double mannWhitneyUTest(final double[] x, final double[] y)
>> +            throws IllegalArgumentException, MathException {
>> +
>> +        ensureDataConformance(x, y);
>> +
>> +        final double Umax = mannWhitneyU(x, y);
>> +
>> +        /*
>> +         * It can be shown that U1 + U2 = n1 * n2
>> +         */
>> +        final double Umin = x.length * y.length - Umax;
>> +
>> +        return calculateAsymptoticPValue(Umin, x.length, y.length);
>> +    }
>> +}
>>
>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java?rev=1053836&view=auto
>> ==============================================================================
>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java (added)
>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTest.java Thu Dec 30 09:52:00 2010
>> @@ -0,0 +1,107 @@
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>> + * contributor license agreements.  See the NOTICE file distributed with
>> + * this work for additional information regarding copyright ownership.
>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>> + * (the "License"); you may not use this file except in compliance with
>> + * the License.  You may obtain a copy of the License at
>> + *
>> + *      http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +package org.apache.commons.math.stat.inference;
>> +
>> +import org.apache.commons.math.MathException;
>> +
>> +/**
>> + * An interface for Wilcoxon signed-rank test.
>> + *
>> + * @version $Revision: $ $Date: $
>> + */
>> +public interface WilcoxonSignedRankTest {
>> +
>> +    /**
>> +     * Computes the <a
>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>> +     * samples or repeated measurements on a single sample.
>> +     * <p>
>> +     * This statistic can be used to perform a Wilcoxon signed ranked test
>> +     * evaluating the null hypothesis that the two related samples or repeated
>> +     * measurements on a single sample has equal mean.
>> +     * </p>
>> +     * <p>
>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>> +     * </p>
>> +     * <p>
>> +     * <strong>Preconditions</strong>:
>> +     * <ul>
>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>> +     * identical) and is symmetric about a common median.</li>
>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>> +     * ordered, so the comparisons greater than, less than, and equal to are
>> +     * meaningful.</li>
>> +     * </ul>
>> +     * </p>
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @return wilcoxonSignedRank statistic
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     */
>> +    double wilcoxonSignedRank(final double[] x, final double[] y)
>> +            throws IllegalArgumentException;
>> +
>> +    /**
>> +     * Returns the <i>observed significance level</i>, or <a href=
>> +     * "http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest.html#pvalue">
>> +     * p-value</a>, associated with a <a
>> +     * href="http://en.wikipedia.org/wiki/Wilcoxon_signed-rank_test">
>> +     * Wilcoxon signed ranked statistic</a> comparing mean for two related
>> +     * samples or repeated measurements on a single sample.
>> +     * <p>
>> +     * Let X<sub>i</sub> denote the i'th individual of the first sample and
>> +     * Y<sub>i</sub> the related i'th individual in the second sample. Let
>> +     * Z<sub>i</sub> = Y<sub>i</sub> - X<sub>i</sub>.
>> +     * </p>
>> +     * <p>
>> +     * <strong>Preconditions</strong>:
>> +     * <ul>
>> +     * <li>The differences Z<sub>i</sub> must be independent.</li>
>> +     * <li>Each Z<sub>i</sub> comes from a continuous population (they must be
>> +     * identical) and is symmetric about a common median.</li>
>> +     * <li>The values that X<sub>i</sub> and Y<sub>i</sub> represent are
>> +     * ordered, so the comparisons greater than, less than, and equal to are
>> +     * meaningful.</li>
>> +     * </ul>
>> +     * </p>
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @param exactPValue
>> +     *            if the exact p-value is wanted (only works for x.length <= 30,
>> +     *            if true and x.length > 30, this is ignored because
>> +     *            calculations may take too long)
>> +     * @return p-value
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     * @throws MathException
>> +     *             if an error occurs computing the p-value
>> +     */
>> +    double wilcoxonSignedRankTest(final double[] x, final double[] y,
>> +            boolean exactPValue) throws IllegalArgumentException,
>> +            MathException;
>> +}
>>
>> Added: commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java?rev=1053836&view=auto
>> ==============================================================================
>> --- commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java (added)
>> +++ commons/proper/math/trunk/src/main/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestImpl.java Thu Dec 30 09:52:00 2010
>> @@ -0,0 +1,271 @@
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>> + * contributor license agreements.  See the NOTICE file distributed with
>> + * this work for additional information regarding copyright ownership.
>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>> + * (the "License"); you may not use this file except in compliance with
>> + * the License.  You may obtain a copy of the License at
>> + *
>> + *      http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +package org.apache.commons.math.stat.inference;
>> +
>> +import org.apache.commons.math.MathException;
>> +import org.apache.commons.math.distribution.NormalDistributionImpl;
>> +import org.apache.commons.math.stat.ranking.NaNStrategy;
>> +import org.apache.commons.math.stat.ranking.NaturalRanking;
>> +import org.apache.commons.math.stat.ranking.TiesStrategy;
>> +import org.apache.commons.math.util.FastMath;
>> +
>> +/**
>> + * An implementation of the Wilcoxon signed-rank test.
>> + *
>> + * @version $Revision: $ $Date: $
>> + */
>> +public class WilcoxonSignedRankTestImpl implements WilcoxonSignedRankTest {
>> +    private NaturalRanking naturalRanking;
>> +
>> +    /**
>> +     * Create a test instance where NaN's are left in place and ties get
>> +     * the average of applicable ranks. Use this unless you are very sure
>> +     * of what you are doing.
>> +     */
>> +    public WilcoxonSignedRankTestImpl() {
>> +        naturalRanking = new NaturalRanking(NaNStrategy.FIXED,
>> +                TiesStrategy.AVERAGE);
>> +    }
>> +
>> +    /**
>> +     * Create a test instance using the given strategies for NaN's and ties.
>> +     * Only use this if you are sure of what you are doing.
>> +     *
>> +     * @param nanStrategy
>> +     *            specifies the strategy that should be used for Double.NaN's
>> +     * @param tiesStrategy
>> +     *            specifies the strategy that should be used for ties
>> +     */
>> +    public WilcoxonSignedRankTestImpl(NaNStrategy nanStrategy,
>> +            TiesStrategy tiesStrategy) {
>> +        naturalRanking = new NaturalRanking(nanStrategy, tiesStrategy);
>> +    }
>> +
>> +    /**
>> +     * Ensures that the provided arrays fulfills the assumptions.
>> +     *
>> +     * @param x
>> +     * @param y
>> +     * @throws IllegalArgumentException
>> +     *             if assumptions are not met
>> +     */
>> +    private void ensureDataConformance(final double[] x, final double[] y)
>> +            throws IllegalArgumentException {
>> +        if (x == null) {
>> +            throw new IllegalArgumentException("x must not be null");
>> +        }
>> +
>> +        if (y == null) {
>> +            throw new IllegalArgumentException("y must not be null");
>> +        }
>> +
>> +        if (x.length != y.length) {
>> +            throw new IllegalArgumentException(
>> +                    "x and y must contain the same number of elements");
>> +        }
>> +
>> +        if (x.length == 0) {
>> +            throw new IllegalArgumentException(
>> +                    "x and y must contain at least one element");
>> +        }
>> +    }
>> +
>> +    /**
>> +     * Calculates y[i] - x[i] for all i
>> +     *
>> +     * @param x
>> +     * @param y
>> +     * @throws IllegalArgumentException
>> +     *             if assumptions are not met
>> +     */
>> +    private double[] calculateDifferences(final double[] x, final double[] y)
>> +            throws IllegalArgumentException {
>> +
>> +        final double[] z = new double[x.length];
>> +
>> +        for (int i = 0; i < x.length; ++i) {
>> +            z[i] = y[i] - x[i];
>> +        }
>> +
>> +        return z;
>> +    }
>> +
>> +    /**
>> +     * Calculates |z[i]| for all i
>> +     *
>> +     * @param z
>> +     * @throws IllegalArgumentException
>> +     *             if assumptions are not met
>> +     */
>> +    private double[] calculateAbsoluteDifferences(final double[] z)
>> +            throws IllegalArgumentException {
>> +        if (z == null) {
>> +            throw new IllegalArgumentException("z must not be null");
>> +        }
>> +
>> +        if (z.length == 0) {
>> +            throw new IllegalArgumentException(
>> +                    "z must contain at least one element");
>> +        }
>> +
>> +        final double[] zAbs = new double[z.length];
>> +
>> +        for (int i = 0; i < z.length; ++i) {
>> +            zAbs[i] = FastMath.abs(z[i]);
>> +        }
>> +
>> +        return zAbs;
>> +    }
>> +
>> +    /**
>> +     * {@inheritDoc}
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @return wilcoxonSignedRank statistic (the larger of W+ and W-)
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met
>> +     */
>> +    public double wilcoxonSignedRank(final double[] x, final double[] y)
>> +            throws IllegalArgumentException {
>> +
>> +        ensureDataConformance(x, y);
>> +
>> +        // throws IllegalArgumentException if x and y are not correctly
>> +        // specified
>> +        final double[] z = calculateDifferences(x, y);
>> +        final double[] zAbs = calculateAbsoluteDifferences(z);
>> +
>> +        final double[] ranks = naturalRanking.rank(zAbs);
>> +
>> +        double Wplus = 0;
>> +
>> +        for (int i = 0; i < z.length; ++i) {
>> +            if (z[i] > 0) {
>> +                Wplus += ranks[i];
>> +            }
>> +        }
>> +
>> +        final int N = x.length;
>> +        final double Wminus = (((double) (N * (N + 1))) / 2.0) - Wplus;
>> +
>> +        return FastMath.max(Wplus, Wminus);
>> +    }
>> +
>> +    /**
>> +     * Algorithm inspired by
>> +     * http://www.fon.hum.uva.nl/Service/Statistics/Signed_Rank_Algorihms.html#C
>> +     * by Rob van Son, Institute of Phonetic Sciences & IFOTT,
>> +     * University of Amsterdam
>> +     *
>> +     * @param Wmax largest Wilcoxon signed rank value
>> +     * @param N number of subjects (corresponding to x.length)
>> +     * @return two-sided exact p-value
>> +     */
>> +    private double calculateExactPValue(final double Wmax, final int N) {
>> +
>> +        // Total number of outcomes (equal to 2^N but a lot faster)
>> +        final int m = 1 << N;
>> +
>> +        int largerRankSums = 0;
>> +
>> +        for (int i = 0; i < m; ++i) {
>> +            int rankSum = 0;
>> +
>> +            // Generate all possible rank sums
>> +            for (int j = 0; j < N; ++j) {
>> +
>> +                // (i >> j) & 1 extract i's j-th bit from the right
>> +                if (((i >> j) & 1) == 1) {
>> +                    rankSum += j + 1;
>> +                }
>> +            }
>> +
>> +            if (rankSum >= Wmax) {
>> +                ++largerRankSums;
>> +            }
>> +        }
>> +
>> +        /*
>> +         * largerRankSums / m gives the one-sided p-value, so it's multiplied
>> +         * with 2 to get the two-sided p-value
>> +         */
>> +        return 2 * ((double) largerRankSums) / ((double) m);
>> +    }
>> +
>> +    /**
>> +     * @param Wmin smallest Wilcoxon signed rank value
>> +     * @param N number of subjects (corresponding to x.length)
>> +     * @return two-sided asymptotic p-value
>> +     * @throws MathException if an error occurs computing the p-value
>> +     */
>> +    private double calculateAsymptoticPValue(final double Wmin, final int N) throws MathException {
>> +
>> +        final double ES = (double) (N * (N + 1)) / 4.0;
>> +
>> +        /* Same as (but saves computations):
>> +         * final double VarW = ((double) (N * (N + 1) * (2*N + 1))) / 24;
>> +         */
>> +        final double VarS = ES * ((double) (2 * N + 1) / 6.0);
>> +
>> +        // - 0.5 is a continuity correction
>> +        final double z = (Wmin - ES - 0.5) / FastMath.sqrt(VarS);
>> +
>> +        final NormalDistributionImpl standardNormal = new NormalDistributionImpl(0, 1);
>> +
>> +        return 2*standardNormal.cumulativeProbability(z);
>> +    }
>> +
>> +    /**
>> +     * {@inheritDoc}
>> +     *
>> +     * @param x
>> +     *            the first sample
>> +     * @param y
>> +     *            the second sample
>> +     * @param exactPValue
>> +     *            if the exact p-value is wanted (only for x.length <= 30)
>> +     * @return p-value
>> +     * @throws IllegalArgumentException
>> +     *             if preconditions are not met or exact p-value is wanted for
>> +     *             when x.length > 30
>> +     * @throws MathException
>> +     *             if an error occurs computing the p-value
>> +     */
>> +    public double wilcoxonSignedRankTest(final double[] x, final double[] y,
>> +            boolean exactPValue) throws IllegalArgumentException,
>> +            MathException {
>> +
>> +        ensureDataConformance(x, y);
>> +
>> +        final int N = x.length;
>> +        final double Wmax = wilcoxonSignedRank(x, y);
>> +
>> +        if (exactPValue && N > 30) {
>> +            throw new IllegalArgumentException("Exact test can only be made for N <= 30.");
>> +        }
>> +
>> +        if (exactPValue) {
>> +            return calculateExactPValue(Wmax, N);
>> +        } else {
>> +            final double Wmin = ( (double)(N*(N+1)) / 2.0 ) - Wmax;
>> +            return calculateAsymptoticPValue(Wmin, N);
>> +        }
>> +    }
>> +}
>>
>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java?rev=1053836&view=auto
>> ==============================================================================
>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java (added)
>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/MannWhitneyUTestTest.java Thu Dec 30 09:52:00 2010
>> @@ -0,0 +1,101 @@
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>> + * contributor license agreements.  See the NOTICE file distributed with
>> + * this work for additional information regarding copyright ownership.
>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>> + * (the "License"); you may not use this file except in compliance with
>> + * the License.  You may obtain a copy of the License at
>> + *
>> + *      http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +package org.apache.commons.math.stat.inference;
>> +
>> +import junit.framework.TestCase;
>> +
>> +/**
>> + * Test cases for the ChiSquareTestImpl class.
>> + *
>> + * @version $Revision: $ $Date: $
>> + */
>> +
>> +public class MannWhitneyUTestTest extends TestCase {
>> +
>> +    protected MannWhitneyUTest testStatistic = new MannWhitneyUTestImpl();
>> +
>> +    public MannWhitneyUTestTest(String name) {
>> +        super(name);
>> +    }
>> +
>> +    public void testMannWhitneyUSimple() throws Exception {
>> +        /* Target values computed using R version 2.11.1
>> +         * x <- c(19, 22, 16, 29, 24)
>> +         * y <- c(20, 11, 17, 12)
>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = FALSE, exact = FALSE, correct = FALSE)
>> +         * W = 17, p-value = 0.08641
>> +         */
>> +        final double x[] = {19, 22, 16, 29, 24};
>> +        final double y[] = {20, 11, 17, 12};
>> +
>> +        assertEquals(17, testStatistic.mannWhitneyU(x, y), 1e-10);
>> +        assertEquals(0.08641, testStatistic.mannWhitneyUTest(x, y), 1e-5);
>> +    }
>> +
>> +
>> +    public void testMannWhitneyUInputValidation() throws Exception {
>> +        /* Samples must be present, i.e. length > 0
>> +         */
>> +        try {
>> +            testStatistic.mannWhitneyUTest(new double[] { }, new double[] { 1.0 });
>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, new double[] { });
>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        /*
>> +         * x and y is null
>> +         */
>> +        try {
>> +            testStatistic.mannWhitneyUTest(null, null);
>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.mannWhitneyUTest(null, null);
>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        /*
>> +         * x or y is null
>> +         */
>> +        try {
>> +            testStatistic.mannWhitneyUTest(null, new double[] { 1.0 });
>> +            fail("x is null (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.mannWhitneyUTest(new double[] { 1.0 }, null);
>> +            fail("y is null (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +    }
>> +}
>>
>> Added: commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java
>> URL: http://svn.apache.org/viewvc/commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java?rev=1053836&view=auto
>> ==============================================================================
>> --- commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java (added)
>> +++ commons/proper/math/trunk/src/test/java/org/apache/commons/math/stat/inference/WilcoxonSignedRankTestTest.java Thu Dec 30 09:52:00 2010
>> @@ -0,0 +1,180 @@
>> +/*
>> + * Licensed to the Apache Software Foundation (ASF) under one or more
>> + * contributor license agreements.  See the NOTICE file distributed with
>> + * this work for additional information regarding copyright ownership.
>> + * The ASF licenses this file to You under the Apache License, Version 2.0
>> + * (the "License"); you may not use this file except in compliance with
>> + * the License.  You may obtain a copy of the License at
>> + *
>> + *      http://www.apache.org/licenses/LICENSE-2.0
>> + *
>> + * Unless required by applicable law or agreed to in writing, software
>> + * distributed under the License is distributed on an "AS IS" BASIS,
>> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
>> + * See the License for the specific language governing permissions and
>> + * limitations under the License.
>> + */
>> +package org.apache.commons.math.stat.inference;
>> +
>> +import junit.framework.TestCase;
>> +
>> +/**
>> + * Test cases for the ChiSquareTestImpl class.
>> + *
>> + * @version $Revision: $ $Date: $
>> + */
>> +
>> +public class WilcoxonSignedRankTestTest extends TestCase {
>> +
>> +    protected WilcoxonSignedRankTest testStatistic = new WilcoxonSignedRankTestImpl();
>> +
>> +    public WilcoxonSignedRankTestTest(String name) {
>> +        super(name);
>> +    }
>> +
>> +    public void testWilcoxonSignedRankSimple() throws Exception {
>> +        /* Target values computed using R version 2.11.1
>> +         * x <- c(1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30)
>> +         * y <- c(0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29)
>> +         */
>> +        final double x[] = {1.83, 0.50, 1.62, 2.48, 1.68, 1.88, 1.55, 3.06, 1.30};
>> +        final double y[] = {0.878, 0.647, 0.598, 2.05, 1.06, 1.29, 1.06, 3.14, 1.29};
>> +
>> +        /* EXACT:
>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = TRUE, correct = FALSE)
>> +         * V = 40, p-value = 0.03906
>> +         *
>> +         * Corresponds to the value obtained in R.
>> +         */
>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>> +        assertEquals(0.03906, testStatistic.wilcoxonSignedRankTest(x, y, true), 1e-5);
>> +
>> +        /* ASYMPTOTIC:
>> +         * wilcox.test(x, y, alternative = "two.sided", mu = 0, paired = TRUE, exact = FALSE, correct = FALSE)
>> +         * V = 40, p-value = 0.03815
>> +         *
>> +         * This is not entirely the same due to different corrects,
>> +         * e.g. http://mlsc.lboro.ac.uk/resources/statistics/wsrt.pdf
>> +         * and src/library/stats/R/wilcox.test.R in the R source
>> +         */
>> +        assertEquals(40, testStatistic.wilcoxonSignedRank(x, y), 1e-10);
>> +        assertEquals(0.0329693812, testStatistic.wilcoxonSignedRankTest(x, y, false), 1e-10);
>> +    }
>> +
>> +    public void testWilcoxonSignedRankInputValidation() throws Exception {
>> +        /*
>> +         * Exact only for sample size <= 30
>> +         */
>> +        final double[] x1 = new double[30];
>> +        final double[] x2 = new double[31];
>> +        final double[] y1 = new double[30];
>> +        final double[] y2 = new double[31];
>> +        for (int i = 0; i < 30; ++i) {
>> +            x1[i] = x2[i] = y1[i] = y2[i] = i;
>> +        }
>> +
>> +        // Exactly 30 is okay
>> +        testStatistic.wilcoxonSignedRankTest(x1, y1, true);
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(x2, y2, true);
>> +            fail("More than 30 samples and exact chosen, IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        /* Samples must be present, i.e. length > 0
>> +         */
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, true);
>> +            fail("x does not contain samples (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { }, new double[] { 1.0 }, false);
>> +            fail("x does not contain samples (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, true);
>> +            fail("y does not contain samples (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, new double[] { }, false);
>> +            fail("y does not contain samples (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        /* Samples not same size, i.e. cannot be pairred
>> +         */
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, true);
>> +            fail("x and y not same size (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0, 2.0 }, new double[] { 3.0 }, false);
>> +            fail("x and y not same size (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        /*
>> +         * x and y is null
>> +         */
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(null, null, true);
>> +            fail("x and y is null (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(null, null, false);
>> +            fail("x and y is null (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        /*
>> +         * x or y is null
>> +         */
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, true);
>> +            fail("x is null (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(null, new double[] { 1.0 }, false);
>> +            fail("x is null (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, true);
>> +            fail("y is null (exact), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +
>> +        try {
>> +            testStatistic.wilcoxonSignedRankTest(new double[] { 1.0 }, null, false);
>> +            fail("y is null (asymptotic), IllegalArgumentException expected");
>> +        } catch (IllegalArgumentException ex) {
>> +            // expected
>> +        }
>> +    }
>> +}
>>
>>
>>
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org