You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Gilles Sadowski <gi...@harfang.homelinux.org> on 2011/09/08 15:30:24 UTC
[Math] "NonPositiveDefiniteMatrixException" message and meaning
Hi.
In revision 1166674, I've added an argument to that exception so that it can
print the value that failed the test.
However, I also wonder whether the message should not be
---CUT---
not positive definite matrix: diagonal element at ({1},{1}) is not strictly larger than {2} ({0})"
---CUT---
instead of the current
---CUT---
not positive definite matrix: diagonal element at ({1},{1}) is smaller than {2} ({0})
---CUT---
In a class where the exceptionmay be thrown ("CholeskyDecompositionImpl"), the
test is (at line 128):
---CUT---
ltI[i] < absolutePositivityThreshold
---CUT---
Which will *not* fail if "absolutePositivityThreshold" is zero.
Changing it to
---CUT---
ltI[i] <= absolutePositivityThreshold
---CUT---
would allow to set the threshold to "0" exactly, for those cases where one
wants to avoid raising an exception (like where the matrix assumed to be
positive definite), but nevertheless wants to retain a basic fool-proof
check.
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Ted Dunning <te...@gmail.com>.
On Fri, Sep 9, 2011 at 1:54 PM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:
> > > The exceptions thrown should not be sub-classes of
> NumberIsTooSmallException
> > > because it is a matrix that is the problem and matrices are not numbers
> and
> > > do not have a total order.
>
> Of course, but as you've pointed out that it is difficult to relate the
> failure (which *is* an inequality test) to the input matrix, I thought that
> it was consistent to report the failure as precisely as possible.
>
Well, it was precision at the cost of all intelligibility.
> > I agree. I don't think a single message is going to work.
>
> Well, it can, by giving less details. IIUC this is what you propose below,
> by removing member fields.
>
Plausible.
> > Ted's
> > analysis is correct - what we are dealing with here is a property of
> > the whole matrix and certainly not a "too small number."
>
> I agree of course.
> I just don't see clearly yet how to report a detailed message.
> We should not forget that the threshold is also a user input, so it could
> be
> construed that this part of the failure story ("a too small number did not
> pass the test") should be reported back. The issue is how to integrate it
> to
> the "top-level" precondition failure.
>
How about these conditions:
- factorization succeeds. No exception thrown.
- rank deficient, positive semi-definite result.
Throws NonPositiveDefiniteMatrixException to the consternation of the
pedantic user. Message says "Input matrix was rank deficient to a tolerance
of {threshold}". Some versions of Cholesky decomposition may elect to
return a result in this case instead of throwing.
- non positive semi-definite result.
Throws NonPositiveDefiniteMatrixException. User is satisfied because this
really does describe the situation. Message says "Input is not positive
semi-definite". No reference to the threshold is really necessary here.
> > How we
> > determine that the property fails may be different for different
> > decomposition methods
>
> Currently, only "CholeskyDecompositionImpl" and
> "RectangularCholeskyDecompositionImpl" trigger this exception. So it should
> be possible to figure out how to customize the message at those two places.
>
I think that what I described above works for both.
> > or other use cases (like how it works as an
> > operator, see discussion here [1]).
>
> So, do you agree to add messages to the "ExceptionContext" at the point
> where the exception is raised? As I understand it, this would mean a quite
> simple "NonPositiveDefiniteMatrixException" (no fields; just a simple
> message with no placeholder), and use "addMessage" at the point of usage
> with a report about the failed test (namely how the threshold is used).
>
As a user, I don't think I would care about more details until I decide that
the implementation is broken. At that point, I will be looking at the
source anyway.
>
> > ... I also think we should be more precise in the
> > exception itself whether it is strict or semi, probably by defining
> > a different exception for positive semi-definite failure. IIRC, we
> > now use the same exception for both kinds of failure.
>
This is the pedantic user issue. It really doesn't matter if the
nomenclature here is *exactly* correct as long as the user understands
instantly what the problem is.
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Phil Steitz <ph...@gmail.com>.
On 9/9/11 1:54 PM, Gilles Sadowski wrote:
> On Fri, Sep 09, 2011 at 11:26:02AM -0700, Phil Steitz wrote:
>> On 9/9/11 9:27 AM, Ted Dunning wrote:
>>> I don't think that is correct.
>>>
>>> It is not the case that there was some element in the input that was too
>>> small. For instance, this matrix is positive definite:
>>>
>>> 3 -3 1
>>> 2 4 2
>>> 1 1 30
>>>
>>> While this one is not
>>>
>>> 3 -3 1
>>> 2 3 2
>>> 1 1 30
>>>
>>> The value of the diagonal that triggers the error message will be some
>>> negative number that does not appear in the original matrix.
>>>
>>> For a non-pivoting Cholesky, I think that the message should be more like
>>> "Input was not positive definite".
>>>
>>> For a pivoting rank-revealing Cholesky, I think that the result should by
>>> "Input was not semi-positive definite" or "Input was positive semi-definite
>>> but had rank {}" depending on the three way test.
>>>
>>> The exceptions thrown should not be sub-classes of NumberIsTooSmallException
>>> because it is a matrix that is the problem and matrices are not numbers and
>>> do not have a total order.
> Of course, but as you've pointed out that it is difficult to relate the
> failure (which *is* an inequality test) to the input matrix, I thought that
> it was consistent to report the failure as precisely as possible.
>
>> I agree. I don't think a single message is going to work.
> Well, it can, by giving less details. IIUC this is what you propose below,
> by removing member fields.
>
>> Ted's
>> analysis is correct - what we are dealing with here is a property of
>> the whole matrix and certainly not a "too small number."
> I agree of course.
> I just don't see clearly yet how to report a detailed message.
> We should not forget that the threshold is also a user input, so it could be
> construed that this part of the failure story ("a too small number did not
> pass the test") should be reported back. The issue is how to integrate it to
> the "top-level" precondition failure.
The user input (or default) is algorithm-dependent. I am not sure
we need to echo that back in the exception. The fact that an
exception is thrown indicates that in the context of the algorithm,
a failure of positive definiteness (or semi-definiteness) was
encountered. See comments to follow on what should be reported back
or put into the exception context.
>> How we
>> determine that the property fails may be different for different
>> decomposition methods
> Currently, only "CholeskyDecompositionImpl" and
> "RectangularCholeskyDecompositionImpl" trigger this exception. So it should
> be possible to figure out how to customize the message at those two places.
> This might be again a situation where the "ExceptionContext" can be used
> (i.e. add a customized message that reports what test fails).
Good place to start. I thought I had seen other direct invocations,
but I guess they have now been removed.
>
>> or other use cases (like how it works as an
>> operator, see discussion here [1]).
> So, do you agree to add messages to the "ExceptionContext" at the point
> where the exception is raised? As I understand it, this would mean a quite
> simple "NonPositiveDefiniteMatrixException" (no fields; just a simple
> message with no placeholder), and use "addMessage" at the point of usage
> with a report about the failed test (namely how the threshold is used).
I am not sure it is necessary to add anything to the exception
context in these cases because I don't know honestly what a caller
is going to do with the information. Basically, I agree with Ted's
comments on this - characterize the failure in language appropriate
to the algorithm at a level that makes sense in the context of the
caller.
>> I think the best thing to do is
>> to drop the member fields from the exception (because their meaning
>> is ambiguous, as more or less agreed in [1]) and allow the different
>> activations of the exception to customize the message (as in Ted's
>> second example above) or to include no message (appropriate in Ted's
>> first example). I also think we should be more precise in the
>> exception itself whether it is strict or semi, probably by defining
>> a different exception for positive semi-definite failure. IIRC, we
>> now use the same exception for both kinds of failure.
> Could you please review the usage of "NonPositiveDefiniteMatrixException"
> in the two classes indicated above, and let me know what messages are
> required (in each of three cases)?
I think Ted's suggestions are reasonable.
Ted - a patch illustrating this - even with just literal messages
for both Cholesky and RectangularCholesky - would be greatly
appreciated.
I also now agree with Ted that my previous suggestion that we split
semi- from strict as a separate exception is not necessary, as long
as we clearly specify in the javadoc (as the methods that trigger
this in linear and random now do) whether strict or semi positive
definiteness is the precondition. An argument could be made that a
boolean for strict/semi would make more sense as an attribute of the
exception than either of the currently defined fields, which I think
should be dropped.
Phil
>
>
> Thanks,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Fri, Sep 09, 2011 at 11:26:02AM -0700, Phil Steitz wrote:
> On 9/9/11 9:27 AM, Ted Dunning wrote:
> > I don't think that is correct.
> >
> > It is not the case that there was some element in the input that was too
> > small. For instance, this matrix is positive definite:
> >
> > 3 -3 1
> > 2 4 2
> > 1 1 30
> >
> > While this one is not
> >
> > 3 -3 1
> > 2 3 2
> > 1 1 30
> >
> > The value of the diagonal that triggers the error message will be some
> > negative number that does not appear in the original matrix.
> >
> > For a non-pivoting Cholesky, I think that the message should be more like
> > "Input was not positive definite".
> >
> > For a pivoting rank-revealing Cholesky, I think that the result should by
> > "Input was not semi-positive definite" or "Input was positive semi-definite
> > but had rank {}" depending on the three way test.
> >
> > The exceptions thrown should not be sub-classes of NumberIsTooSmallException
> > because it is a matrix that is the problem and matrices are not numbers and
> > do not have a total order.
Of course, but as you've pointed out that it is difficult to relate the
failure (which *is* an inequality test) to the input matrix, I thought that
it was consistent to report the failure as precisely as possible.
> I agree. I don't think a single message is going to work.
Well, it can, by giving less details. IIUC this is what you propose below,
by removing member fields.
> Ted's
> analysis is correct - what we are dealing with here is a property of
> the whole matrix and certainly not a "too small number."
I agree of course.
I just don't see clearly yet how to report a detailed message.
We should not forget that the threshold is also a user input, so it could be
construed that this part of the failure story ("a too small number did not
pass the test") should be reported back. The issue is how to integrate it to
the "top-level" precondition failure.
> How we
> determine that the property fails may be different for different
> decomposition methods
Currently, only "CholeskyDecompositionImpl" and
"RectangularCholeskyDecompositionImpl" trigger this exception. So it should
be possible to figure out how to customize the message at those two places.
This might be again a situation where the "ExceptionContext" can be used
(i.e. add a customized message that reports what test fails).
> or other use cases (like how it works as an
> operator, see discussion here [1]).
So, do you agree to add messages to the "ExceptionContext" at the point
where the exception is raised? As I understand it, this would mean a quite
simple "NonPositiveDefiniteMatrixException" (no fields; just a simple
message with no placeholder), and use "addMessage" at the point of usage
with a report about the failed test (namely how the threshold is used).
> I think the best thing to do is
> to drop the member fields from the exception (because their meaning
> is ambiguous, as more or less agreed in [1]) and allow the different
> activations of the exception to customize the message (as in Ted's
> second example above) or to include no message (appropriate in Ted's
> first example). I also think we should be more precise in the
> exception itself whether it is strict or semi, probably by defining
> a different exception for positive semi-definite failure. IIRC, we
> now use the same exception for both kinds of failure.
Could you please review the usage of "NonPositiveDefiniteMatrixException"
in the two classes indicated above, and let me know what messages are
required (in each of three cases)?
Thanks,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Phil Steitz <ph...@gmail.com>.
On 9/9/11 9:27 AM, Ted Dunning wrote:
> I don't think that is correct.
>
> It is not the case that there was some element in the input that was too
> small. For instance, this matrix is positive definite:
>
> 3 -3 1
> 2 4 2
> 1 1 30
>
> While this one is not
>
> 3 -3 1
> 2 3 2
> 1 1 30
>
> The value of the diagonal that triggers the error message will be some
> negative number that does not appear in the original matrix.
>
> For a non-pivoting Cholesky, I think that the message should be more like
> "Input was not positive definite".
>
> For a pivoting rank-revealing Cholesky, I think that the result should by
> "Input was not semi-positive definite" or "Input was positive semi-definite
> but had rank {}" depending on the three way test.
>
> The exceptions thrown should not be sub-classes of NumberIsTooSmallException
> because it is a matrix that is the problem and matrices are not numbers and
> do not have a total order.
I agree. I don't think a single message is going to work. Ted's
analysis is correct - what we are dealing with here is a property of
the whole matrix and certainly not a "too small number." How we
determine that the property fails may be different for different
decomposition methods or other use cases (like how it works as an
operator, see discussion here [1]). I think the best thing to do is
to drop the member fields from the exception (because their meaning
is ambiguous, as more or less agreed in [1]) and allow the different
activations of the exception to customize the message (as in Ted's
second example above) or to include no message (appropriate in Ted's
first example). I also think we should be more precise in the
exception itself whether it is strict or semi, probably by defining
a different exception for positive semi-definite failure. IIRC, we
now use the same exception for both kinds of failure.
Phil
[1] http://markmail.org/message/h3utw6u62lmz2al2
>
>
> On Fri, Sep 9, 2011 at 4:39 AM, Gilles Sadowski <
> gilles@harfang.homelinux.org> wrote:
>
>>> "Non-positive definiteness in input detected at diagonal element number
>> xxx"
>>
>> What do you think of the following message?
>>
>> <element value> is smaller than, or equal to, the minimum (<threshold
>> value>):
>> not positive definite matrix: value <element value> at index <element
>> index>
>>
>> where, in actual messages, the <...> will be the numeric values of course.
>>
>> "NonPositiveDefiniteMatrixException" would inherit from
>> "NumberIsTooSmallException". The rationale being that it exactly matches
>> the
>> test and that it is meaningless to report an element of the original
>> matrix,
>> as you've explained.
>>
>>> [...]
>> Gilles
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Ted Dunning <te...@gmail.com>.
I don't think that is correct.
It is not the case that there was some element in the input that was too
small. For instance, this matrix is positive definite:
3 -3 1
2 4 2
1 1 30
While this one is not
3 -3 1
2 3 2
1 1 30
The value of the diagonal that triggers the error message will be some
negative number that does not appear in the original matrix.
For a non-pivoting Cholesky, I think that the message should be more like
"Input was not positive definite".
For a pivoting rank-revealing Cholesky, I think that the result should by
"Input was not semi-positive definite" or "Input was positive semi-definite
but had rank {}" depending on the three way test.
The exceptions thrown should not be sub-classes of NumberIsTooSmallException
because it is a matrix that is the problem and matrices are not numbers and
do not have a total order.
On Fri, Sep 9, 2011 at 4:39 AM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:
> > "Non-positive definiteness in input detected at diagonal element number
> xxx"
>
> What do you think of the following message?
>
> <element value> is smaller than, or equal to, the minimum (<threshold
> value>):
> not positive definite matrix: value <element value> at index <element
> index>
>
> where, in actual messages, the <...> will be the numeric values of course.
>
> "NonPositiveDefiniteMatrixException" would inherit from
> "NumberIsTooSmallException". The rationale being that it exactly matches
> the
> test and that it is meaningless to report an element of the original
> matrix,
> as you've explained.
>
> > [...]
>
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Greg Sterijevski <gs...@gmail.com>.
Looks good to me.
On Fri, Sep 9, 2011 at 6:39 AM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:
> On Thu, Sep 08, 2011 at 04:55:46PM -0700, Ted Dunning wrote:
> > OK.
> >
> > Replace that with the correct value. I meant it to be an index.
> >
> > That doesn't change my other points. There is an inherent problem with
> > "less than" comments when you have subtracted several other elements
> > previously and only now notices that the remainder is less than some
> other
> > adjusted value. The user cannot relate either value back to anything
> that
> > they have and they may well look at their own matrix and see the opposite
> > condition. It is better to give a slightly more abstract message such as
> > "Non-positive definiteness in input detected at diagonal element number
> xxx"
>
> What do you think of the following message?
>
> <element value> is smaller than, or equal to, the minimum (<threshold
> value>):
> not positive definite matrix: value <element value> at index <element
> index>
>
> where, in actual messages, the <...> will be the numeric values of course.
>
> "NonPositiveDefiniteMatrixException" would inherit from
> "NumberIsTooSmallException". The rationale being that it exactly matches
> the
> test and that it is meaningless to report an element of the original
> matrix,
> as you've explained.
>
> > [...]
>
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Thu, Sep 08, 2011 at 04:55:46PM -0700, Ted Dunning wrote:
> OK.
>
> Replace that with the correct value. I meant it to be an index.
>
> That doesn't change my other points. There is an inherent problem with
> "less than" comments when you have subtracted several other elements
> previously and only now notices that the remainder is less than some other
> adjusted value. The user cannot relate either value back to anything that
> they have and they may well look at their own matrix and see the opposite
> condition. It is better to give a slightly more abstract message such as
> "Non-positive definiteness in input detected at diagonal element number xxx"
What do you think of the following message?
<element value> is smaller than, or equal to, the minimum (<threshold value>):
not positive definite matrix: value <element value> at index <element index>
where, in actual messages, the <...> will be the numeric values of course.
"NonPositiveDefiniteMatrixException" would inherit from
"NumberIsTooSmallException". The rationale being that it exactly matches the
test and that it is meaningless to report an element of the original matrix,
as you've explained.
> [...]
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Ted Dunning <te...@gmail.com>.
Actually, I think that the test in the Cholesky decomposition should be
three-pronged:
lt[i] > threshold proceed normally, use sqrt(lt[i])
lt[i] >= -threshold treat as zero. Either return rank-deficient result
if pivoting or
signal that original argument is not positive
definite.
lt[i] < -threshold treat as indication that original argument is not
semi positive definite.
This is different from being simply rank deficient
These tests, as framed above, are obviously sequential. This can be avoided
if you replace the second test with abs(lt[i]) <= threshold
On Fri, Sep 9, 2011 at 4:13 AM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:
> > > Replace that with the correct value. I meant it to be an index.
> > >
> > > That doesn't change my other points. There is an inherent problem with
> > > "less than" comments when you have subtracted several other elements
> > > previously and only now notices that the remainder is less than some
> other
> > > adjusted value. The user cannot relate either value back to anything
> that
> > > they have and they may well look at their own matrix and see the
> opposite
> > > condition. It is better to give a slightly more abstract message such
> as
> > > "Non-positive definiteness in input detected at diagonal element number
> > > xxx"
> > > where you will have to fill in the xxx since you know which
> place-holders
> > > are which. NPD-ness is the condition that the decomposition talks
> about to
> > > the user but that condition doesn't occur at any single point in the
> > > matrix... it is a property of the whole. All the algorithm can do is
> say
> > > "Badness happened, I found it this far along".
>
> OK, I understand. But do you agree that the test should be changed from
>
> ltI[i] < absolutePositivityThreshold
>
> to
>
> ltI[i] <= absolutePositivityThreshold
>
> ?
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Thu, Sep 08, 2011 at 09:27:18PM -0500, Greg Sterijevski wrote:
> I agree with Ted. You sometimes run into this issue with psuedoinverses. You
> discover a zero eigenvalue at index i. The user's natural inclination is to
> attribute it to the corresponding column of the data matrix.
>
> -Greg
>
> On Thu, Sep 8, 2011 at 6:55 PM, Ted Dunning <te...@gmail.com> wrote:
>
> > OK.
> >
> > Replace that with the correct value. I meant it to be an index.
> >
> > That doesn't change my other points. There is an inherent problem with
> > "less than" comments when you have subtracted several other elements
> > previously and only now notices that the remainder is less than some other
> > adjusted value. The user cannot relate either value back to anything that
> > they have and they may well look at their own matrix and see the opposite
> > condition. It is better to give a slightly more abstract message such as
> > "Non-positive definiteness in input detected at diagonal element number
> > xxx"
> > where you will have to fill in the xxx since you know which place-holders
> > are which. NPD-ness is the condition that the decomposition talks about to
> > the user but that condition doesn't occur at any single point in the
> > matrix... it is a property of the whole. All the algorithm can do is say
> > "Badness happened, I found it this far along".
OK, I understand. But do you agree that the test should be changed from
ltI[i] < absolutePositivityThreshold
to
ltI[i] <= absolutePositivityThreshold
?
Gilles
> > On Thu, Sep 8, 2011 at 3:27 PM, Gilles Sadowski <
> > gilles@harfang.homelinux.org> wrote:
> >
> > > > The matrix was detected to not be positive definite at diagonal
> > > element
> > > > {3}.
> > > >
> > > > Care should be taken to make sure that this comparison uses appropriate
> > > fuzz
> > > > so that cases that are simply rank-deficient get appropriate treatment.
> > >
> > > Sorry, I don't understand much of the above.
> > > However from reading the "{3}" I wonder whether there may be a
> > > misunderstanding. In the message pattern, "{0}", "{1}", "{2}" are
> > > placeholders:
> > > "{0}" stands for the value being tested (i.e. "ltI[i]" in the code
> > > excerpt)
> > > "{1}" stands for the row (and column) index (i.e. "i")
> > > "{2}" stands for the threshold.
> > >
> > > There is no "{3}" placeholder.
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Greg Sterijevski <gs...@gmail.com>.
I agree with Ted. You sometimes run into this issue with psuedoinverses. You
discover a zero eigenvalue at index i. The user's natural inclination is to
attribute it to the corresponding column of the data matrix.
-Greg
On Thu, Sep 8, 2011 at 6:55 PM, Ted Dunning <te...@gmail.com> wrote:
> OK.
>
> Replace that with the correct value. I meant it to be an index.
>
> That doesn't change my other points. There is an inherent problem with
> "less than" comments when you have subtracted several other elements
> previously and only now notices that the remainder is less than some other
> adjusted value. The user cannot relate either value back to anything that
> they have and they may well look at their own matrix and see the opposite
> condition. It is better to give a slightly more abstract message such as
> "Non-positive definiteness in input detected at diagonal element number
> xxx"
> where you will have to fill in the xxx since you know which place-holders
> are which. NPD-ness is the condition that the decomposition talks about to
> the user but that condition doesn't occur at any single point in the
> matrix... it is a property of the whole. All the algorithm can do is say
> "Badness happened, I found it this far along".
>
> On Thu, Sep 8, 2011 at 3:27 PM, Gilles Sadowski <
> gilles@harfang.homelinux.org> wrote:
>
> > > The matrix was detected to not be positive definite at diagonal
> > element
> > > {3}.
> > >
> > > Care should be taken to make sure that this comparison uses appropriate
> > fuzz
> > > so that cases that are simply rank-deficient get appropriate treatment.
> >
> > Sorry, I don't understand much of the above.
> > However from reading the "{3}" I wonder whether there may be a
> > misunderstanding. In the message pattern, "{0}", "{1}", "{2}" are
> > placeholders:
> > "{0}" stands for the value being tested (i.e. "ltI[i]" in the code
> > excerpt)
> > "{1}" stands for the row (and column) index (i.e. "i")
> > "{2}" stands for the threshold.
> >
> > There is no "{3}" placeholder.
> >
>
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Ted Dunning <te...@gmail.com>.
OK.
Replace that with the correct value. I meant it to be an index.
That doesn't change my other points. There is an inherent problem with
"less than" comments when you have subtracted several other elements
previously and only now notices that the remainder is less than some other
adjusted value. The user cannot relate either value back to anything that
they have and they may well look at their own matrix and see the opposite
condition. It is better to give a slightly more abstract message such as
"Non-positive definiteness in input detected at diagonal element number xxx"
where you will have to fill in the xxx since you know which place-holders
are which. NPD-ness is the condition that the decomposition talks about to
the user but that condition doesn't occur at any single point in the
matrix... it is a property of the whole. All the algorithm can do is say
"Badness happened, I found it this far along".
On Thu, Sep 8, 2011 at 3:27 PM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:
> > The matrix was detected to not be positive definite at diagonal
> element
> > {3}.
> >
> > Care should be taken to make sure that this comparison uses appropriate
> fuzz
> > so that cases that are simply rank-deficient get appropriate treatment.
>
> Sorry, I don't understand much of the above.
> However from reading the "{3}" I wonder whether there may be a
> misunderstanding. In the message pattern, "{0}", "{1}", "{2}" are
> placeholders:
> "{0}" stands for the value being tested (i.e. "ltI[i]" in the code
> excerpt)
> "{1}" stands for the row (and column) index (i.e. "i")
> "{2}" stands for the threshold.
>
> There is no "{3}" placeholder.
>
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Thu, Sep 08, 2011 at 09:36:47AM -0700, Ted Dunning wrote:
> Since the diagonal elements are heavily modified during the Cholesky
> decomposition, it isn't clear what these actually mean.
>
> With pivoting, the meaning of these is even less clear.
>
> Also, I thought that the test for non-positive definiteness was whether the
> diagonal element after reduction from previous rows and columns was
> negative. This is only a comparison between two elements in the case of the
> second diagonal element. For all subsequent elements, the question is more
> subtle.
>
> Perhaps the message should be more like:
>
> The matrix was detected to not be positive definite at diagonal element
> {3}.
>
> Care should be taken to make sure that this comparison uses appropriate fuzz
> so that cases that are simply rank-deficient get appropriate treatment.
Sorry, I don't understand much of the above.
However from reading the "{3}" I wonder whether there may be a
misunderstanding. In the message pattern, "{0}", "{1}", "{2}" are
placeholders:
"{0}" stands for the value being tested (i.e. "ltI[i]" in the code excerpt)
"{1}" stands for the row (and column) index (i.e. "i")
"{2}" stands for the threshold.
There is no "{3}" placeholder.
Regards,
Gilles
> On Thu, Sep 8, 2011 at 6:30 AM, Gilles Sadowski <
> gilles@harfang.homelinux.org> wrote:
>
> > Hi.
> >
> > In revision 1166674, I've added an argument to that exception so that it
> > can
> > print the value that failed the test.
> > However, I also wonder whether the message should not be
> > ---CUT---
> > not positive definite matrix: diagonal element at ({1},{1}) is not
> > strictly larger than {2} ({0})"
> > ---CUT---
> > instead of the current
> > ---CUT---
> > not positive definite matrix: diagonal element at ({1},{1}) is smaller
> > than {2} ({0})
> > ---CUT---
> >
> > In a class where the exceptionmay be thrown ("CholeskyDecompositionImpl"),
> > the
> > test is (at line 128):
> > ---CUT---
> > ltI[i] < absolutePositivityThreshold
> > ---CUT---
> > Which will *not* fail if "absolutePositivityThreshold" is zero.
> >
> > Changing it to
> > ---CUT---
> > ltI[i] <= absolutePositivityThreshold
> > ---CUT---
> > would allow to set the threshold to "0" exactly, for those cases where one
> > wants to avoid raising an exception (like where the matrix assumed to be
> > positive definite), but nevertheless wants to retain a basic fool-proof
> > check.
> >
> >
> > Regards,
> > Gilles
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] "NonPositiveDefiniteMatrixException" message and meaning
Posted by Ted Dunning <te...@gmail.com>.
Since the diagonal elements are heavily modified during the Cholesky
decomposition, it isn't clear what these actually mean.
With pivoting, the meaning of these is even less clear.
Also, I thought that the test for non-positive definiteness was whether the
diagonal element after reduction from previous rows and columns was
negative. This is only a comparison between two elements in the case of the
second diagonal element. For all subsequent elements, the question is more
subtle.
Perhaps the message should be more like:
The matrix was detected to not be positive definite at diagonal element
{3}.
Care should be taken to make sure that this comparison uses appropriate fuzz
so that cases that are simply rank-deficient get appropriate treatment.
On Thu, Sep 8, 2011 at 6:30 AM, Gilles Sadowski <
gilles@harfang.homelinux.org> wrote:
> Hi.
>
> In revision 1166674, I've added an argument to that exception so that it
> can
> print the value that failed the test.
> However, I also wonder whether the message should not be
> ---CUT---
> not positive definite matrix: diagonal element at ({1},{1}) is not
> strictly larger than {2} ({0})"
> ---CUT---
> instead of the current
> ---CUT---
> not positive definite matrix: diagonal element at ({1},{1}) is smaller
> than {2} ({0})
> ---CUT---
>
> In a class where the exceptionmay be thrown ("CholeskyDecompositionImpl"),
> the
> test is (at line 128):
> ---CUT---
> ltI[i] < absolutePositivityThreshold
> ---CUT---
> Which will *not* fail if "absolutePositivityThreshold" is zero.
>
> Changing it to
> ---CUT---
> ltI[i] <= absolutePositivityThreshold
> ---CUT---
> would allow to set the threshold to "0" exactly, for those cases where one
> wants to avoid raising an exception (like where the matrix assumed to be
> positive definite), but nevertheless wants to retain a basic fool-proof
> check.
>
>
> Regards,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>