[MATH] Interest in large patches for small cleanup / performance changes?

[Sean Owen <sr...@apache.org>]

In Math, is there any appetite for large patches containing many
instances of particular micro-optimizations? Examples:

- Replace:
    a[i][j] = a[i][j] + foo;
  with:
    a[i][j] += foo;
  … which is faster/leaner in the byte code by a little bit. It might
make a difference in many nested, tight loops.
- Inefficient toArray() calls with 0-length arg
- Using Map.entrySet() instead of keySet() + get()s
- Unnecessarily non-static private methods/classes
- StringBuffer vs StringBuilder
- etc.

There are some non-functional but still possibly useful, simplifying
changes too:

- Add @Deprecated annotations
- Fix broken method refs in javadoc
- Removing unnecessary boxing/unboxing
- Using foreach where possible
- etc.

I could go on for a while. Most of what I might otherwise suggest is
style-level stuff like fixing C-style array declarations which is
unlikely to be useful enough to justify. Or, it would involve changing
signatures somewhere, which is probably not cool.

If there’s interest in these sorts of things I can generate one or
more patches easily. The downside is the disruption of large patches,
perhaps breaking existing patches.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Luc Maisonobe <lu...@spaceroots.org>]

Le 03/11/2013 14:43, Gilles a écrit :
> On Sun, 3 Nov 2013 10:30:40 +0000, Sean Owen wrote:
>> On Sun, Nov 3, 2013 at 4:17 AM, Ted Dunning <te...@gmail.com>
>> wrote:
>>> Does this actually matter after the JIT takes hold?  And if the JIT
>>> doesn't
>>> care to optimize this away, does it even matter?
>>
>> Unclear. There aren't hard guarantees about what the JIT does, but
>> transformations like this should be easy and local. Doesn't optimize
>> != doesn't matter though. The 2D load/store is harder to match, and
>> there's one place it matters a bit in EigenDecomposition. I always
>> figure it helps to write an increment/decrement as what it is -- all
>> the better that it can't hurt speed.
>>
>> These are all fairly trivial. The motivation was more from consistency
>> in most cases, with an occasional small runtime benefit. I was more
>> wondering if, it were just a matter of pressing a button, it would be
>> desirable to zap a lot like this. I think the right approach is indeed
>> to propose a couple targeted changes and move on.
> 
> Standardization is the key concept. A worthy goal even it brings zero
> performance improvement.
> 
>>
>> Others that might be of interest, as food for thought:
>> - Standardizing literals? like saying "0.3" instead of ".3" or ".3d",
> 
> If the number is like 0.3, it probably has to be declared as a
> "private static final" field with a telling name.
> 
> A case that occurs often is "zero" (as "double").
> I prefer "0d".

I really don't like the d suffix. It reminds me nightmares about
Fortran. 0.0 is reasonably readable by everyone even without knowing
Java syntax, so go for it.

> 
>> and writing "2L" not "2l" since the latter looks like "21"
> 
> Same here.
> For zero as long, I'm for "0L".

Sure.

best regards,
Luc

> 
>> - Using log1p() for computing log(1+p) with a tiny bit more accuracy
>> in a few places
> 
> Certainly.
> 
> 
> Best,
> Gilles
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 3 Nov 2013 14:45:00 +0000, Sean Owen wrote:
> On Sun, Nov 3, 2013 at 1:43 PM, Gilles <gi...@harfang.homelinux.org> 
> wrote:
>> A case that occurs often is "zero" (as "double").
>> I prefer "0d".
>
> (I'm certainly not going to propose to change it, but my personal
> preference is 0.0 because it's obviously decimal even to newbies and
> can't be confused with a hex value. But this is trivial.)

No problem; "0.0" is fine.[1]
Hopefully, this can be agreed upon and become one of those dreaded
little rules...

>
>
>>> - Using log1p() for computing log(1+p) with a tiny bit more 
>>> accuracy
>>> in a few places
>>
>>
>> Certainly.
>
> OK. Maybe I should proceed with a few more JIRAs for consideration?
> I'm concerned about generating work and noise for committers, but, if
> you guys are OK with a couple more, I can propose a few.

IMHO, it's definitely not noise. The more so that you actually fix
those things!

Thanks,
Gilles


[1] Shall we say that I was concerned that we'd require contributors to
     type three characters instead of two... :-P


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Sean Owen <sr...@gmail.com>]

On Sun, Nov 3, 2013 at 1:43 PM, Gilles <gi...@harfang.homelinux.org> wrote:
> A case that occurs often is "zero" (as "double").
> I prefer "0d".

(I'm certainly not going to propose to change it, but my personal
preference is 0.0 because it's obviously decimal even to newbies and
can't be confused with a hex value. But this is trivial.)


>> - Using log1p() for computing log(1+p) with a tiny bit more accuracy
>> in a few places
>
>
> Certainly.

OK. Maybe I should proceed with a few more JIRAs for consideration?
I'm concerned about generating work and noise for committers, but, if
you guys are OK with a couple more, I can propose a few.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 3 Nov 2013 10:30:40 +0000, Sean Owen wrote:
> On Sun, Nov 3, 2013 at 4:17 AM, Ted Dunning <te...@gmail.com> 
> wrote:
>> Does this actually matter after the JIT takes hold?  And if the JIT 
>> doesn't
>> care to optimize this away, does it even matter?
>
> Unclear. There aren't hard guarantees about what the JIT does, but
> transformations like this should be easy and local. Doesn't optimize
> != doesn't matter though. The 2D load/store is harder to match, and
> there's one place it matters a bit in EigenDecomposition. I always
> figure it helps to write an increment/decrement as what it is -- all
> the better that it can't hurt speed.
>
> These are all fairly trivial. The motivation was more from 
> consistency
> in most cases, with an occasional small runtime benefit. I was more
> wondering if, it were just a matter of pressing a button, it would be
> desirable to zap a lot like this. I think the right approach is 
> indeed
> to propose a couple targeted changes and move on.

Standardization is the key concept. A worthy goal even it brings zero
performance improvement.

>
> Others that might be of interest, as food for thought:
> - Standardizing literals? like saying "0.3" instead of ".3" or ".3d",

If the number is like 0.3, it probably has to be declared as a
"private static final" field with a telling name.

A case that occurs often is "zero" (as "double").
I prefer "0d".

> and writing "2L" not "2l" since the latter looks like "21"

Same here.
For zero as long, I'm for "0L".

> - Using log1p() for computing log(1+p) with a tiny bit more accuracy
> in a few places

Certainly.


Best,
Gilles


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Sean Owen <sr...@gmail.com>]

On Sun, Nov 3, 2013 at 4:17 AM, Ted Dunning <te...@gmail.com> wrote:
> Does this actually matter after the JIT takes hold?  And if the JIT doesn't
> care to optimize this away, does it even matter?

Unclear. There aren't hard guarantees about what the JIT does, but
transformations like this should be easy and local. Doesn't optimize
!= doesn't matter though. The 2D load/store is harder to match, and
there's one place it matters a bit in EigenDecomposition. I always
figure it helps to write an increment/decrement as what it is -- all
the better that it can't hurt speed.

These are all fairly trivial. The motivation was more from consistency
in most cases, with an occasional small runtime benefit. I was more
wondering if, it were just a matter of pressing a button, it would be
desirable to zap a lot like this. I think the right approach is indeed
to propose a couple targeted changes and move on.

Others that might be of interest, as food for thought:
- Standardizing literals? like saying "0.3" instead of ".3" or ".3d",
and writing "2L" not "2l" since the latter looks like "21"
- Using log1p() for computing log(1+p) with a tiny bit more accuracy
in a few places

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Ted Dunning <te...@gmail.com>]

How many of these actually matter any more?


On Sat, Nov 2, 2013 at 7:52 AM, Sean Owen <sr...@apache.org> wrote:

> In Math, is there any appetite for large patches containing many
> instances of particular micro-optimizations? Examples:
>
> - Replace:
>     a[i][j] = a[i][j] + foo;
>   with:
>     a[i][j] += foo;
>   … which is faster/leaner in the byte code by a little bit. It might
> make a difference in many nested, tight loops.
>

Does this actually matter after the JIT takes hold?  And if the JIT doesn't
care to optimize this away, does it even matter?



> - Inefficient toArray() calls with 0-length arg
> - Using Map.entrySet() instead of keySet() + get()s
>

I think that this actually really does matter, but escape analysis has
gotten dramatically better lately and may make the associated object
creation much less of an issue.


> - Unnecessarily non-static private methods/classes
>

This is stylistic and important.


> - StringBuffer vs StringBuilder
>

I know for a fact that escape analysis in recent JVM's gets rid of the
locks in most StringBuilder idioms and this just doesn't matter any more.

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Thomas Neidhart <th...@gmail.com>]

On 11/02/2013 06:35 PM, Sean Owen wrote:
> On Sat, Nov 2, 2013 at 4:49 PM, Phil Steitz <ph...@gmail.com> wrote:
>> If the improvements make a material difference [1], by all means
>> open tickets and submit patches.  If they are just cosmetic, my
>> personal opinion is this should be done in small commits
>> incrementally if at all [2].  It is OK to open master tickets to
>> track style / optimization changes and do the work to review and
>> commit them incrementally.  The master ticket should be preceded by
>> discussion here to make sure we are all in agreement that the
>> changes are appropriate.
>>
>> You mention quite a few different kinds of thing above.  It is
>> probably best to break up into different discussion threads by
>> topical area and then open master tickets for the ones we agree to
>> fix / standardize.  Regarding the javadoc errors (broken javadoc
>> references, missing / incorrect @deprecates), there is no need for
>> discussion - just open tickets and we can commit the fixes.
> 
> I agree with that. I'm going to file two JIRAs and probably leave it
> for now. No point in getting into minutiae.
> 
> Let me turn it around: are there general clean-up tasks that people
> have wanted to do for a while but not gotten to? Maybe I could take a
> request or two and run with them.

Hi Sean,

imho the code is pretty clean and every committer takes a lot of effort
to keep it that way. Of course there are small little things here and
there that can be improved, as you noticed.

We have some areas that clearly need improvements:

 * userguide
 * examples

Sometimes the userguide is a bit out-of-date, or simply missing for a
package. I started with some examples under src/userguide, which is a
separate project with xchart as dependency to create graphs, which can
then also be used in the userguide.

If you are interested in working on some examples how to use
commons-math in a real-world scenario or improving the userguide, would
be awesome.

Other, maybe more interesting things are providing patches for open
issues / feature requests. You may also want to take a look at the

https://wiki.apache.org/commons/MathWishList

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 10 Nov 2013 00:27:28 +0100, Thomas Neidhart wrote:
> On 11/09/2013 11:21 PM, Gilles wrote:
>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>> [...]
>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>> found any.  There are quite a few that are similar, but differ 
>>>>>>> in
>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>> included / not included, etc.).  Please do not "collapse" 
>>>>>>> messages
>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>
>>>>>> FAILED_BRACKETING
>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>
>>>>> Look at the messages.  These are different.  They convey 
>>>>> different
>>>>> information and are appropriate in different contexts.  See 
>>>>> below.
>>>>
>>>> I've argued that context information should be constructed at the
>>>> point where the exception is thrown (where the context is known).
>>>> Not all combinations of exceptions and context need be present in
>>>> the pattern list.
>>>> This is the essence of my proposal below.
>>>>
>>>>>>
>>>>>> My position: the error (failed bracketing) should have its own
>>>>>> exception
>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>> message
>>>>>> built at exception instantiation.
>>>>>>
>>>>>> If we want to include an indication of location (despite it is
>>>>>> already
>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>> add methods
>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>> (?).
>>>>>> Then, we would have thos patterns in the list:
>>>>>>
>>>>>> BRACKETING
>>>>>> LINE_SEARCH
>>>>>>
>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>> intended to be
>>>>>> included in an exception.
>>>>>>
>>>>>>
>>>>>> A second "interesting" case is
>>>>>>
>>>>>> INVALID_ROUNDING_METHOD
>>>>>>
>>>>>> which mixes documentation with error description. Does anyone
>>>>>> really thinks
>>>>>> that the enumeration of the rounding methods in the error 
>>>>>> message
>>>>>> is necessary
>>>>>> or even helpful?
>>>>>
>>>>> When I throw an exception, I want to provide an error message 
>>>>> that
>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>> looking at a log or stack trace can make sense of.  That 
>>>>> sometimes
>>>>> means restating preconditions, sometimes pointing to boundary
>>>>> conditions, sometimes giving hints describing common causes of 
>>>>> the
>>>>> exception - lots of different things that depend on the API, the
>>>>> activation context and the nature of the exception.  The natural 
>>>>> way
>>>>> to do this is to use natural language sentences.  Please allow me 
>>>>> to
>>>>> retain a straightforward way to construct these messages and to
>>>>> maintain the specificity and meaning of the messages.
>>>>
>>>> IMHO, the level of details in the message is not needed: if the
>>>> exception
>>>> was thrown, the user should probably look at the documentation,
>>>> rather
>>>> than try another value at random; I'd say that it is harmful to
>>>> tempt the
>>>> users with something like "Pick another number". ;-)
>>>>
>>>> [Shouldn't we rather provide function where the rounding type is
>>>> an enum?]
>>>>
>>>> The main problem in those discussions is that you consider only 
>>>> "toy"
>>>> situations, where the message generated by Commons Math should
>>>> make sense
>>>> wherever the exception is caught, and even if it is not caught.
>>>
>>> What you keep failing to acknowledge is that in many real world
>>> applications, reading exception stack traces and application logs
>>> that contain error messages is an important operational activity.
>>> Having clear error messages that make sense in the context of the
>>> stack trace or application activation context makes the job of 
>>> those
>>> maintaining and debugging those applications easier.  However hard
>>> we decide to make it, I will continue to provide these.
>>
>> IMO, the real problem is old habits, period. Despite your repeating 
>> it
>> over and over, I never expressed anything in the sense of having 
>> less
>> information in the error messages. [I don't get what the stack trace 
>> has
>> to do here. And I just gave you a real example where whatever 
>> details
>> CM tries to provide, it will _never_ be sufficient because it cannot
>> know why the call failed; I suggested that the _same_ amount of
>> (necessary but not sufficient) information could perhaps be provided
>> with "little block" patterns glued with "addMessage" (or an 
>> improvement
>> thereof).]
>> Specific exceptions always provide more information than less 
>> specific
>> ones. Keeping low-level message (e.g. precondition failure) does not
>> preclude adding more specific messages when the context is known 
>> (that
>> happens in the code, and every little variant does not need to be
>> hard-coded in the currently overly long list of patterns).
>> My proposals were solely aimed at making the "preparation" of the
>> messages more efficient from a developer's perspective (e.g. no 
>> scanning
>> of 300+ patterns).
>> Stalling the experiment in endless arguments makes it less and less
>> worth trying.
>>
>> All in all, the main argument seems to always be that if the user
>> cannot see the difference, it is not worth changing the design.
>
> Which is also a pragmatic and valid approach here imho.

My point was that something can be worth, even if the user does not
see the difference. Sometimes the value appears later, if only in the
form of simplified code.

Some things are touched even if it really does not make a difference
to the user, e.g. blank spaces, indentation, etc.
This is for the _developer_.

>
> If there are no real user complaints about this topic (and I am not
> aware of any) and no other solution will greatly enhance the current
> state, it is really not worth doing it.

This thread was about "small cleanup"...

>
> Part of my day job is to debug very complex systems and the most
> important thing is that you get what you expect, i.e. according to 
> the
> contract of a method, which btw also includes the method name. 
> Detailed
> error messages are nice to have but not really required (as long as 
> you
> understand the purpose of the code which any user/developer of CM 
> should
> do).
>
> More meaningful error messages would make sense if our targeted 
> audience
> are really end-users but I think this is a bit far-fetched.
>
> My conclusion: the way we handle exceptions is not perfect but 
> totally
> sufficient imho.

Which way?
I recall that we discussed the inclusion of an ExceptionContext class,
which I implemented following a supposed "consensus". After not really
upgrading the code to take advantage of it, it is deemed not necessary
(because all was well and "sufficient" before it existed, I guess).



Gilles


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/10/13 9:39 AM, Thomas Neidhart wrote:
> On 11/10/2013 05:32 PM, Phil Steitz wrote:
>> On 11/10/13 1:15 AM, Thomas Neidhart wrote:
>>> On 11/10/2013 07:03 AM, Phil Steitz wrote:
>>>> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>>>>> On 11/09/2013 11:21 PM, Gilles wrote:
>>>>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>>>>> [...]
>>>>>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>>>>>> included / not included, etc.).  Please do not "collapse" messages
>>>>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>>>>> FAILED_BRACKETING
>>>>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>>>>> Look at the messages.  These are different.  They convey different
>>>>>>>>> information and are appropriate in different contexts.  See below.
>>>>>>>> I've argued that context information should be constructed at the
>>>>>>>> point where the exception is thrown (where the context is known).
>>>>>>>> Not all combinations of exceptions and context need be present in
>>>>>>>> the pattern list.
>>>>>>>> This is the essence of my proposal below.
>>>>>>>>
>>>>>>>>>> My position: the error (failed bracketing) should have its own
>>>>>>>>>> exception
>>>>>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>>>>>> message
>>>>>>>>>> built at exception instantiation.
>>>>>>>>>>
>>>>>>>>>> If we want to include an indication of location (despite it is
>>>>>>>>>> already
>>>>>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>>>>>> add methods
>>>>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>>>>>> (?).
>>>>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>>>>
>>>>>>>>>> BRACKETING
>>>>>>>>>> LINE_SEARCH
>>>>>>>>>>
>>>>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>>>>> intended to be
>>>>>>>>>> included in an exception.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> A second "interesting" case is
>>>>>>>>>>
>>>>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>>>>
>>>>>>>>>> which mixes documentation with error description. Does anyone
>>>>>>>>>> really thinks
>>>>>>>>>> that the enumeration of the rounding methods in the error message
>>>>>>>>>> is necessary
>>>>>>>>>> or even helpful?
>>>>>>>>> When I throw an exception, I want to provide an error message that
>>>>>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>>>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>>>>> conditions, sometimes giving hints describing common causes of the
>>>>>>>>> exception - lots of different things that depend on the API, the
>>>>>>>>> activation context and the nature of the exception.  The natural way
>>>>>>>>> to do this is to use natural language sentences.  Please allow me to
>>>>>>>>> retain a straightforward way to construct these messages and to
>>>>>>>>> maintain the specificity and meaning of the messages.
>>>>>>>> IMHO, the level of details in the message is not needed: if the
>>>>>>>> exception
>>>>>>>> was thrown, the user should probably look at the documentation,
>>>>>>>> rather
>>>>>>>> than try another value at random; I'd say that it is harmful to
>>>>>>>> tempt the
>>>>>>>> users with something like "Pick another number". ;-)
>>>>>>>>
>>>>>>>> [Shouldn't we rather provide function where the rounding type is
>>>>>>>> an enum?]
>>>>>>>>
>>>>>>>> The main problem in those discussions is that you consider only "toy"
>>>>>>>> situations, where the message generated by Commons Math should
>>>>>>>> make sense
>>>>>>>> wherever the exception is caught, and even if it is not caught.
>>>>>>> What you keep failing to acknowledge is that in many real world
>>>>>>> applications, reading exception stack traces and application logs
>>>>>>> that contain error messages is an important operational activity.
>>>>>>> Having clear error messages that make sense in the context of the
>>>>>>> stack trace or application activation context makes the job of those
>>>>>>> maintaining and debugging those applications easier.  However hard
>>>>>>> we decide to make it, I will continue to provide these.
>>>>>> IMO, the real problem is old habits, period. Despite your repeating it
>>>>>> over and over, I never expressed anything in the sense of having less
>>>>>> information in the error messages. [I don't get what the stack trace has
>>>>>> to do here. And I just gave you a real example where whatever details
>>>>>> CM tries to provide, it will _never_ be sufficient because it cannot
>>>>>> know why the call failed; I suggested that the _same_ amount of
>>>>>> (necessary but not sufficient) information could perhaps be provided
>>>>>> with "little block" patterns glued with "addMessage" (or an improvement
>>>>>> thereof).]
>>>>>> Specific exceptions always provide more information than less specific
>>>>>> ones. Keeping low-level message (e.g. precondition failure) does not
>>>>>> preclude adding more specific messages when the context is known (that
>>>>>> happens in the code, and every little variant does not need to be
>>>>>> hard-coded in the currently overly long list of patterns).
>>>>>> My proposals were solely aimed at making the "preparation" of the
>>>>>> messages more efficient from a developer's perspective (e.g. no scanning
>>>>>> of 300+ patterns).
>>>>>> Stalling the experiment in endless arguments makes it less and less
>>>>>> worth trying.
>>>>>>
>>>>>> All in all, the main argument seems to always be that if the user
>>>>>> cannot see the difference, it is not worth changing the design.
>>>>> Which is also a pragmatic and valid approach here imho.
>>>>>
>>>>> If there are no real user complaints about this topic (and I am not
>>>>> aware of any) and no other solution will greatly enhance the current
>>>>> state, it is really not worth doing it.
>>>>>
>>>>> Part of my day job is to debug very complex systems and the most
>>>>> important thing is that you get what you expect, i.e. according to the
>>>>> contract of a method, which btw also includes the method name. Detailed
>>>>> error messages are nice to have but not really required (as long as you
>>>>> understand the purpose of the code which any user/developer of CM should
>>>>> do).
>>>>>
>>>>> More meaningful error messages would make sense if our targeted audience
>>>>> are really end-users but I think this is a bit far-fetched.
>>>> Not really.  Does not have to be end users, just someone looking at
>>>> an application log that reproduces the messages or the stack traces
>>>> themselves.  Sometimes operations / production support teams do not
>>>> have access or time to look at source code or javadoc.  Informative
>>>> error messages that give more information about the failure can be
>>>> useful to these people.  Sure, you can push all of that off to the
>>>> client app developers; but it can make *their* lives easier too to
>>>> get more information, especially information about parameter values,
>>>> which invariants were violated, etc.  The simplest and easiest to
>>>> digest way to do that is to provide good error messages.
>>> If the contract is violated, then it is a bug and has to go to 3rd level
>>> support (aka developer) anyway and this you can see immediately if there
>>> is an IllegalArgumentException.
>> Not necessarily.  Could be a data or environment problem.
>>> For algorithm related problems, like MaxIterationExceededExceptions or
>>> similar ones can you really express why this happened? I think it is
>>> much more robust to take such exceptions into account in the first
>>> place, i.e. algorithms may not converge for certain inputs and a client
>>> app developer has to present a meaningful error (in the context of the
>>> application!) to a user or the log file.
>>>
>>> I even think that more detailed error messages give people the illusion
>>> that they can skip proper exception handling as CM already does it so well.
>> No, they just provide more info that can be useful in
>> troubleshooting or debugging, the most important of which is usually
>> info about application state when the error happened.
>>> Just an example: if you try to open a non-existent file with the Java
>>> API, do you get the error code of the respective system call on your
>>> operating system? No, you get a FileNotFoundException, but what do you
>>> do with it? 
>>> Would you log the error message contained in the exception
>>> or a more specific one in the context of your application?
>>>
>>> Imho, the first option makes applications very hard to maintain.
>> It is not either/or.  Depends on the application use case.  If on
>> the other hand you *don't* provide decent error messages, developers
>> don't have this choice.
>>> Now the Java API is quite easy to understand and a lot of people know it
>>> very well, but take the example further for any 3rd party library your
>>> application may be using. How is your operation engineer supposed to
>>> know all error states of all the used 3rd party libraries and put them
>>> in the right context? If he/she is lucky enough there is a an operations
>>> manual for their own application ...
>> Yep, such manuals exist all over the place in the real world.  And
>> the first failure data capture (FFDC) in the logs is used both by
>> first-level support as well as when issues are escalated.   As Bloch
>> points out, when failures are hard to reproduce, the FFDC data is
>> all that those working a production issue have to go on.  For this
>> reason, "it is critically important that an Exception's toString
>> method return as much information as possible about the cause of the
>> failure as possible..." [1]
>>> Interpreting and proper handling of 3rd party code is the job of the
>>> client app developer, and he/she has to do it right. If you may get an
>>> exception you have to take care about it, everything else will just
>>> create headaches.
>> Well, in real-world applications, headaches happen.  Preparing those
>> who have to deal with them with good FFDC can make it much easier to
>> deal with them. You are right that top notch developers using [math]
>> will manage all of this themselves, carefully preserving all context
>> data around failures and doing what they need to do to provide
>> themselves and whoever else helps support their apps with the info
>> they need.  Unfortunately, not all developers consistently do this
>> and when failures from lower-level libraries make it into stack
>> traces, it really helps for them to provide informative messages. 
>> Moving to unchecked exceptions uniformly makes this even more important.
> I am somehow lost, as from my understanding we have all these things in
> CM already. There are mainly two classes of exceptions that may occur:
>
>  * invalid input
>  * algorithm did not converge
>
> For both we provide in most cases meaningful error messages together
> with the exception. In the case of invalid input, Bloch clearly states
> that these are programming errors, and if something like this appears in
> a production environment you should rather question the development
> process than the FFDC policy of 3rd party libraries
>
> And for the case of convergence exceptions, I think it is very difficult
> to express more meaningful information into the error message than we
> already have. It is highly dependent on the context in which an
> algorithm is used, thus again, the application developer is in the best
> position to provide meaningful error information if something fails, as
> he/she knows precisely the context / purpose.
>
> I know the simplex algorithm quite well, and it may happen, dependent on
> the problem definition that the SimplexSolver may not find a solution
> after x iterations. So what do you put as an error message in such a case?
>
> The constraints may be too tight, the convergence criteria too strict,
> ... but in the end the solver could not find a solution. The developer
> must figure out why (or ask on the mailinglist ;-) and adjust the
> problem or parameters to the solver in order to make it work.
>
> So I agree with all the points stated in Effective Java and I think we
> sufficiently applied them in CM.

I agree with you here.  I just want to maintain the ability to keep
doing what we are already doing, which I agree is a good job
providing informative error messages.  Sorry for the noise.


Phil
>
> There are parts in CM where I have no clue how to use it and this is the
> more important issue to tackle imho.
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Thomas Neidhart <th...@gmail.com>]

On 11/10/2013 05:32 PM, Phil Steitz wrote:
> On 11/10/13 1:15 AM, Thomas Neidhart wrote:
>> On 11/10/2013 07:03 AM, Phil Steitz wrote:
>>> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>>>> On 11/09/2013 11:21 PM, Gilles wrote:
>>>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>>>> [...]
>>>>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>>>>> included / not included, etc.).  Please do not "collapse" messages
>>>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>>>> FAILED_BRACKETING
>>>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>>>> Look at the messages.  These are different.  They convey different
>>>>>>>> information and are appropriate in different contexts.  See below.
>>>>>>> I've argued that context information should be constructed at the
>>>>>>> point where the exception is thrown (where the context is known).
>>>>>>> Not all combinations of exceptions and context need be present in
>>>>>>> the pattern list.
>>>>>>> This is the essence of my proposal below.
>>>>>>>
>>>>>>>>> My position: the error (failed bracketing) should have its own
>>>>>>>>> exception
>>>>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>>>>> message
>>>>>>>>> built at exception instantiation.
>>>>>>>>>
>>>>>>>>> If we want to include an indication of location (despite it is
>>>>>>>>> already
>>>>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>>>>> add methods
>>>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>>>>> (?).
>>>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>>>
>>>>>>>>> BRACKETING
>>>>>>>>> LINE_SEARCH
>>>>>>>>>
>>>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>>>> intended to be
>>>>>>>>> included in an exception.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> A second "interesting" case is
>>>>>>>>>
>>>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>>>
>>>>>>>>> which mixes documentation with error description. Does anyone
>>>>>>>>> really thinks
>>>>>>>>> that the enumeration of the rounding methods in the error message
>>>>>>>>> is necessary
>>>>>>>>> or even helpful?
>>>>>>>> When I throw an exception, I want to provide an error message that
>>>>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>>>> conditions, sometimes giving hints describing common causes of the
>>>>>>>> exception - lots of different things that depend on the API, the
>>>>>>>> activation context and the nature of the exception.  The natural way
>>>>>>>> to do this is to use natural language sentences.  Please allow me to
>>>>>>>> retain a straightforward way to construct these messages and to
>>>>>>>> maintain the specificity and meaning of the messages.
>>>>>>> IMHO, the level of details in the message is not needed: if the
>>>>>>> exception
>>>>>>> was thrown, the user should probably look at the documentation,
>>>>>>> rather
>>>>>>> than try another value at random; I'd say that it is harmful to
>>>>>>> tempt the
>>>>>>> users with something like "Pick another number". ;-)
>>>>>>>
>>>>>>> [Shouldn't we rather provide function where the rounding type is
>>>>>>> an enum?]
>>>>>>>
>>>>>>> The main problem in those discussions is that you consider only "toy"
>>>>>>> situations, where the message generated by Commons Math should
>>>>>>> make sense
>>>>>>> wherever the exception is caught, and even if it is not caught.
>>>>>> What you keep failing to acknowledge is that in many real world
>>>>>> applications, reading exception stack traces and application logs
>>>>>> that contain error messages is an important operational activity.
>>>>>> Having clear error messages that make sense in the context of the
>>>>>> stack trace or application activation context makes the job of those
>>>>>> maintaining and debugging those applications easier.  However hard
>>>>>> we decide to make it, I will continue to provide these.
>>>>> IMO, the real problem is old habits, period. Despite your repeating it
>>>>> over and over, I never expressed anything in the sense of having less
>>>>> information in the error messages. [I don't get what the stack trace has
>>>>> to do here. And I just gave you a real example where whatever details
>>>>> CM tries to provide, it will _never_ be sufficient because it cannot
>>>>> know why the call failed; I suggested that the _same_ amount of
>>>>> (necessary but not sufficient) information could perhaps be provided
>>>>> with "little block" patterns glued with "addMessage" (or an improvement
>>>>> thereof).]
>>>>> Specific exceptions always provide more information than less specific
>>>>> ones. Keeping low-level message (e.g. precondition failure) does not
>>>>> preclude adding more specific messages when the context is known (that
>>>>> happens in the code, and every little variant does not need to be
>>>>> hard-coded in the currently overly long list of patterns).
>>>>> My proposals were solely aimed at making the "preparation" of the
>>>>> messages more efficient from a developer's perspective (e.g. no scanning
>>>>> of 300+ patterns).
>>>>> Stalling the experiment in endless arguments makes it less and less
>>>>> worth trying.
>>>>>
>>>>> All in all, the main argument seems to always be that if the user
>>>>> cannot see the difference, it is not worth changing the design.
>>>> Which is also a pragmatic and valid approach here imho.
>>>>
>>>> If there are no real user complaints about this topic (and I am not
>>>> aware of any) and no other solution will greatly enhance the current
>>>> state, it is really not worth doing it.
>>>>
>>>> Part of my day job is to debug very complex systems and the most
>>>> important thing is that you get what you expect, i.e. according to the
>>>> contract of a method, which btw also includes the method name. Detailed
>>>> error messages are nice to have but not really required (as long as you
>>>> understand the purpose of the code which any user/developer of CM should
>>>> do).
>>>>
>>>> More meaningful error messages would make sense if our targeted audience
>>>> are really end-users but I think this is a bit far-fetched.
>>> Not really.  Does not have to be end users, just someone looking at
>>> an application log that reproduces the messages or the stack traces
>>> themselves.  Sometimes operations / production support teams do not
>>> have access or time to look at source code or javadoc.  Informative
>>> error messages that give more information about the failure can be
>>> useful to these people.  Sure, you can push all of that off to the
>>> client app developers; but it can make *their* lives easier too to
>>> get more information, especially information about parameter values,
>>> which invariants were violated, etc.  The simplest and easiest to
>>> digest way to do that is to provide good error messages.
>> If the contract is violated, then it is a bug and has to go to 3rd level
>> support (aka developer) anyway and this you can see immediately if there
>> is an IllegalArgumentException.
> 
> Not necessarily.  Could be a data or environment problem.
>>
>> For algorithm related problems, like MaxIterationExceededExceptions or
>> similar ones can you really express why this happened? I think it is
>> much more robust to take such exceptions into account in the first
>> place, i.e. algorithms may not converge for certain inputs and a client
>> app developer has to present a meaningful error (in the context of the
>> application!) to a user or the log file.
>>
>> I even think that more detailed error messages give people the illusion
>> that they can skip proper exception handling as CM already does it so well.
> 
> No, they just provide more info that can be useful in
> troubleshooting or debugging, the most important of which is usually
> info about application state when the error happened.
>>
>> Just an example: if you try to open a non-existent file with the Java
>> API, do you get the error code of the respective system call on your
>> operating system? No, you get a FileNotFoundException, but what do you
>> do with it? 
>> Would you log the error message contained in the exception
>> or a more specific one in the context of your application?
>>
>> Imho, the first option makes applications very hard to maintain.
> 
> It is not either/or.  Depends on the application use case.  If on
> the other hand you *don't* provide decent error messages, developers
> don't have this choice.
>>
>> Now the Java API is quite easy to understand and a lot of people know it
>> very well, but take the example further for any 3rd party library your
>> application may be using. How is your operation engineer supposed to
>> know all error states of all the used 3rd party libraries and put them
>> in the right context? If he/she is lucky enough there is a an operations
>> manual for their own application ...
> 
> Yep, such manuals exist all over the place in the real world.  And
> the first failure data capture (FFDC) in the logs is used both by
> first-level support as well as when issues are escalated.   As Bloch
> points out, when failures are hard to reproduce, the FFDC data is
> all that those working a production issue have to go on.  For this
> reason, "it is critically important that an Exception's toString
> method return as much information as possible about the cause of the
> failure as possible..." [1]
>>
>> Interpreting and proper handling of 3rd party code is the job of the
>> client app developer, and he/she has to do it right. If you may get an
>> exception you have to take care about it, everything else will just
>> create headaches.
> 
> Well, in real-world applications, headaches happen.  Preparing those
> who have to deal with them with good FFDC can make it much easier to
> deal with them. You are right that top notch developers using [math]
> will manage all of this themselves, carefully preserving all context
> data around failures and doing what they need to do to provide
> themselves and whoever else helps support their apps with the info
> they need.  Unfortunately, not all developers consistently do this
> and when failures from lower-level libraries make it into stack
> traces, it really helps for them to provide informative messages. 
> Moving to unchecked exceptions uniformly makes this even more important.

I am somehow lost, as from my understanding we have all these things in
CM already. There are mainly two classes of exceptions that may occur:

 * invalid input
 * algorithm did not converge

For both we provide in most cases meaningful error messages together
with the exception. In the case of invalid input, Bloch clearly states
that these are programming errors, and if something like this appears in
a production environment you should rather question the development
process than the FFDC policy of 3rd party libraries.

And for the case of convergence exceptions, I think it is very difficult
to express more meaningful information into the error message than we
already have. It is highly dependent on the context in which an
algorithm is used, thus again, the application developer is in the best
position to provide meaningful error information if something fails, as
he/she knows precisely the context / purpose.

I know the simplex algorithm quite well, and it may happen, dependent on
the problem definition that the SimplexSolver may not find a solution
after x iterations. So what do you put as an error message in such a case?

The constraints may be too tight, the convergence criteria too strict,
... but in the end the solver could not find a solution. The developer
must figure out why (or ask on the mailinglist ;-) and adjust the
problem or parameters to the solver in order to make it work.

So I agree with all the points stated in Effective Java and I think we
sufficiently applied them in CM.

There are parts in CM where I have no clue how to use it and this is the
more important issue to tackle imho.

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/10/13 1:15 AM, Thomas Neidhart wrote:
> On 11/10/2013 07:03 AM, Phil Steitz wrote:
>> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>>> On 11/09/2013 11:21 PM, Gilles wrote:
>>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>>> [...]
>>>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>>>> included / not included, etc.).  Please do not "collapse" messages
>>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>>> FAILED_BRACKETING
>>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>>> Look at the messages.  These are different.  They convey different
>>>>>>> information and are appropriate in different contexts.  See below.
>>>>>> I've argued that context information should be constructed at the
>>>>>> point where the exception is thrown (where the context is known).
>>>>>> Not all combinations of exceptions and context need be present in
>>>>>> the pattern list.
>>>>>> This is the essence of my proposal below.
>>>>>>
>>>>>>>> My position: the error (failed bracketing) should have its own
>>>>>>>> exception
>>>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>>>> message
>>>>>>>> built at exception instantiation.
>>>>>>>>
>>>>>>>> If we want to include an indication of location (despite it is
>>>>>>>> already
>>>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>>>> add methods
>>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>>>> (?).
>>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>>
>>>>>>>> BRACKETING
>>>>>>>> LINE_SEARCH
>>>>>>>>
>>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>>> intended to be
>>>>>>>> included in an exception.
>>>>>>>>
>>>>>>>>
>>>>>>>> A second "interesting" case is
>>>>>>>>
>>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>>
>>>>>>>> which mixes documentation with error description. Does anyone
>>>>>>>> really thinks
>>>>>>>> that the enumeration of the rounding methods in the error message
>>>>>>>> is necessary
>>>>>>>> or even helpful?
>>>>>>> When I throw an exception, I want to provide an error message that
>>>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>>> conditions, sometimes giving hints describing common causes of the
>>>>>>> exception - lots of different things that depend on the API, the
>>>>>>> activation context and the nature of the exception.  The natural way
>>>>>>> to do this is to use natural language sentences.  Please allow me to
>>>>>>> retain a straightforward way to construct these messages and to
>>>>>>> maintain the specificity and meaning of the messages.
>>>>>> IMHO, the level of details in the message is not needed: if the
>>>>>> exception
>>>>>> was thrown, the user should probably look at the documentation,
>>>>>> rather
>>>>>> than try another value at random; I'd say that it is harmful to
>>>>>> tempt the
>>>>>> users with something like "Pick another number". ;-)
>>>>>>
>>>>>> [Shouldn't we rather provide function where the rounding type is
>>>>>> an enum?]
>>>>>>
>>>>>> The main problem in those discussions is that you consider only "toy"
>>>>>> situations, where the message generated by Commons Math should
>>>>>> make sense
>>>>>> wherever the exception is caught, and even if it is not caught.
>>>>> What you keep failing to acknowledge is that in many real world
>>>>> applications, reading exception stack traces and application logs
>>>>> that contain error messages is an important operational activity.
>>>>> Having clear error messages that make sense in the context of the
>>>>> stack trace or application activation context makes the job of those
>>>>> maintaining and debugging those applications easier.  However hard
>>>>> we decide to make it, I will continue to provide these.
>>>> IMO, the real problem is old habits, period. Despite your repeating it
>>>> over and over, I never expressed anything in the sense of having less
>>>> information in the error messages. [I don't get what the stack trace has
>>>> to do here. And I just gave you a real example where whatever details
>>>> CM tries to provide, it will _never_ be sufficient because it cannot
>>>> know why the call failed; I suggested that the _same_ amount of
>>>> (necessary but not sufficient) information could perhaps be provided
>>>> with "little block" patterns glued with "addMessage" (or an improvement
>>>> thereof).]
>>>> Specific exceptions always provide more information than less specific
>>>> ones. Keeping low-level message (e.g. precondition failure) does not
>>>> preclude adding more specific messages when the context is known (that
>>>> happens in the code, and every little variant does not need to be
>>>> hard-coded in the currently overly long list of patterns).
>>>> My proposals were solely aimed at making the "preparation" of the
>>>> messages more efficient from a developer's perspective (e.g. no scanning
>>>> of 300+ patterns).
>>>> Stalling the experiment in endless arguments makes it less and less
>>>> worth trying.
>>>>
>>>> All in all, the main argument seems to always be that if the user
>>>> cannot see the difference, it is not worth changing the design.
>>> Which is also a pragmatic and valid approach here imho.
>>>
>>> If there are no real user complaints about this topic (and I am not
>>> aware of any) and no other solution will greatly enhance the current
>>> state, it is really not worth doing it.
>>>
>>> Part of my day job is to debug very complex systems and the most
>>> important thing is that you get what you expect, i.e. according to the
>>> contract of a method, which btw also includes the method name. Detailed
>>> error messages are nice to have but not really required (as long as you
>>> understand the purpose of the code which any user/developer of CM should
>>> do).
>>>
>>> More meaningful error messages would make sense if our targeted audience
>>> are really end-users but I think this is a bit far-fetched.
>> Not really.  Does not have to be end users, just someone looking at
>> an application log that reproduces the messages or the stack traces
>> themselves.  Sometimes operations / production support teams do not
>> have access or time to look at source code or javadoc.  Informative
>> error messages that give more information about the failure can be
>> useful to these people.  Sure, you can push all of that off to the
>> client app developers; but it can make *their* lives easier too to
>> get more information, especially information about parameter values,
>> which invariants were violated, etc.  The simplest and easiest to
>> digest way to do that is to provide good error messages.
> If the contract is violated, then it is a bug and has to go to 3rd level
> support (aka developer) anyway and this you can see immediately if there
> is an IllegalArgumentException.

Not necessarily.  Could be a data or environment problem.
>
> For algorithm related problems, like MaxIterationExceededExceptions or
> similar ones can you really express why this happened? I think it is
> much more robust to take such exceptions into account in the first
> place, i.e. algorithms may not converge for certain inputs and a client
> app developer has to present a meaningful error (in the context of the
> application!) to a user or the log file.
>
> I even think that more detailed error messages give people the illusion
> that they can skip proper exception handling as CM already does it so well.

No, they just provide more info that can be useful in
troubleshooting or debugging, the most important of which is usually
info about application state when the error happened.
>
> Just an example: if you try to open a non-existent file with the Java
> API, do you get the error code of the respective system call on your
> operating system? No, you get a FileNotFoundException, but what do you
> do with it? 
> Would you log the error message contained in the exception
> or a more specific one in the context of your application?
>
> Imho, the first option makes applications very hard to maintain.

It is not either/or.  Depends on the application use case.  If on
the other hand you *don't* provide decent error messages, developers
don't have this choice.
>
> Now the Java API is quite easy to understand and a lot of people know it
> very well, but take the example further for any 3rd party library your
> application may be using. How is your operation engineer supposed to
> know all error states of all the used 3rd party libraries and put them
> in the right context? If he/she is lucky enough there is a an operations
> manual for their own application ...

Yep, such manuals exist all over the place in the real world.  And
the first failure data capture (FFDC) in the logs is used both by
first-level support as well as when issues are escalated.   As Bloch
points out, when failures are hard to reproduce, the FFDC data is
all that those working a production issue have to go on.  For this
reason, "it is critically important that an Exception's toString
method return as much information as possible about the cause of the
failure as possible..." [1]
>
> Interpreting and proper handling of 3rd party code is the job of the
> client app developer, and he/she has to do it right. If you may get an
> exception you have to take care about it, everything else will just
> create headaches.

Well, in real-world applications, headaches happen.  Preparing those
who have to deal with them with good FFDC can make it much easier to
deal with them. You are right that top notch developers using [math]
will manage all of this themselves, carefully preserving all context
data around failures and doing what they need to do to provide
themselves and whoever else helps support their apps with the info
they need.  Unfortunately, not all developers consistently do this
and when failures from lower-level libraries make it into stack
traces, it really helps for them to provide informative messages. 
Moving to unchecked exceptions uniformly makes this even more important.

Phil

[1] Effective Java - Item 45 "Include failure-capture data
information in detail messages"
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
> .
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 10 Nov 2013 16:11:59 +0100, Thomas Neidhart wrote:
> On 11/10/2013 03:04 PM, Gilles wrote:
>> On Sun, 10 Nov 2013 10:15:18 +0100, Thomas Neidhart wrote:
>>> On 11/10/2013 07:03 AM, Phil Steitz wrote:
>>>> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>>>>> On 11/09/2013 11:21 PM, Gilles wrote:
>>>>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>>>>> [...]
>>>>>>>>>>> I have scanned for exact duplicates quite a few times and 
>>>>>>>>>>> never
>>>>>>>>>>> found any.  There are quite a few that are similar, but 
>>>>>>>>>>> differ in
>>>>>>>>>>> material ways (strict versus non-strict inequalities, 
>>>>>>>>>>> endpoints
>>>>>>>>>>> included / not included, etc.).  Please do not "collapse"
>>>>>>>>>>> messages
>>>>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>>>>> FAILED_BRACKETING
>>>>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>>>>> Look at the messages.  These are different.  They convey 
>>>>>>>>> different
>>>>>>>>> information and are appropriate in different contexts.  See 
>>>>>>>>> below.
>>>>>>>> I've argued that context information should be constructed at 
>>>>>>>> the
>>>>>>>> point where the exception is thrown (where the context is 
>>>>>>>> known).
>>>>>>>> Not all combinations of exceptions and context need be present 
>>>>>>>> in
>>>>>>>> the pattern list.
>>>>>>>> This is the essence of my proposal below.
>>>>>>>>
>>>>>>>>>> My position: the error (failed bracketing) should have its 
>>>>>>>>>> own
>>>>>>>>>> exception
>>>>>>>>>> type. The varying contexts could (do not have to) be part of 
>>>>>>>>>> the
>>>>>>>>>> message
>>>>>>>>>> built at exception instantiation.
>>>>>>>>>>
>>>>>>>>>> If we want to include an indication of location (despite it 
>>>>>>>>>> is
>>>>>>>>>> already
>>>>>>>>>> part of the stack trace, so it is _redundant_), we could 
>>>>>>>>>> perhaps
>>>>>>>>>> add methods
>>>>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats 
>>>>>>>>>> pattern)"
>>>>>>>>>> (?).
>>>>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>>>>
>>>>>>>>>> BRACKETING
>>>>>>>>>> LINE_SEARCH
>>>>>>>>>>
>>>>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>>>>> intended to be
>>>>>>>>>> included in an exception.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> A second "interesting" case is
>>>>>>>>>>
>>>>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>>>>
>>>>>>>>>> which mixes documentation with error description. Does 
>>>>>>>>>> anyone
>>>>>>>>>> really thinks
>>>>>>>>>> that the enumeration of the rounding methods in the error 
>>>>>>>>>> message
>>>>>>>>>> is necessary
>>>>>>>>>> or even helpful?
>>>>>>>>> When I throw an exception, I want to provide an error message 
>>>>>>>>> that
>>>>>>>>> is meaningful in the context of the caller, i.e., that 
>>>>>>>>> someone
>>>>>>>>> looking at a log or stack trace can make sense of.  That 
>>>>>>>>> sometimes
>>>>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>>>>> conditions, sometimes giving hints describing common causes 
>>>>>>>>> of the
>>>>>>>>> exception - lots of different things that depend on the API, 
>>>>>>>>> the
>>>>>>>>> activation context and the nature of the exception.  The 
>>>>>>>>> natural
>>>>>>>>> way
>>>>>>>>> to do this is to use natural language sentences.  Please 
>>>>>>>>> allow
>>>>>>>>> me to
>>>>>>>>> retain a straightforward way to construct these messages and 
>>>>>>>>> to
>>>>>>>>> maintain the specificity and meaning of the messages.
>>>>>>>> IMHO, the level of details in the message is not needed: if 
>>>>>>>> the
>>>>>>>> exception
>>>>>>>> was thrown, the user should probably look at the 
>>>>>>>> documentation,
>>>>>>>> rather
>>>>>>>> than try another value at random; I'd say that it is harmful 
>>>>>>>> to
>>>>>>>> tempt the
>>>>>>>> users with something like "Pick another number". ;-)
>>>>>>>>
>>>>>>>> [Shouldn't we rather provide function where the rounding type 
>>>>>>>> is
>>>>>>>> an enum?]
>>>>>>>>
>>>>>>>> The main problem in those discussions is that you consider 
>>>>>>>> only
>>>>>>>> "toy"
>>>>>>>> situations, where the message generated by Commons Math should
>>>>>>>> make sense
>>>>>>>> wherever the exception is caught, and even if it is not 
>>>>>>>> caught.
>>>>>>> What you keep failing to acknowledge is that in many real world
>>>>>>> applications, reading exception stack traces and application 
>>>>>>> logs
>>>>>>> that contain error messages is an important operational 
>>>>>>> activity.
>>>>>>> Having clear error messages that make sense in the context of 
>>>>>>> the
>>>>>>> stack trace or application activation context makes the job of 
>>>>>>> those
>>>>>>> maintaining and debugging those applications easier.  However 
>>>>>>> hard
>>>>>>> we decide to make it, I will continue to provide these.
>>>>>> IMO, the real problem is old habits, period. Despite your 
>>>>>> repeating it
>>>>>> over and over, I never expressed anything in the sense of having 
>>>>>> less
>>>>>> information in the error messages. [I don't get what the stack
>>>>>> trace has
>>>>>> to do here. And I just gave you a real example where whatever 
>>>>>> details
>>>>>> CM tries to provide, it will _never_ be sufficient because it 
>>>>>> cannot
>>>>>> know why the call failed; I suggested that the _same_ amount of
>>>>>> (necessary but not sufficient) information could perhaps be 
>>>>>> provided
>>>>>> with "little block" patterns glued with "addMessage" (or an
>>>>>> improvement
>>>>>> thereof).]
>>>>>> Specific exceptions always provide more information than less 
>>>>>> specific
>>>>>> ones. Keeping low-level message (e.g. precondition failure) does 
>>>>>> not
>>>>>> preclude adding more specific messages when the context is known 
>>>>>> (that
>>>>>> happens in the code, and every little variant does not need to 
>>>>>> be
>>>>>> hard-coded in the currently overly long list of patterns).
>>>>>> My proposals were solely aimed at making the "preparation" of 
>>>>>> the
>>>>>> messages more efficient from a developer's perspective (e.g. no
>>>>>> scanning
>>>>>> of 300+ patterns).
>>>>>> Stalling the experiment in endless arguments makes it less and 
>>>>>> less
>>>>>> worth trying.
>>>>>>
>>>>>> All in all, the main argument seems to always be that if the 
>>>>>> user
>>>>>> cannot see the difference, it is not worth changing the design.
>>>>> Which is also a pragmatic and valid approach here imho.
>>>>>
>>>>> If there are no real user complaints about this topic (and I am 
>>>>> not
>>>>> aware of any) and no other solution will greatly enhance the 
>>>>> current
>>>>> state, it is really not worth doing it.
>>>>>
>>>>> Part of my day job is to debug very complex systems and the most
>>>>> important thing is that you get what you expect, i.e. according 
>>>>> to the
>>>>> contract of a method, which btw also includes the method name. 
>>>>> Detailed
>>>>> error messages are nice to have but not really required (as long 
>>>>> as you
>>>>> understand the purpose of the code which any user/developer of CM
>>>>> should
>>>>> do).
>>>>>
>>>>> More meaningful error messages would make sense if our targeted
>>>>> audience
>>>>> are really end-users but I think this is a bit far-fetched.
>>>>
>>>> Not really.  Does not have to be end users, just someone looking 
>>>> at
>>>> an application log that reproduces the messages or the stack 
>>>> traces
>>>> themselves.  Sometimes operations / production support teams do 
>>>> not
>>>> have access or time to look at source code or javadoc.  
>>>> Informative
>>>> error messages that give more information about the failure can be
>>>> useful to these people.  Sure, you can push all of that off to the
>>>> client app developers; but it can make *their* lives easier too to
>>>> get more information, especially information about parameter 
>>>> values,
>>>> which invariants were violated, etc.  The simplest and easiest to
>>>> digest way to do that is to provide good error messages.
>>>
>>> If the contract is violated, then it is a bug and has to go to 3rd 
>>> level
>>> support (aka developer) anyway and this you can see immediately if 
>>> there
>>> is an IllegalArgumentException.
>>>
>>> For algorithm related problems, like MaxIterationExceededExceptions 
>>> or
>>> similar ones can you really express why this happened? I think it 
>>> is
>>> much more robust to take such exceptions into account in the first
>>> place, i.e. algorithms may not converge for certain inputs and a 
>>> client
>>> app developer has to present a meaningful error (in the context of 
>>> the
>>> application!) to a user or the log file.
>>>
>>> I even think that more detailed error messages give people the 
>>> illusion
>>> that they can skip proper exception handling as CM already does it 
>>> so
>>> well.
>>>
>>> Just an example: if you try to open a non-existent file with the 
>>> Java
>>> API, do you get the error code of the respective system call on 
>>> your
>>> operating system? No, you get a FileNotFoundException, but what do 
>>> you
>>> do with it? Would you log the error message contained in the 
>>> exception
>>> or a more specific one in the context of your application?
>>>
>>> Imho, the first option makes applications very hard to maintain.
>>>
>>> Now the Java API is quite easy to understand and a lot of people 
>>> know it
>>> very well, but take the example further for any 3rd party library 
>>> your
>>> application may be using. How is your operation engineer supposed 
>>> to
>>> know all error states of all the used 3rd party libraries and put 
>>> them
>>> in the right context? If he/she is lucky enough there is a an 
>>> operations
>>> manual for their own application ...
>>>
>>> Interpreting and proper handling of 3rd party code is the job of 
>>> the
>>> client app developer, and he/she has to do it right. If you may get 
>>> an
>>> exception you have to take care about it, everything else will just
>>> create headaches.
>>
>> Very good account using the practical perspective of a multi-layered
>> application (I mean, where each layer is indeed developed by 
>> different
>> teams)!
>> [I had put forward this kind of arguments in my first raising of the
>> not-so-good feeling I had with the CM exception framework.]
>>
>> As a low-level library, the main focus must indeed to encapsulate 
>> error
>> conditions in the most flexible way for the layer next up (i.e. 
>> specific
>> exception classes) so that it can also act in a flexible way (catch, 
>> not
>> catch, catch and rethrow a context-specific exception).
>>
>> As I had said (too many times to be counted), this does not stand in
>> opposition with providing detailed error messages for the cases 
>> where
>> the intermediate layers do not care to catch the low level errors.
>> What top-level users can see from CM (through the stack trace) is 
>> the
>> best we can show (and that is quite fine by me, sorry to repeat 
>> myself
>> so often...). But my conviction is that we can do better internally 
>> (cf.
>> "small cleanup" of the error patterns list), and that we must do 
>> better
>> for the application or library developers who use CM as a component.
>>
>> Both these tasks should be thought about together; an improvement in
>> one area is likely to improve the other. [Again: I totally expect 
>> that
>> this will not change anything for a user who browses the stack trace
>> generated by a CM exception that was not caught.]
>>
>> So the real question was whether we are going to thwart any and all
>> attempts aimed at improvement on the basis of the single case of a
>> top-level user reading the low-level textual error messages.
>
> If you have a good proposal to reduce the number of localized 
> messages
> and thus make it easier to develop new code for CM I would be very 
> much
> in favor of it.

Hmm, I think I explained my proposal... Was it twice, three, ten times?
It's just not trying to have documentation inside the error messages.
Documentation contains all the details; error messages can be terse.

> As Luc said, it could be quite hard to come up with patterns that 
> really
> work in a multi-language context, but I did not try it myself or have
> seen proposals to overcome this complexity.

IMHO, this is a false problem. It is not necessary to write a 
syntactically
correct and complete sentence to perfectly convey the meaning of an 
exception
raised by such low-level checks as occur in CM.
Either the "operator" knows what happened (and why the exception was 
raised)
and he does not need a full and syntactically correct set of sentences 
to
figure out what must be changed for the call to succeed; or he doesn't 
and
it is fairly improbable that he can just guess without referring to the
documentation.

>
> I also did not follow so much the exception discussions as I am quite
> agnostic about it, but I wanted to point out that I consider good
> documentation and examples / user guide much more important and we
> should put our focus also on these issues (what I did try to do 
> lately
> and am going to continue).

These are orthogonal considerations.

Gilles

>
> Thomas


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Thomas Neidhart <th...@gmail.com>]

On 11/10/2013 03:04 PM, Gilles wrote:
> On Sun, 10 Nov 2013 10:15:18 +0100, Thomas Neidhart wrote:
>> On 11/10/2013 07:03 AM, Phil Steitz wrote:
>>> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>>>> On 11/09/2013 11:21 PM, Gilles wrote:
>>>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>>>> [...]
>>>>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>>>>> included / not included, etc.).  Please do not "collapse"
>>>>>>>>>> messages
>>>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>>>> FAILED_BRACKETING
>>>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>>>> Look at the messages.  These are different.  They convey different
>>>>>>>> information and are appropriate in different contexts.  See below.
>>>>>>> I've argued that context information should be constructed at the
>>>>>>> point where the exception is thrown (where the context is known).
>>>>>>> Not all combinations of exceptions and context need be present in
>>>>>>> the pattern list.
>>>>>>> This is the essence of my proposal below.
>>>>>>>
>>>>>>>>> My position: the error (failed bracketing) should have its own
>>>>>>>>> exception
>>>>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>>>>> message
>>>>>>>>> built at exception instantiation.
>>>>>>>>>
>>>>>>>>> If we want to include an indication of location (despite it is
>>>>>>>>> already
>>>>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>>>>> add methods
>>>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>>>>> (?).
>>>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>>>
>>>>>>>>> BRACKETING
>>>>>>>>> LINE_SEARCH
>>>>>>>>>
>>>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>>>> intended to be
>>>>>>>>> included in an exception.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> A second "interesting" case is
>>>>>>>>>
>>>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>>>
>>>>>>>>> which mixes documentation with error description. Does anyone
>>>>>>>>> really thinks
>>>>>>>>> that the enumeration of the rounding methods in the error message
>>>>>>>>> is necessary
>>>>>>>>> or even helpful?
>>>>>>>> When I throw an exception, I want to provide an error message that
>>>>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>>>> conditions, sometimes giving hints describing common causes of the
>>>>>>>> exception - lots of different things that depend on the API, the
>>>>>>>> activation context and the nature of the exception.  The natural
>>>>>>>> way
>>>>>>>> to do this is to use natural language sentences.  Please allow
>>>>>>>> me to
>>>>>>>> retain a straightforward way to construct these messages and to
>>>>>>>> maintain the specificity and meaning of the messages.
>>>>>>> IMHO, the level of details in the message is not needed: if the
>>>>>>> exception
>>>>>>> was thrown, the user should probably look at the documentation,
>>>>>>> rather
>>>>>>> than try another value at random; I'd say that it is harmful to
>>>>>>> tempt the
>>>>>>> users with something like "Pick another number". ;-)
>>>>>>>
>>>>>>> [Shouldn't we rather provide function where the rounding type is
>>>>>>> an enum?]
>>>>>>>
>>>>>>> The main problem in those discussions is that you consider only
>>>>>>> "toy"
>>>>>>> situations, where the message generated by Commons Math should
>>>>>>> make sense
>>>>>>> wherever the exception is caught, and even if it is not caught.
>>>>>> What you keep failing to acknowledge is that in many real world
>>>>>> applications, reading exception stack traces and application logs
>>>>>> that contain error messages is an important operational activity.
>>>>>> Having clear error messages that make sense in the context of the
>>>>>> stack trace or application activation context makes the job of those
>>>>>> maintaining and debugging those applications easier.  However hard
>>>>>> we decide to make it, I will continue to provide these.
>>>>> IMO, the real problem is old habits, period. Despite your repeating it
>>>>> over and over, I never expressed anything in the sense of having less
>>>>> information in the error messages. [I don't get what the stack
>>>>> trace has
>>>>> to do here. And I just gave you a real example where whatever details
>>>>> CM tries to provide, it will _never_ be sufficient because it cannot
>>>>> know why the call failed; I suggested that the _same_ amount of
>>>>> (necessary but not sufficient) information could perhaps be provided
>>>>> with "little block" patterns glued with "addMessage" (or an
>>>>> improvement
>>>>> thereof).]
>>>>> Specific exceptions always provide more information than less specific
>>>>> ones. Keeping low-level message (e.g. precondition failure) does not
>>>>> preclude adding more specific messages when the context is known (that
>>>>> happens in the code, and every little variant does not need to be
>>>>> hard-coded in the currently overly long list of patterns).
>>>>> My proposals were solely aimed at making the "preparation" of the
>>>>> messages more efficient from a developer's perspective (e.g. no
>>>>> scanning
>>>>> of 300+ patterns).
>>>>> Stalling the experiment in endless arguments makes it less and less
>>>>> worth trying.
>>>>>
>>>>> All in all, the main argument seems to always be that if the user
>>>>> cannot see the difference, it is not worth changing the design.
>>>> Which is also a pragmatic and valid approach here imho.
>>>>
>>>> If there are no real user complaints about this topic (and I am not
>>>> aware of any) and no other solution will greatly enhance the current
>>>> state, it is really not worth doing it.
>>>>
>>>> Part of my day job is to debug very complex systems and the most
>>>> important thing is that you get what you expect, i.e. according to the
>>>> contract of a method, which btw also includes the method name. Detailed
>>>> error messages are nice to have but not really required (as long as you
>>>> understand the purpose of the code which any user/developer of CM
>>>> should
>>>> do).
>>>>
>>>> More meaningful error messages would make sense if our targeted
>>>> audience
>>>> are really end-users but I think this is a bit far-fetched.
>>>
>>> Not really.  Does not have to be end users, just someone looking at
>>> an application log that reproduces the messages or the stack traces
>>> themselves.  Sometimes operations / production support teams do not
>>> have access or time to look at source code or javadoc.  Informative
>>> error messages that give more information about the failure can be
>>> useful to these people.  Sure, you can push all of that off to the
>>> client app developers; but it can make *their* lives easier too to
>>> get more information, especially information about parameter values,
>>> which invariants were violated, etc.  The simplest and easiest to
>>> digest way to do that is to provide good error messages.
>>
>> If the contract is violated, then it is a bug and has to go to 3rd level
>> support (aka developer) anyway and this you can see immediately if there
>> is an IllegalArgumentException.
>>
>> For algorithm related problems, like MaxIterationExceededExceptions or
>> similar ones can you really express why this happened? I think it is
>> much more robust to take such exceptions into account in the first
>> place, i.e. algorithms may not converge for certain inputs and a client
>> app developer has to present a meaningful error (in the context of the
>> application!) to a user or the log file.
>>
>> I even think that more detailed error messages give people the illusion
>> that they can skip proper exception handling as CM already does it so
>> well.
>>
>> Just an example: if you try to open a non-existent file with the Java
>> API, do you get the error code of the respective system call on your
>> operating system? No, you get a FileNotFoundException, but what do you
>> do with it? Would you log the error message contained in the exception
>> or a more specific one in the context of your application?
>>
>> Imho, the first option makes applications very hard to maintain.
>>
>> Now the Java API is quite easy to understand and a lot of people know it
>> very well, but take the example further for any 3rd party library your
>> application may be using. How is your operation engineer supposed to
>> know all error states of all the used 3rd party libraries and put them
>> in the right context? If he/she is lucky enough there is a an operations
>> manual for their own application ...
>>
>> Interpreting and proper handling of 3rd party code is the job of the
>> client app developer, and he/she has to do it right. If you may get an
>> exception you have to take care about it, everything else will just
>> create headaches.
> 
> Very good account using the practical perspective of a multi-layered
> application (I mean, where each layer is indeed developed by different
> teams)!
> [I had put forward this kind of arguments in my first raising of the
> not-so-good feeling I had with the CM exception framework.]
> 
> As a low-level library, the main focus must indeed to encapsulate error
> conditions in the most flexible way for the layer next up (i.e. specific
> exception classes) so that it can also act in a flexible way (catch, not
> catch, catch and rethrow a context-specific exception).
> 
> As I had said (too many times to be counted), this does not stand in
> opposition with providing detailed error messages for the cases where
> the intermediate layers do not care to catch the low level errors.
> What top-level users can see from CM (through the stack trace) is the
> best we can show (and that is quite fine by me, sorry to repeat myself
> so often...). But my conviction is that we can do better internally (cf.
> "small cleanup" of the error patterns list), and that we must do better
> for the application or library developers who use CM as a component.
> 
> Both these tasks should be thought about together; an improvement in
> one area is likely to improve the other. [Again: I totally expect that
> this will not change anything for a user who browses the stack trace
> generated by a CM exception that was not caught.]
> 
> So the real question was whether we are going to thwart any and all
> attempts aimed at improvement on the basis of the single case of a
> top-level user reading the low-level textual error messages.

If you have a good proposal to reduce the number of localized messages
and thus make it easier to develop new code for CM I would be very much
in favor of it.

As Luc said, it could be quite hard to come up with patterns that really
work in a multi-language context, but I did not try it myself or have
seen proposals to overcome this complexity.

I also did not follow so much the exception discussions as I am quite
agnostic about it, but I wanted to point out that I consider good
documentation and examples / user guide much more important and we
should put our focus also on these issues (what I did try to do lately
and am going to continue).

Thomas


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 10 Nov 2013 10:15:18 +0100, Thomas Neidhart wrote:
> On 11/10/2013 07:03 AM, Phil Steitz wrote:
>> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>>> On 11/09/2013 11:21 PM, Gilles wrote:
>>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>>> [...]
>>>>>>>>> I have scanned for exact duplicates quite a few times and 
>>>>>>>>> never
>>>>>>>>> found any.  There are quite a few that are similar, but 
>>>>>>>>> differ in
>>>>>>>>> material ways (strict versus non-strict inequalities, 
>>>>>>>>> endpoints
>>>>>>>>> included / not included, etc.).  Please do not "collapse" 
>>>>>>>>> messages
>>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>>> FAILED_BRACKETING
>>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>>> Look at the messages.  These are different.  They convey 
>>>>>>> different
>>>>>>> information and are appropriate in different contexts.  See 
>>>>>>> below.
>>>>>> I've argued that context information should be constructed at 
>>>>>> the
>>>>>> point where the exception is thrown (where the context is 
>>>>>> known).
>>>>>> Not all combinations of exceptions and context need be present 
>>>>>> in
>>>>>> the pattern list.
>>>>>> This is the essence of my proposal below.
>>>>>>
>>>>>>>> My position: the error (failed bracketing) should have its own
>>>>>>>> exception
>>>>>>>> type. The varying contexts could (do not have to) be part of 
>>>>>>>> the
>>>>>>>> message
>>>>>>>> built at exception instantiation.
>>>>>>>>
>>>>>>>> If we want to include an indication of location (despite it is
>>>>>>>> already
>>>>>>>> part of the stack trace, so it is _redundant_), we could 
>>>>>>>> perhaps
>>>>>>>> add methods
>>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats 
>>>>>>>> pattern)"
>>>>>>>> (?).
>>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>>
>>>>>>>> BRACKETING
>>>>>>>> LINE_SEARCH
>>>>>>>>
>>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>>> intended to be
>>>>>>>> included in an exception.
>>>>>>>>
>>>>>>>>
>>>>>>>> A second "interesting" case is
>>>>>>>>
>>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>>
>>>>>>>> which mixes documentation with error description. Does anyone
>>>>>>>> really thinks
>>>>>>>> that the enumeration of the rounding methods in the error 
>>>>>>>> message
>>>>>>>> is necessary
>>>>>>>> or even helpful?
>>>>>>> When I throw an exception, I want to provide an error message 
>>>>>>> that
>>>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>>>> looking at a log or stack trace can make sense of.  That 
>>>>>>> sometimes
>>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>>> conditions, sometimes giving hints describing common causes of 
>>>>>>> the
>>>>>>> exception - lots of different things that depend on the API, 
>>>>>>> the
>>>>>>> activation context and the nature of the exception.  The 
>>>>>>> natural way
>>>>>>> to do this is to use natural language sentences.  Please allow 
>>>>>>> me to
>>>>>>> retain a straightforward way to construct these messages and to
>>>>>>> maintain the specificity and meaning of the messages.
>>>>>> IMHO, the level of details in the message is not needed: if the
>>>>>> exception
>>>>>> was thrown, the user should probably look at the documentation,
>>>>>> rather
>>>>>> than try another value at random; I'd say that it is harmful to
>>>>>> tempt the
>>>>>> users with something like "Pick another number". ;-)
>>>>>>
>>>>>> [Shouldn't we rather provide function where the rounding type is
>>>>>> an enum?]
>>>>>>
>>>>>> The main problem in those discussions is that you consider only 
>>>>>> "toy"
>>>>>> situations, where the message generated by Commons Math should
>>>>>> make sense
>>>>>> wherever the exception is caught, and even if it is not caught.
>>>>> What you keep failing to acknowledge is that in many real world
>>>>> applications, reading exception stack traces and application logs
>>>>> that contain error messages is an important operational activity.
>>>>> Having clear error messages that make sense in the context of the
>>>>> stack trace or application activation context makes the job of 
>>>>> those
>>>>> maintaining and debugging those applications easier.  However 
>>>>> hard
>>>>> we decide to make it, I will continue to provide these.
>>>> IMO, the real problem is old habits, period. Despite your 
>>>> repeating it
>>>> over and over, I never expressed anything in the sense of having 
>>>> less
>>>> information in the error messages. [I don't get what the stack 
>>>> trace has
>>>> to do here. And I just gave you a real example where whatever 
>>>> details
>>>> CM tries to provide, it will _never_ be sufficient because it 
>>>> cannot
>>>> know why the call failed; I suggested that the _same_ amount of
>>>> (necessary but not sufficient) information could perhaps be 
>>>> provided
>>>> with "little block" patterns glued with "addMessage" (or an 
>>>> improvement
>>>> thereof).]
>>>> Specific exceptions always provide more information than less 
>>>> specific
>>>> ones. Keeping low-level message (e.g. precondition failure) does 
>>>> not
>>>> preclude adding more specific messages when the context is known 
>>>> (that
>>>> happens in the code, and every little variant does not need to be
>>>> hard-coded in the currently overly long list of patterns).
>>>> My proposals were solely aimed at making the "preparation" of the
>>>> messages more efficient from a developer's perspective (e.g. no 
>>>> scanning
>>>> of 300+ patterns).
>>>> Stalling the experiment in endless arguments makes it less and 
>>>> less
>>>> worth trying.
>>>>
>>>> All in all, the main argument seems to always be that if the user
>>>> cannot see the difference, it is not worth changing the design.
>>> Which is also a pragmatic and valid approach here imho.
>>>
>>> If there are no real user complaints about this topic (and I am not
>>> aware of any) and no other solution will greatly enhance the 
>>> current
>>> state, it is really not worth doing it.
>>>
>>> Part of my day job is to debug very complex systems and the most
>>> important thing is that you get what you expect, i.e. according to 
>>> the
>>> contract of a method, which btw also includes the method name. 
>>> Detailed
>>> error messages are nice to have but not really required (as long as 
>>> you
>>> understand the purpose of the code which any user/developer of CM 
>>> should
>>> do).
>>>
>>> More meaningful error messages would make sense if our targeted 
>>> audience
>>> are really end-users but I think this is a bit far-fetched.
>>
>> Not really.  Does not have to be end users, just someone looking at
>> an application log that reproduces the messages or the stack traces
>> themselves.  Sometimes operations / production support teams do not
>> have access or time to look at source code or javadoc.  Informative
>> error messages that give more information about the failure can be
>> useful to these people.  Sure, you can push all of that off to the
>> client app developers; but it can make *their* lives easier too to
>> get more information, especially information about parameter values,
>> which invariants were violated, etc.  The simplest and easiest to
>> digest way to do that is to provide good error messages.
>
> If the contract is violated, then it is a bug and has to go to 3rd 
> level
> support (aka developer) anyway and this you can see immediately if 
> there
> is an IllegalArgumentException.
>
> For algorithm related problems, like MaxIterationExceededExceptions 
> or
> similar ones can you really express why this happened? I think it is
> much more robust to take such exceptions into account in the first
> place, i.e. algorithms may not converge for certain inputs and a 
> client
> app developer has to present a meaningful error (in the context of 
> the
> application!) to a user or the log file.
>
> I even think that more detailed error messages give people the 
> illusion
> that they can skip proper exception handling as CM already does it so 
> well.
>
> Just an example: if you try to open a non-existent file with the Java
> API, do you get the error code of the respective system call on your
> operating system? No, you get a FileNotFoundException, but what do 
> you
> do with it? Would you log the error message contained in the 
> exception
> or a more specific one in the context of your application?
>
> Imho, the first option makes applications very hard to maintain.
>
> Now the Java API is quite easy to understand and a lot of people know 
> it
> very well, but take the example further for any 3rd party library 
> your
> application may be using. How is your operation engineer supposed to
> know all error states of all the used 3rd party libraries and put 
> them
> in the right context? If he/she is lucky enough there is a an 
> operations
> manual for their own application ...
>
> Interpreting and proper handling of 3rd party code is the job of the
> client app developer, and he/she has to do it right. If you may get 
> an
> exception you have to take care about it, everything else will just
> create headaches.

Very good account using the practical perspective of a multi-layered
application (I mean, where each layer is indeed developed by different
teams)!
[I had put forward this kind of arguments in my first raising of the
not-so-good feeling I had with the CM exception framework.]

As a low-level library, the main focus must indeed to encapsulate error
conditions in the most flexible way for the layer next up (i.e. 
specific
exception classes) so that it can also act in a flexible way (catch, 
not
catch, catch and rethrow a context-specific exception).

As I had said (too many times to be counted), this does not stand in
opposition with providing detailed error messages for the cases where
the intermediate layers do not care to catch the low level errors.
What top-level users can see from CM (through the stack trace) is the
best we can show (and that is quite fine by me, sorry to repeat myself
so often...). But my conviction is that we can do better internally 
(cf.
"small cleanup" of the error patterns list), and that we must do better
for the application or library developers who use CM as a component.

Both these tasks should be thought about together; an improvement in
one area is likely to improve the other. [Again: I totally expect that
this will not change anything for a user who browses the stack trace
generated by a CM exception that was not caught.]

So the real question was whether we are going to thwart any and all
attempts aimed at improvement on the basis of the single case of a
top-level user reading the low-level textual error messages.


Gilles


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Thomas Neidhart <th...@gmail.com>]

On 11/10/2013 07:03 AM, Phil Steitz wrote:
> On 11/9/13 3:27 PM, Thomas Neidhart wrote:
>> On 11/09/2013 11:21 PM, Gilles wrote:
>>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>>> [...]
>>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>>> included / not included, etc.).  Please do not "collapse" messages
>>>>>>>> at the expense of loss of specificity or correctness.
>>>>>>> FAILED_BRACKETING
>>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>>> Look at the messages.  These are different.  They convey different
>>>>>> information and are appropriate in different contexts.  See below.
>>>>> I've argued that context information should be constructed at the
>>>>> point where the exception is thrown (where the context is known).
>>>>> Not all combinations of exceptions and context need be present in
>>>>> the pattern list.
>>>>> This is the essence of my proposal below.
>>>>>
>>>>>>> My position: the error (failed bracketing) should have its own
>>>>>>> exception
>>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>>> message
>>>>>>> built at exception instantiation.
>>>>>>>
>>>>>>> If we want to include an indication of location (despite it is
>>>>>>> already
>>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>>> add methods
>>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>>> (?).
>>>>>>> Then, we would have thos patterns in the list:
>>>>>>>
>>>>>>> BRACKETING
>>>>>>> LINE_SEARCH
>>>>>>>
>>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>>> intended to be
>>>>>>> included in an exception.
>>>>>>>
>>>>>>>
>>>>>>> A second "interesting" case is
>>>>>>>
>>>>>>> INVALID_ROUNDING_METHOD
>>>>>>>
>>>>>>> which mixes documentation with error description. Does anyone
>>>>>>> really thinks
>>>>>>> that the enumeration of the rounding methods in the error message
>>>>>>> is necessary
>>>>>>> or even helpful?
>>>>>> When I throw an exception, I want to provide an error message that
>>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>>>> means restating preconditions, sometimes pointing to boundary
>>>>>> conditions, sometimes giving hints describing common causes of the
>>>>>> exception - lots of different things that depend on the API, the
>>>>>> activation context and the nature of the exception.  The natural way
>>>>>> to do this is to use natural language sentences.  Please allow me to
>>>>>> retain a straightforward way to construct these messages and to
>>>>>> maintain the specificity and meaning of the messages.
>>>>> IMHO, the level of details in the message is not needed: if the
>>>>> exception
>>>>> was thrown, the user should probably look at the documentation,
>>>>> rather
>>>>> than try another value at random; I'd say that it is harmful to
>>>>> tempt the
>>>>> users with something like "Pick another number". ;-)
>>>>>
>>>>> [Shouldn't we rather provide function where the rounding type is
>>>>> an enum?]
>>>>>
>>>>> The main problem in those discussions is that you consider only "toy"
>>>>> situations, where the message generated by Commons Math should
>>>>> make sense
>>>>> wherever the exception is caught, and even if it is not caught.
>>>> What you keep failing to acknowledge is that in many real world
>>>> applications, reading exception stack traces and application logs
>>>> that contain error messages is an important operational activity.
>>>> Having clear error messages that make sense in the context of the
>>>> stack trace or application activation context makes the job of those
>>>> maintaining and debugging those applications easier.  However hard
>>>> we decide to make it, I will continue to provide these.
>>> IMO, the real problem is old habits, period. Despite your repeating it
>>> over and over, I never expressed anything in the sense of having less
>>> information in the error messages. [I don't get what the stack trace has
>>> to do here. And I just gave you a real example where whatever details
>>> CM tries to provide, it will _never_ be sufficient because it cannot
>>> know why the call failed; I suggested that the _same_ amount of
>>> (necessary but not sufficient) information could perhaps be provided
>>> with "little block" patterns glued with "addMessage" (or an improvement
>>> thereof).]
>>> Specific exceptions always provide more information than less specific
>>> ones. Keeping low-level message (e.g. precondition failure) does not
>>> preclude adding more specific messages when the context is known (that
>>> happens in the code, and every little variant does not need to be
>>> hard-coded in the currently overly long list of patterns).
>>> My proposals were solely aimed at making the "preparation" of the
>>> messages more efficient from a developer's perspective (e.g. no scanning
>>> of 300+ patterns).
>>> Stalling the experiment in endless arguments makes it less and less
>>> worth trying.
>>>
>>> All in all, the main argument seems to always be that if the user
>>> cannot see the difference, it is not worth changing the design.
>> Which is also a pragmatic and valid approach here imho.
>>
>> If there are no real user complaints about this topic (and I am not
>> aware of any) and no other solution will greatly enhance the current
>> state, it is really not worth doing it.
>>
>> Part of my day job is to debug very complex systems and the most
>> important thing is that you get what you expect, i.e. according to the
>> contract of a method, which btw also includes the method name. Detailed
>> error messages are nice to have but not really required (as long as you
>> understand the purpose of the code which any user/developer of CM should
>> do).
>>
>> More meaningful error messages would make sense if our targeted audience
>> are really end-users but I think this is a bit far-fetched.
> 
> Not really.  Does not have to be end users, just someone looking at
> an application log that reproduces the messages or the stack traces
> themselves.  Sometimes operations / production support teams do not
> have access or time to look at source code or javadoc.  Informative
> error messages that give more information about the failure can be
> useful to these people.  Sure, you can push all of that off to the
> client app developers; but it can make *their* lives easier too to
> get more information, especially information about parameter values,
> which invariants were violated, etc.  The simplest and easiest to
> digest way to do that is to provide good error messages.

If the contract is violated, then it is a bug and has to go to 3rd level
support (aka developer) anyway and this you can see immediately if there
is an IllegalArgumentException.

For algorithm related problems, like MaxIterationExceededExceptions or
similar ones can you really express why this happened? I think it is
much more robust to take such exceptions into account in the first
place, i.e. algorithms may not converge for certain inputs and a client
app developer has to present a meaningful error (in the context of the
application!) to a user or the log file.

I even think that more detailed error messages give people the illusion
that they can skip proper exception handling as CM already does it so well.

Just an example: if you try to open a non-existent file with the Java
API, do you get the error code of the respective system call on your
operating system? No, you get a FileNotFoundException, but what do you
do with it? Would you log the error message contained in the exception
or a more specific one in the context of your application?

Imho, the first option makes applications very hard to maintain.

Now the Java API is quite easy to understand and a lot of people know it
very well, but take the example further for any 3rd party library your
application may be using. How is your operation engineer supposed to
know all error states of all the used 3rd party libraries and put them
in the right context? If he/she is lucky enough there is a an operations
manual for their own application ...

Interpreting and proper handling of 3rd party code is the job of the
client app developer, and he/she has to do it right. If you may get an
exception you have to take care about it, everything else will just
create headaches.

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/9/13 3:27 PM, Thomas Neidhart wrote:
> On 11/09/2013 11:21 PM, Gilles wrote:
>> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>>> [...]
>>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>>> included / not included, etc.).  Please do not "collapse" messages
>>>>>>> at the expense of loss of specificity or correctness.
>>>>>> FAILED_BRACKETING
>>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>>> INVALID_BRACKETING_PARAMETERS
>>>>> Look at the messages.  These are different.  They convey different
>>>>> information and are appropriate in different contexts.  See below.
>>>> I've argued that context information should be constructed at the
>>>> point where the exception is thrown (where the context is known).
>>>> Not all combinations of exceptions and context need be present in
>>>> the pattern list.
>>>> This is the essence of my proposal below.
>>>>
>>>>>> My position: the error (failed bracketing) should have its own
>>>>>> exception
>>>>>> type. The varying contexts could (do not have to) be part of the
>>>>>> message
>>>>>> built at exception instantiation.
>>>>>>
>>>>>> If we want to include an indication of location (despite it is
>>>>>> already
>>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>>> add methods
>>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>>> (?).
>>>>>> Then, we would have thos patterns in the list:
>>>>>>
>>>>>> BRACKETING
>>>>>> LINE_SEARCH
>>>>>>
>>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>>> intended to be
>>>>>> included in an exception.
>>>>>>
>>>>>>
>>>>>> A second "interesting" case is
>>>>>>
>>>>>> INVALID_ROUNDING_METHOD
>>>>>>
>>>>>> which mixes documentation with error description. Does anyone
>>>>>> really thinks
>>>>>> that the enumeration of the rounding methods in the error message
>>>>>> is necessary
>>>>>> or even helpful?
>>>>> When I throw an exception, I want to provide an error message that
>>>>> is meaningful in the context of the caller, i.e., that someone
>>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>>> means restating preconditions, sometimes pointing to boundary
>>>>> conditions, sometimes giving hints describing common causes of the
>>>>> exception - lots of different things that depend on the API, the
>>>>> activation context and the nature of the exception.  The natural way
>>>>> to do this is to use natural language sentences.  Please allow me to
>>>>> retain a straightforward way to construct these messages and to
>>>>> maintain the specificity and meaning of the messages.
>>>> IMHO, the level of details in the message is not needed: if the
>>>> exception
>>>> was thrown, the user should probably look at the documentation,
>>>> rather
>>>> than try another value at random; I'd say that it is harmful to
>>>> tempt the
>>>> users with something like "Pick another number". ;-)
>>>>
>>>> [Shouldn't we rather provide function where the rounding type is
>>>> an enum?]
>>>>
>>>> The main problem in those discussions is that you consider only "toy"
>>>> situations, where the message generated by Commons Math should
>>>> make sense
>>>> wherever the exception is caught, and even if it is not caught.
>>> What you keep failing to acknowledge is that in many real world
>>> applications, reading exception stack traces and application logs
>>> that contain error messages is an important operational activity.
>>> Having clear error messages that make sense in the context of the
>>> stack trace or application activation context makes the job of those
>>> maintaining and debugging those applications easier.  However hard
>>> we decide to make it, I will continue to provide these.
>> IMO, the real problem is old habits, period. Despite your repeating it
>> over and over, I never expressed anything in the sense of having less
>> information in the error messages. [I don't get what the stack trace has
>> to do here. And I just gave you a real example where whatever details
>> CM tries to provide, it will _never_ be sufficient because it cannot
>> know why the call failed; I suggested that the _same_ amount of
>> (necessary but not sufficient) information could perhaps be provided
>> with "little block" patterns glued with "addMessage" (or an improvement
>> thereof).]
>> Specific exceptions always provide more information than less specific
>> ones. Keeping low-level message (e.g. precondition failure) does not
>> preclude adding more specific messages when the context is known (that
>> happens in the code, and every little variant does not need to be
>> hard-coded in the currently overly long list of patterns).
>> My proposals were solely aimed at making the "preparation" of the
>> messages more efficient from a developer's perspective (e.g. no scanning
>> of 300+ patterns).
>> Stalling the experiment in endless arguments makes it less and less
>> worth trying.
>>
>> All in all, the main argument seems to always be that if the user
>> cannot see the difference, it is not worth changing the design.
> Which is also a pragmatic and valid approach here imho.
>
> If there are no real user complaints about this topic (and I am not
> aware of any) and no other solution will greatly enhance the current
> state, it is really not worth doing it.
>
> Part of my day job is to debug very complex systems and the most
> important thing is that you get what you expect, i.e. according to the
> contract of a method, which btw also includes the method name. Detailed
> error messages are nice to have but not really required (as long as you
> understand the purpose of the code which any user/developer of CM should
> do).
>
> More meaningful error messages would make sense if our targeted audience
> are really end-users but I think this is a bit far-fetched.

Not really.  Does not have to be end users, just someone looking at
an application log that reproduces the messages or the stack traces
themselves.  Sometimes operations / production support teams do not
have access or time to look at source code or javadoc.  Informative
error messages that give more information about the failure can be
useful to these people.  Sure, you can push all of that off to the
client app developers; but it can make *their* lives easier too to
get more information, especially information about parameter values,
which invariants were violated, etc.  The simplest and easiest to
digest way to do that is to provide good error messages.

Phil
>
> My conclusion: the way we handle exceptions is not perfect but totally
> sufficient imho.
>
> Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Thomas Neidhart <th...@gmail.com>]

On 11/09/2013 11:21 PM, Gilles wrote:
> On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
>> On 11/5/13 5:21 AM, Gilles wrote:
>>>>>> [...]
>>>>>> I have scanned for exact duplicates quite a few times and never
>>>>>> found any.  There are quite a few that are similar, but differ in
>>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>>> included / not included, etc.).  Please do not "collapse" messages
>>>>>> at the expense of loss of specificity or correctness.
>>>>>
>>>>> FAILED_BRACKETING
>>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>>> INVALID_BRACKETING_PARAMETERS
>>>>
>>>> Look at the messages.  These are different.  They convey different
>>>> information and are appropriate in different contexts.  See below.
>>>
>>> I've argued that context information should be constructed at the
>>> point where the exception is thrown (where the context is known).
>>> Not all combinations of exceptions and context need be present in
>>> the pattern list.
>>> This is the essence of my proposal below.
>>>
>>>>>
>>>>> My position: the error (failed bracketing) should have its own
>>>>> exception
>>>>> type. The varying contexts could (do not have to) be part of the
>>>>> message
>>>>> built at exception instantiation.
>>>>>
>>>>> If we want to include an indication of location (despite it is
>>>>> already
>>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>>> add methods
>>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>>> (?).
>>>>> Then, we would have thos patterns in the list:
>>>>>
>>>>> BRACKETING
>>>>> LINE_SEARCH
>>>>>
>>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>>> intended to be
>>>>> included in an exception.
>>>>>
>>>>>
>>>>> A second "interesting" case is
>>>>>
>>>>> INVALID_ROUNDING_METHOD
>>>>>
>>>>> which mixes documentation with error description. Does anyone
>>>>> really thinks
>>>>> that the enumeration of the rounding methods in the error message
>>>>> is necessary
>>>>> or even helpful?
>>>>
>>>> When I throw an exception, I want to provide an error message that
>>>> is meaningful in the context of the caller, i.e., that someone
>>>> looking at a log or stack trace can make sense of.  That sometimes
>>>> means restating preconditions, sometimes pointing to boundary
>>>> conditions, sometimes giving hints describing common causes of the
>>>> exception - lots of different things that depend on the API, the
>>>> activation context and the nature of the exception.  The natural way
>>>> to do this is to use natural language sentences.  Please allow me to
>>>> retain a straightforward way to construct these messages and to
>>>> maintain the specificity and meaning of the messages.
>>>
>>> IMHO, the level of details in the message is not needed: if the
>>> exception
>>> was thrown, the user should probably look at the documentation,
>>> rather
>>> than try another value at random; I'd say that it is harmful to
>>> tempt the
>>> users with something like "Pick another number". ;-)
>>>
>>> [Shouldn't we rather provide function where the rounding type is
>>> an enum?]
>>>
>>> The main problem in those discussions is that you consider only "toy"
>>> situations, where the message generated by Commons Math should
>>> make sense
>>> wherever the exception is caught, and even if it is not caught.
>>
>> What you keep failing to acknowledge is that in many real world
>> applications, reading exception stack traces and application logs
>> that contain error messages is an important operational activity.
>> Having clear error messages that make sense in the context of the
>> stack trace or application activation context makes the job of those
>> maintaining and debugging those applications easier.  However hard
>> we decide to make it, I will continue to provide these.
> 
> IMO, the real problem is old habits, period. Despite your repeating it
> over and over, I never expressed anything in the sense of having less
> information in the error messages. [I don't get what the stack trace has
> to do here. And I just gave you a real example where whatever details
> CM tries to provide, it will _never_ be sufficient because it cannot
> know why the call failed; I suggested that the _same_ amount of
> (necessary but not sufficient) information could perhaps be provided
> with "little block" patterns glued with "addMessage" (or an improvement
> thereof).]
> Specific exceptions always provide more information than less specific
> ones. Keeping low-level message (e.g. precondition failure) does not
> preclude adding more specific messages when the context is known (that
> happens in the code, and every little variant does not need to be
> hard-coded in the currently overly long list of patterns).
> My proposals were solely aimed at making the "preparation" of the
> messages more efficient from a developer's perspective (e.g. no scanning
> of 300+ patterns).
> Stalling the experiment in endless arguments makes it less and less
> worth trying.
> 
> All in all, the main argument seems to always be that if the user
> cannot see the difference, it is not worth changing the design.

Which is also a pragmatic and valid approach here imho.

If there are no real user complaints about this topic (and I am not
aware of any) and no other solution will greatly enhance the current
state, it is really not worth doing it.

Part of my day job is to debug very complex systems and the most
important thing is that you get what you expect, i.e. according to the
contract of a method, which btw also includes the method name. Detailed
error messages are nice to have but not really required (as long as you
understand the purpose of the code which any user/developer of CM should
do).

More meaningful error messages would make sense if our targeted audience
are really end-users but I think this is a bit far-fetched.

My conclusion: the way we handle exceptions is not perfect but totally
sufficient imho.

Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sat, 09 Nov 2013 13:13:05 -0800, Phil Steitz wrote:
> On 11/5/13 5:21 AM, Gilles wrote:
>>>>> [...]
>>>>> I have scanned for exact duplicates quite a few times and never
>>>>> found any.  There are quite a few that are similar, but differ in
>>>>> material ways (strict versus non-strict inequalities, endpoints
>>>>> included / not included, etc.).  Please do not "collapse" 
>>>>> messages
>>>>> at the expense of loss of specificity or correctness.
>>>>
>>>> FAILED_BRACKETING
>>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>>> INVALID_BRACKETING_PARAMETERS
>>>
>>> Look at the messages.  These are different.  They convey different
>>> information and are appropriate in different contexts.  See below.
>>
>> I've argued that context information should be constructed at the
>> point where the exception is thrown (where the context is known).
>> Not all combinations of exceptions and context need be present in
>> the pattern list.
>> This is the essence of my proposal below.
>>
>>>>
>>>> My position: the error (failed bracketing) should have its own
>>>> exception
>>>> type. The varying contexts could (do not have to) be part of the
>>>> message
>>>> built at exception instantiation.
>>>>
>>>> If we want to include an indication of location (despite it is
>>>> already
>>>> part of the stack trace, so it is _redundant_), we could perhaps
>>>> add methods
>>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>>> (?).
>>>> Then, we would have thos patterns in the list:
>>>>
>>>> BRACKETING
>>>> LINE_SEARCH
>>>>
>>>> Note: INVALID and FAILED are redundant since the pattern is
>>>> intended to be
>>>> included in an exception.
>>>>
>>>>
>>>> A second "interesting" case is
>>>>
>>>> INVALID_ROUNDING_METHOD
>>>>
>>>> which mixes documentation with error description. Does anyone
>>>> really thinks
>>>> that the enumeration of the rounding methods in the error message
>>>> is necessary
>>>> or even helpful?
>>>
>>> When I throw an exception, I want to provide an error message that
>>> is meaningful in the context of the caller, i.e., that someone
>>> looking at a log or stack trace can make sense of.  That sometimes
>>> means restating preconditions, sometimes pointing to boundary
>>> conditions, sometimes giving hints describing common causes of the
>>> exception - lots of different things that depend on the API, the
>>> activation context and the nature of the exception.  The natural 
>>> way
>>> to do this is to use natural language sentences.  Please allow me 
>>> to
>>> retain a straightforward way to construct these messages and to
>>> maintain the specificity and meaning of the messages.
>>
>> IMHO, the level of details in the message is not needed: if the
>> exception
>> was thrown, the user should probably look at the documentation,
>> rather
>> than try another value at random; I'd say that it is harmful to
>> tempt the
>> users with something like "Pick another number". ;-)
>>
>> [Shouldn't we rather provide function where the rounding type is
>> an enum?]
>>
>> The main problem in those discussions is that you consider only 
>> "toy"
>> situations, where the message generated by Commons Math should
>> make sense
>> wherever the exception is caught, and even if it is not caught.
>
> What you keep failing to acknowledge is that in many real world
> applications, reading exception stack traces and application logs
> that contain error messages is an important operational activity.
> Having clear error messages that make sense in the context of the
> stack trace or application activation context makes the job of those
> maintaining and debugging those applications easier.  However hard
> we decide to make it, I will continue to provide these.

IMO, the real problem is old habits, period. Despite your repeating it
over and over, I never expressed anything in the sense of having less
information in the error messages. [I don't get what the stack trace 
has
to do here. And I just gave you a real example where whatever details
CM tries to provide, it will _never_ be sufficient because it cannot
know why the call failed; I suggested that the _same_ amount of
(necessary but not sufficient) information could perhaps be provided
with "little block" patterns glued with "addMessage" (or an improvement
thereof).]
Specific exceptions always provide more information than less specific
ones. Keeping low-level message (e.g. precondition failure) does not
preclude adding more specific messages when the context is known (that
happens in the code, and every little variant does not need to be
hard-coded in the currently overly long list of patterns).
My proposals were solely aimed at making the "preparation" of the
messages more efficient from a developer's perspective (e.g. no 
scanning
of 300+ patterns).
Stalling the experiment in endless arguments makes it less and less
worth trying.

All in all, the main argument seems to always be that if the user
cannot see the difference, it is not worth changing the design.


Gilles


>
> Phil
>
>
>> [I sometimes get a "failed bracketing" but knowing the values of 
>> "the
>> endpoints [that] have the same sign" does not really help. I'd
>> rather need
>> to catch the exception, add more context info, rethrow, recatch, 
>> etc.
>> And all this is quite more expensive than activating logging for
>> those
>> rare cases where numerical problems in the simulation trigger the
>> exception.]
>> Again and again, I do not mean that CM should not generate error
>> messages,
>> only that context info beyond a plain description of what happened 
>> is
>> rarely usable a few layers above the failed call. And that context
>> info
>> could be provided with much less than 300+ different messages.
>>
>> Having little "building blocks" would also make it easier to 
>> retrieve
>> pattern/value pairs, as Luc seems willing to do, and more stable,
>> since
>> a single placeholder is unlikely to change meaning, while a
>> sentence that
>> contains many, could be turned differently so that the previous
>> placeholder
>> index could now refer to a different value.
>>
>>
>> Gilles
>>
>>
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/5/13 5:21 AM, Gilles wrote:
>>>> [...]
>>>> I have scanned for exact duplicates quite a few times and never
>>>> found any.  There are quite a few that are similar, but differ in
>>>> material ways (strict versus non-strict inequalities, endpoints
>>>> included / not included, etc.).  Please do not "collapse" messages
>>>> at the expense of loss of specificity or correctness.
>>>
>>> FAILED_BRACKETING
>>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>>> INVALID_BRACKETING_PARAMETERS
>>
>> Look at the messages.  These are different.  They convey different
>> information and are appropriate in different contexts.  See below.
>
> I've argued that context information should be constructed at the
> point where the exception is thrown (where the context is known).
> Not all combinations of exceptions and context need be present in
> the pattern list.
> This is the essence of my proposal below.
>
>>>
>>> My position: the error (failed bracketing) should have its own
>>> exception
>>> type. The varying contexts could (do not have to) be part of the
>>> message
>>> built at exception instantiation.
>>>
>>> If we want to include an indication of location (despite it is
>>> already
>>> part of the stack trace, so it is _redundant_), we could perhaps
>>> add methods
>>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)"
>>> (?).
>>> Then, we would have thos patterns in the list:
>>>
>>> BRACKETING
>>> LINE_SEARCH
>>>
>>> Note: INVALID and FAILED are redundant since the pattern is
>>> intended to be
>>> included in an exception.
>>>
>>>
>>> A second "interesting" case is
>>>
>>> INVALID_ROUNDING_METHOD
>>>
>>> which mixes documentation with error description. Does anyone
>>> really thinks
>>> that the enumeration of the rounding methods in the error message
>>> is necessary
>>> or even helpful?
>>
>> When I throw an exception, I want to provide an error message that
>> is meaningful in the context of the caller, i.e., that someone
>> looking at a log or stack trace can make sense of.  That sometimes
>> means restating preconditions, sometimes pointing to boundary
>> conditions, sometimes giving hints describing common causes of the
>> exception - lots of different things that depend on the API, the
>> activation context and the nature of the exception.  The natural way
>> to do this is to use natural language sentences.  Please allow me to
>> retain a straightforward way to construct these messages and to
>> maintain the specificity and meaning of the messages.
>
> IMHO, the level of details in the message is not needed: if the
> exception
> was thrown, the user should probably look at the documentation,
> rather
> than try another value at random; I'd say that it is harmful to
> tempt the
> users with something like "Pick another number". ;-)
>
> [Shouldn't we rather provide function where the rounding type is
> an enum?]
>
> The main problem in those discussions is that you consider only "toy"
> situations, where the message generated by Commons Math should
> make sense
> wherever the exception is caught, and even if it is not caught.

What you keep failing to acknowledge is that in many real world
applications, reading exception stack traces and application logs
that contain error messages is an important operational activity. 
Having clear error messages that make sense in the context of the
stack trace or application activation context makes the job of those
maintaining and debugging those applications easier.  However hard
we decide to make it, I will continue to provide these.

Phil


> [I sometimes get a "failed bracketing" but knowing the values of "the
> endpoints [that] have the same sign" does not really help. I'd
> rather need
> to catch the exception, add more context info, rethrow, recatch, etc.
> And all this is quite more expensive than activating logging for
> those
> rare cases where numerical problems in the simulation trigger the
> exception.]
> Again and again, I do not mean that CM should not generate error
> messages,
> only that context info beyond a plain description of what happened is
> rarely usable a few layers above the failed call. And that context
> info
> could be provided with much less than 300+ different messages.
>
> Having little "building blocks" would also make it easier to retrieve
> pattern/value pairs, as Luc seems willing to do, and more stable,
> since
> a single placeholder is unlikely to change meaning, while a
> sentence that
> contains many, could be turned differently so that the previous
> placeholder
> index could now refer to a different value.
>
>
> Gilles
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

>>> [...]
>>> I have scanned for exact duplicates quite a few times and never
>>> found any.  There are quite a few that are similar, but differ in
>>> material ways (strict versus non-strict inequalities, endpoints
>>> included / not included, etc.).  Please do not "collapse" messages
>>> at the expense of loss of specificity or correctness.
>>
>> FAILED_BRACKETING
>> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
>> INVALID_BRACKETING_PARAMETERS
>
> Look at the messages.  These are different.  They convey different
> information and are appropriate in different contexts.  See below.

I've argued that context information should be constructed at the
point where the exception is thrown (where the context is known).
Not all combinations of exceptions and context need be present in
the pattern list.
This is the essence of my proposal below.

>>
>> My position: the error (failed bracketing) should have its own
>> exception
>> type. The varying contexts could (do not have to) be part of the
>> message
>> built at exception instantiation.
>>
>> If we want to include an indication of location (despite it is
>> already
>> part of the stack trace, so it is _redundant_), we could perhaps
>> add methods
>> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)" 
>> (?).
>> Then, we would have thos patterns in the list:
>>
>> BRACKETING
>> LINE_SEARCH
>>
>> Note: INVALID and FAILED are redundant since the pattern is
>> intended to be
>> included in an exception.
>>
>>
>> A second "interesting" case is
>>
>> INVALID_ROUNDING_METHOD
>>
>> which mixes documentation with error description. Does anyone
>> really thinks
>> that the enumeration of the rounding methods in the error message
>> is necessary
>> or even helpful?
>
> When I throw an exception, I want to provide an error message that
> is meaningful in the context of the caller, i.e., that someone
> looking at a log or stack trace can make sense of.  That sometimes
> means restating preconditions, sometimes pointing to boundary
> conditions, sometimes giving hints describing common causes of the
> exception - lots of different things that depend on the API, the
> activation context and the nature of the exception.  The natural way
> to do this is to use natural language sentences.  Please allow me to
> retain a straightforward way to construct these messages and to
> maintain the specificity and meaning of the messages.

IMHO, the level of details in the message is not needed: if the 
exception
was thrown, the user should probably look at the documentation, rather
than try another value at random; I'd say that it is harmful to tempt 
the
users with something like "Pick another number". ;-)

[Shouldn't we rather provide function where the rounding type is an 
enum?]

The main problem in those discussions is that you consider only "toy"
situations, where the message generated by Commons Math should make 
sense
wherever the exception is caught, and even if it is not caught.
[I sometimes get a "failed bracketing" but knowing the values of "the
endpoints [that] have the same sign" does not really help. I'd rather 
need
to catch the exception, add more context info, rethrow, recatch, etc.
And all this is quite more expensive than activating logging for those
rare cases where numerical problems in the simulation trigger the
exception.]
Again and again, I do not mean that CM should not generate error 
messages,
only that context info beyond a plain description of what happened is
rarely usable a few layers above the failed call. And that context info
could be provided with much less than 300+ different messages.

Having little "building blocks" would also make it easier to retrieve
pattern/value pairs, as Luc seems willing to do, and more stable, since
a single placeholder is unlikely to change meaning, while a sentence 
that
contains many, could be turned differently so that the previous 
placeholder
index could now refer to a different value.


Gilles


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/4/13 10:11 AM, Gilles wrote:
> On Mon, 04 Nov 2013 09:47:32 -0800, Phil Steitz wrote:
>> On 11/4/13 2:08 AM, Luc Maisonobe wrote:
>>> Le 04/11/2013 00:59, Gilles a écrit :
>>>> On Sun, 03 Nov 2013 15:33:12 -0800, Phil Steitz wrote:
>>>>> On 11/3/13 2:57 PM, Gilles wrote:
>>>>>> On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
>>>>>>> Le 03/11/2013 20:17, Ted Dunning a écrit :
>>>>>>>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe
>>>>>>>> <lu...@spaceroots.org> wrote:
>>>>>>>>
>>>>>>>>>> I had proposed that error messages be incrementally built
>>>>>>>>>> from
>>>>>>>>>> simple
>>>>>>>>>> "base" patterns, to be assembled either at the point
>>>>>>>>>> where the
>>>>>>>>>> exception
>>>>>>>>>> is going to be thrown or inside specific exceptions[2] (or a
>>>>>>>>>> combination
>>>>>>>>>> of both).
>>>>>>>>> It often doesn't work. Sentences constructions are completely
>>>>>>>>> different
>>>>>>>>> in different languages, and it is impossible to simply
>>>>>>>>> buid up
>>>>>>>>> from
>>>>>>>>> elementary components that are individually translated and
>>>>>>>>> assembled
>>>>>>>>> later. See all the documentation about the ancient gettext
>>>>>>>>> for
>>>>>>>>> example.
>>>>>> I think that a message good enough to convey the cause of the
>>>>>> failure can
>>>>>> in many cases be built from such blocks. I concede that it
>>>>>> may not be
>>>>>> perfectly formulated in a natural language, but even if it
>>>>>> where,
>>>>>> the error
>>>>>> message are _rarely_ if ever clear,
>>>>> I disagree with that statement.  I think we have in general
>>>>> done a
>>>>> pretty good job producing clear, understandable exception error
>>>>> messages, which are very useful to users of a library.
>>>> My statement is not that the error message is not a well written
>>>> English (or French) sentence. It is that being in a correct
>>>> natural
>>>> language does not help in figuring out what the caller code did to
>>>> trigger the error.
>>>> The low level cause can be conveyed with the "little blocks" too.
>>>>
>>>> It seems we go in circles; every time I have to assure that I do
>>>> not, and never did, propose to suppress error messages!
>>>>
>>>> What I suggested is to try and see whether the "ExceptionContext"
>>>> can be used more.
>>> Exceptioncontext has been add more than two years ago (revision
>>> 1099771,
>>> 2011-05-05). Since then its setValue method is called in only
>>> two places
>>> in regular code (not counting test code): in SymmLQ.java and in
>>> ConjugateGradient, so this key/value features seems a failure to
>>> me. Its
>>> message part on the other hand is used everywhere, and I even
>>> use it
>>> outside of Commons Math since I introduced the
>>> ExceptionContextProvider
>>> interface in September 2001 (we had a short discussion about it
>>> there:
>>> <http://markmail.org/message/q7gv5mqx62w735pc>).
>>>
>>> So I agree ExceptionContext is a useful class, and perhaps it
>>> would need
>>> some changes. As per the comments above, I would suggest to further
>>> improve the used part, typically allowing to retrieve the
>>> patterns and
>>> arguments which I really miss, as the patterns are already set
>>> everywhere and can be used externally to also identify a particular
>>> exception. I have no real idea about improving the currently almost
>>> unused key/value feature, but fear it would imply a tremendous
>>> work.
>>>
>>>
>>>>> For the code
>>>>> I work on, I will continue to do that, whatever contortions are
>>>>> required to create them.  I personally don't mind working with
>>>>> the
>>>>> message pattern setup we have.  The one improvement I would
>>>>> appreciate / incrementally help with is an attempt at organizing
>>>>> them a little better.
>>>> That was the purpose of my suggestions, which came up to
>>>> implementing
>>>> the "ExceptionContext".
>>>> Again, this may not be the answer to all wishes in matter of
>>>> conveying
>>>> error information, but how will we know if we don't even try?
>>>>
>>>> It is quite possible that many duplicates ended up in the error
>>>> message list just because it was easy to miss the one that
>>>> should have
>>>> been reused or that the existing ones where not quite
>>>> appropriate to
>>>> the exception being thrown.
>>> This may be true, but finding it is also a lot of work, with little
>>> benefit. Feel free to do it if you want, though.
>>
>> I have scanned for exact duplicates quite a few times and never
>> found any.  There are quite a few that are similar, but differ in
>> material ways (strict versus non-strict inequalities, endpoints
>> included / not included, etc.).  Please do not "collapse" messages
>> at the expense of loss of specificity or correctness.
>
> FAILED_BRACKETING
> UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
> INVALID_BRACKETING_PARAMETERS

Look at the messages.  These are different.  They convey different
information and are appropriate in different contexts.  See below.
>
> My position: the error (failed bracketing) should have its own
> exception
> type. The varying contexts could (do not have to) be part of the
> message
> built at exception instantiation.
>
> If we want to include an indication of location (despite it is
> already
> part of the stack trace, so it is _redundant_), we could perhaps
> add methods
> to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)" (?).
> Then, we would have thos patterns in the list:
>
> BRACKETING
> LINE_SEARCH
>
> Note: INVALID and FAILED are redundant since the pattern is
> intended to be
> included in an exception.
>
>
> A second "interesting" case is
>
> INVALID_ROUNDING_METHOD
>
> which mixes documentation with error description. Does anyone
> really thinks
> that the enumeration of the rounding methods in the error message
> is necessary
> or even helpful?

When I throw an exception, I want to provide an error message that
is meaningful in the context of the caller, i.e., that someone
looking at a log or stack trace can make sense of.  That sometimes
means restating preconditions, sometimes pointing to boundary
conditions, sometimes giving hints describing common causes of the
exception - lots of different things that depend on the API, the
activation context and the nature of the exception.  The natural way
to do this is to use natural language sentences.  Please allow me to
retain a straightforward way to construct these messages and to
maintain the specificity and meaning of the messages.

Phil
>
>
> Gilles
>
>> [...]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Mon, 04 Nov 2013 09:47:32 -0800, Phil Steitz wrote:
> On 11/4/13 2:08 AM, Luc Maisonobe wrote:
>> Le 04/11/2013 00:59, Gilles a écrit :
>>> On Sun, 03 Nov 2013 15:33:12 -0800, Phil Steitz wrote:
>>>> On 11/3/13 2:57 PM, Gilles wrote:
>>>>> On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
>>>>>> Le 03/11/2013 20:17, Ted Dunning a écrit :
>>>>>>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe
>>>>>>> <lu...@spaceroots.org> wrote:
>>>>>>>
>>>>>>>>> I had proposed that error messages be incrementally built 
>>>>>>>>> from
>>>>>>>>> simple
>>>>>>>>> "base" patterns, to be assembled either at the point where 
>>>>>>>>> the
>>>>>>>>> exception
>>>>>>>>> is going to be thrown or inside specific exceptions[2] (or a
>>>>>>>>> combination
>>>>>>>>> of both).
>>>>>>>> It often doesn't work. Sentences constructions are completely
>>>>>>>> different
>>>>>>>> in different languages, and it is impossible to simply buid up
>>>>>>>> from
>>>>>>>> elementary components that are individually translated and
>>>>>>>> assembled
>>>>>>>> later. See all the documentation about the ancient gettext for
>>>>>>>> example.
>>>>> I think that a message good enough to convey the cause of the
>>>>> failure can
>>>>> in many cases be built from such blocks. I concede that it may 
>>>>> not be
>>>>> perfectly formulated in a natural language, but even if it where,
>>>>> the error
>>>>> message are _rarely_ if ever clear,
>>>> I disagree with that statement.  I think we have in general done a
>>>> pretty good job producing clear, understandable exception error
>>>> messages, which are very useful to users of a library.
>>> My statement is not that the error message is not a well written
>>> English (or French) sentence. It is that being in a correct natural
>>> language does not help in figuring out what the caller code did to
>>> trigger the error.
>>> The low level cause can be conveyed with the "little blocks" too.
>>>
>>> It seems we go in circles; every time I have to assure that I do
>>> not, and never did, propose to suppress error messages!
>>>
>>> What I suggested is to try and see whether the "ExceptionContext"
>>> can be used more.
>> Exceptioncontext has been add more than two years ago (revision 
>> 1099771,
>> 2011-05-05). Since then its setValue method is called in only two 
>> places
>> in regular code (not counting test code): in SymmLQ.java and in
>> ConjugateGradient, so this key/value features seems a failure to me. 
>> Its
>> message part on the other hand is used everywhere, and I even use it
>> outside of Commons Math since I introduced the 
>> ExceptionContextProvider
>> interface in September 2001 (we had a short discussion about it 
>> there:
>> <http://markmail.org/message/q7gv5mqx62w735pc>).
>>
>> So I agree ExceptionContext is a useful class, and perhaps it would 
>> need
>> some changes. As per the comments above, I would suggest to further
>> improve the used part, typically allowing to retrieve the patterns 
>> and
>> arguments which I really miss, as the patterns are already set
>> everywhere and can be used externally to also identify a particular
>> exception. I have no real idea about improving the currently almost
>> unused key/value feature, but fear it would imply a tremendous work.
>>
>>
>>>> For the code
>>>> I work on, I will continue to do that, whatever contortions are
>>>> required to create them.  I personally don't mind working with the
>>>> message pattern setup we have.  The one improvement I would
>>>> appreciate / incrementally help with is an attempt at organizing
>>>> them a little better.
>>> That was the purpose of my suggestions, which came up to 
>>> implementing
>>> the "ExceptionContext".
>>> Again, this may not be the answer to all wishes in matter of 
>>> conveying
>>> error information, but how will we know if we don't even try?
>>>
>>> It is quite possible that many duplicates ended up in the error
>>> message list just because it was easy to miss the one that should 
>>> have
>>> been reused or that the existing ones where not quite appropriate 
>>> to
>>> the exception being thrown.
>> This may be true, but finding it is also a lot of work, with little
>> benefit. Feel free to do it if you want, though.
>
> I have scanned for exact duplicates quite a few times and never
> found any.  There are quite a few that are similar, but differ in
> material ways (strict versus non-strict inequalities, endpoints
> included / not included, etc.).  Please do not "collapse" messages
> at the expense of loss of specificity or correctness.

FAILED_BRACKETING
UNABLE_TO_BRACKET_OPTIMUM_IN_LINE_SEARCH
INVALID_BRACKETING_PARAMETERS

My position: the error (failed bracketing) should have its own 
exception
type. The varying contexts could (do not have to) be part of the 
message
built at exception instantiation.

If we want to include an indication of location (despite it is already
part of the stack trace, so it is _redundant_), we could perhaps add 
methods
to the "ExceptionContext", e.g. "where(LocalizeFormats pattern)" (?).
Then, we would have thos patterns in the list:

BRACKETING
LINE_SEARCH

Note: INVALID and FAILED are redundant since the pattern is intended to 
be
included in an exception.


A second "interesting" case is

INVALID_ROUNDING_METHOD

which mixes documentation with error description. Does anyone really 
thinks
that the enumeration of the rounding methods in the error message is 
necessary
or even helpful?


Gilles

> [...]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/4/13 2:08 AM, Luc Maisonobe wrote:
> Le 04/11/2013 00:59, Gilles a écrit :
>> On Sun, 03 Nov 2013 15:33:12 -0800, Phil Steitz wrote:
>>> On 11/3/13 2:57 PM, Gilles wrote:
>>>> On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
>>>>> Le 03/11/2013 20:17, Ted Dunning a écrit :
>>>>>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe
>>>>>> <lu...@spaceroots.org> wrote:
>>>>>>
>>>>>>>> I had proposed that error messages be incrementally built from
>>>>>>>> simple
>>>>>>>> "base" patterns, to be assembled either at the point where the
>>>>>>>> exception
>>>>>>>> is going to be thrown or inside specific exceptions[2] (or a
>>>>>>>> combination
>>>>>>>> of both).
>>>>>>> It often doesn't work. Sentences constructions are completely
>>>>>>> different
>>>>>>> in different languages, and it is impossible to simply buid up
>>>>>>> from
>>>>>>> elementary components that are individually translated and
>>>>>>> assembled
>>>>>>> later. See all the documentation about the ancient gettext for
>>>>>>> example.
>>>> I think that a message good enough to convey the cause of the
>>>> failure can
>>>> in many cases be built from such blocks. I concede that it may not be
>>>> perfectly formulated in a natural language, but even if it where,
>>>> the error
>>>> message are _rarely_ if ever clear,
>>> I disagree with that statement.  I think we have in general done a
>>> pretty good job producing clear, understandable exception error
>>> messages, which are very useful to users of a library.
>> My statement is not that the error message is not a well written
>> English (or French) sentence. It is that being in a correct natural
>> language does not help in figuring out what the caller code did to
>> trigger the error.
>> The low level cause can be conveyed with the "little blocks" too.
>>
>> It seems we go in circles; every time I have to assure that I do
>> not, and never did, propose to suppress error messages!
>>
>> What I suggested is to try and see whether the "ExceptionContext"
>> can be used more.
> Exceptioncontext has been add more than two years ago (revision 1099771,
> 2011-05-05). Since then its setValue method is called in only two places
> in regular code (not counting test code): in SymmLQ.java and in
> ConjugateGradient, so this key/value features seems a failure to me. Its
> message part on the other hand is used everywhere, and I even use it
> outside of Commons Math since I introduced the ExceptionContextProvider
> interface in September 2001 (we had a short discussion about it there:
> <http://markmail.org/message/q7gv5mqx62w735pc>).
>
> So I agree ExceptionContext is a useful class, and perhaps it would need
> some changes. As per the comments above, I would suggest to further
> improve the used part, typically allowing to retrieve the patterns and
> arguments which I really miss, as the patterns are already set
> everywhere and can be used externally to also identify a particular
> exception. I have no real idea about improving the currently almost
> unused key/value feature, but fear it would imply a tremendous work.
>
>
>>> For the code
>>> I work on, I will continue to do that, whatever contortions are
>>> required to create them.  I personally don't mind working with the
>>> message pattern setup we have.  The one improvement I would
>>> appreciate / incrementally help with is an attempt at organizing
>>> them a little better.
>> That was the purpose of my suggestions, which came up to implementing
>> the "ExceptionContext".
>> Again, this may not be the answer to all wishes in matter of conveying
>> error information, but how will we know if we don't even try?
>>
>> It is quite possible that many duplicates ended up in the error
>> message list just because it was easy to miss the one that should have
>> been reused or that the existing ones where not quite appropriate to
>> the exception being thrown.
> This may be true, but finding it is also a lot of work, with little
> benefit. Feel free to do it if you want, though.

I have scanned for exact duplicates quite a few times and never
found any.  There are quite a few that are similar, but differ in
material ways (strict versus non-strict inequalities, endpoints
included / not included, etc.).  Please do not "collapse" messages
at the expense of loss of specificity or correctness.

Phil
>
> best regards,
> Luc
>
>> Gilles
>>
>>> Phil
>>>> an one needs to refer to the apidocs
>>>> in order to understand what caused the exception. There the error
>>>> condition
>>>> is hopefully (as it should) explained in a nice and clear sentences.
>>>>
>>>> There are drawbacks in having these hundreds of messages patterns.
>>>>
>>>>>>
>>>>>> Modern printf implementations deal with this by numbered
>>>>>> arguments.  This
>>>>>> is not a problem any more.
>>>>> Which means you have a complete message with a sentence that
>>>>> simply has
>>>>> placeholders for variables parts. What I understand about Gilles
>>>>> proposal is to go further in the way of small blocks like
>>>>> COLUMN_INDEX,
>>>>> CONSTRAINT, EVALUATION, INDEX, NORM, and build up from there. Is
>>>>> it what
>>>>> was initially meant?
>>>> Not sure I understand what you wrote. But, yes; where a
>>>> juxtaposition of
>>>> the blocks is good enough, we could readily use it. That will
>>>> reduce the list
>>>> quite substantially. When we get rid of all the similar patterns,
>>>> it will be
>>>> more acceptable to keep a few longer patters where really necessary.
>>>>
>>>>
>>>> Regards,
>>>> Gilles
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Mon, 04 Nov 2013 20:48:34 +0100, Luc Maisonobe wrote:
> Le 04/11/2013 19:49, Gilles a écrit :
>>>>>> [...]
>>>>
>>>> What I suggested is to try and see whether the "ExceptionContext"
>>>> can be used more.
>>>
>>> Exceptioncontext has been add more than two years ago (revision 
>>> 1099771,
>>> 2011-05-05). Since then its setValue method is called in only two 
>>> places
>>> in regular code (not counting test code): in SymmLQ.java and in
>>> ConjugateGradient, so this key/value features seems a failure to 
>>> me. Its
>>> message part on the other hand is used everywhere, and I even use 
>>> it
>>> outside of Commons Math since I introduced the 
>>> ExceptionContextProvider
>>> interface in September 2001 (we had a short discussion about it 
>>> there:
>>> <http://markmail.org/message/q7gv5mqx62w735pc>).
>>
>> There is a bit of confusion here; the ExceptionContextProider was 
>> part of
>> the initial batch of features:
>>   https://issues.apache.org/jira/browse/MATH-566
>> What you added a few months later (not in 2001 :-) was a method
>>   Exception getException()
>> to the "ExceptionContextProvider" interface.
>>
>> And by the same token as you used above, this could count as a 
>> "failure"
>> too since it is used only once in the code
>> ("o.a.c.m.ode.events.EventState").
>
> You are right. It is public and I use it outside [math].
>
>>
>> I remind that the "setValue" feature was primarily intended for 
>> _user_ code
>> (and suggested in a discussion, with the broader Commons community, 
>> as a
>> service to users). [I was pretty sure at the time already that it 
>> wouldn't
>> have much use inside CM.]
>
> Thanks for the reminding, I was not aware of it and thought it was
> intended so we supplement our exceptions when we throw them. I 
> personaly
> have no use case about adding up something to an exception that was
> already thrown, but understand someone else may have such cases. I 
> only
> have cases when I want to retrieve information from exception, as I
> consider them immutable _once they have been thrown_ (which is simply
> due to my own limitation).
>
>>
>> Moreover, the sole uses would probably be superseded by logging 
>> statements
>> if the possibility existed in CM.  This is really a pure 
>> annoyance[1] for a
>> library that purports to be used in scientific applications, where 
>> one
>> sometimes needs to recover how an algorithm arrived to a solution 
>> (e.g. in
>> order to assess whether the chosen algorithm is appropriate for the 
>> problem
>> at hand).
>>
>>>
>>> So I agree ExceptionContext is a useful class, and perhaps it would 
>>> need
>>> some changes. As per the comments above, I would suggest to further
>>> improve the used part, typically allowing to retrieve the patterns 
>>> and
>>> arguments which I really miss,
>>
>>> as the patterns are already set
>>> everywhere and can be used externally to also identify a particular
>>> exception.
>>
>> IMO, this is a very bad (TM) idea since it assumes that a given 
>> pattern
>> is here to stay forever. [Whereas it surely counts as "unstable" or
>> "implementation detail", not to be counted on more than the value
>> returned by the "toString" method.]
>
> I agree, but it is already present and needs almost no work (except
> adding one or two methods in the single ExceptionContext class).

Of course, it can be done. Easy is not the same as good.

I'd have no problem to add those, if it'll make you happy to play with
the patterns in application code; but it should be marked as "internal"
and "subject to change without notice" (see below).

>>
>> If you actually have use cases for catching a specific exception, 
>> this
>> is a positive argument for implementing specific exceptions!
>
> Yes from a theroetical point of view but we already know it does not
> work in practice as we cannot document properly what is thrown since 
> we
> have switched to unchecked exception.

This is just a false proposition!
All exceptions can be documented irrespective of being checked or 
unchecked.

> So the exact same argument holds
> for both cases: user has to dig into the code (not documentation),

This is also false, in principle, since all exceptions which a call can
throw must be documented. If it is sometimes true in practice, it 
counts
as a bug (of the documentation).

> and
> code changes can break this anytime without the user noticing it (we 
> can
> change the pattern, and we can change the exception type, user code 
> will
> still compile and run ... until it randomly breaks at runtime).

Breaking the contract is breaking the contract; compile- vs run-time 
has
nothing to do with it.

What I say is that the developers should be _allowed_ to change 
anything
that pertains to internal utilities without notice.
The localization utilities are internal, the exception types are not.

>
> So both methods have the same advantages, both methods have the same
> drawbacks, and one method implies lot of works.
>
>> It is beyond me that you can argue to use unspecific instances of
>> "MathIllegalArgumentException" and then add code that digs into the
>> internals of those.
>> Java enables one to catch specific exception types[2]; our job is to
>> provide those types (i.e. define more exception types, not less).
>> Everything points to the same direction, except perhaps old habits
>> (old, but by no means standard).
>
> It is beyond me that you can argue to change code that runs and may 
> fail
> sometimes by other code that will runs and may fail for the exact 
> same
> reasons, without adding any new security. Changing code only for the
> sake of changing code (and satisfying only theoretical peace of mind)
> mainly introduces new bugs.

No, that is not my argument: It is rather that users must not rely on
internals. And that we should not advertize internals, and that we 
should
explicitly warn users to not use internal code (or use it with the 
knowledge
that it can change, and that they will have to modify their code 
accordingly
at any new release).

>
> Hey, don't take the previous paragraph too seriously, I was only 
> kidding
> there! I agree accurate exception types are cleaner. What I say is
> simply they are *not* safer [...]

[See above.]

> and its a lot of work to change.

But it can be done incrementally.
It is a pity that we have to argue to no end since; you agree that is
cleaner, but it changes nothing. Each time a newcomer asks a question 
that
touches on some unfinished job, we restart the discussions at the exact
same point (and sometimes even backwards from what was agreed before).

Gilles

> > [...]


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Luc Maisonobe <lu...@spaceroots.org>]

Le 04/11/2013 19:49, Gilles a écrit :
>>>>> [...]
>>>
>>> What I suggested is to try and see whether the "ExceptionContext"
>>> can be used more.
>>
>> Exceptioncontext has been add more than two years ago (revision 1099771,
>> 2011-05-05). Since then its setValue method is called in only two places
>> in regular code (not counting test code): in SymmLQ.java and in
>> ConjugateGradient, so this key/value features seems a failure to me. Its
>> message part on the other hand is used everywhere, and I even use it
>> outside of Commons Math since I introduced the ExceptionContextProvider
>> interface in September 2001 (we had a short discussion about it there:
>> <http://markmail.org/message/q7gv5mqx62w735pc>).
> 
> There is a bit of confusion here; the ExceptionContextProider was part of
> the initial batch of features:
>   https://issues.apache.org/jira/browse/MATH-566
> What you added a few months later (not in 2001 :-) was a method
>   Exception getException()
> to the "ExceptionContextProvider" interface.
> 
> And by the same token as you used above, this could count as a "failure"
> too since it is used only once in the code
> ("o.a.c.m.ode.events.EventState").

You are right. It is public and I use it outside [math].

> 
> I remind that the "setValue" feature was primarily intended for _user_ code
> (and suggested in a discussion, with the broader Commons community, as a
> service to users). [I was pretty sure at the time already that it wouldn't
> have much use inside CM.]

Thanks for the reminding, I was not aware of it and thought it was
intended so we supplement our exceptions when we throw them. I personaly
have no use case about adding up something to an exception that was
already thrown, but understand someone else may have such cases. I only
have cases when I want to retrieve information from exception, as I
consider them immutable _once they have been thrown_ (which is simply
due to my own limitation).

> 
> Moreover, the sole uses would probably be superseded by logging statements
> if the possibility existed in CM.  This is really a pure annoyance[1] for a
> library that purports to be used in scientific applications, where one
> sometimes needs to recover how an algorithm arrived to a solution (e.g. in
> order to assess whether the chosen algorithm is appropriate for the problem
> at hand).
> 
>>
>> So I agree ExceptionContext is a useful class, and perhaps it would need
>> some changes. As per the comments above, I would suggest to further
>> improve the used part, typically allowing to retrieve the patterns and
>> arguments which I really miss,
> 
>> as the patterns are already set
>> everywhere and can be used externally to also identify a particular
>> exception.
> 
> IMO, this is a very bad (TM) idea since it assumes that a given pattern
> is here to stay forever. [Whereas it surely counts as "unstable" or
> "implementation detail", not to be counted on more than the value
> returned by the "toString" method.]

I agree, but it is already present and needs almost no work (except
adding one or two methods in the single ExceptionContext class).

> 
> If you actually have use cases for catching a specific exception, this
> is a positive argument for implementing specific exceptions!

Yes from a theroetical point of view but we already know it does not
work in practice as we cannot document properly what is thrown since we
have switched to unchecked exception. So the exact same argument holds
for both cases: user has to dig into the code (not documentation), and
code changes can break this anytime without the user noticing it (we can
change the pattern, and we can change the exception type, user code will
still compile and run ... until it randomly breaks at runtime).

So both methods have the same advantages, both methods have the same
drawbacks, and one method implies lot of works.

> It is beyond me that you can argue to use unspecific instances of
> "MathIllegalArgumentException" and then add code that digs into the
> internals of those.
> Java enables one to catch specific exception types[2]; our job is to
> provide those types (i.e. define more exception types, not less).
> Everything points to the same direction, except perhaps old habits
> (old, but by no means standard).

It is beyond me that you can argue to change code that runs and may fail
sometimes by other code that will runs and may fail for the exact same
reasons, without adding any new security. Changing code only for the
sake of changing code (and satisfying only theoretical peace of mind)
mainly introduces new bugs.

Hey, don't take the previous paragraph too seriously, I was only kidding
there! I agree accurate exception types are cleaner. What I say is
simply they are *not* safer and its a lot of work to change.

Luc

> 
>> I have no real idea about improving the currently almost
>> unused key/value feature, but fear it would imply a tremendous work
> 
> From an internal POV, my opinion is that it can be removed.
> I don't know what you mean by improving here. The feature is there, and
> can be used. [But we can decide that we don't want to maintain a tool
> which nobody sees how it can be useful inside CM, and which nobody uses
> outside CM.]
> 
>>>
>>>> For the code
>>>> I work on, I will continue to do that, whatever contortions are
>>>> required to create them.  I personally don't mind working with the
>>>> message pattern setup we have.  The one improvement I would
>>>> appreciate / incrementally help with is an attempt at organizing
>>>> them a little better.
>>>
>>> That was the purpose of my suggestions, which came up to implementing
>>> the "ExceptionContext".
>>> Again, this may not be the answer to all wishes in matter of conveying
>>> error information, but how will we know if we don't even try?
>>>
>>> It is quite possible that many duplicates ended up in the error
>>> message list just because it was easy to miss the one that should have
>>> been reused or that the existing ones where not quite appropriate to
>>> the exception being thrown.
>>
>> This may be true, but finding it is also a lot of work, with little
>> benefit. Feel free to do it if you want, though.
> 
> Yes, it's a repetitive work (a "grep" for each pattern and see whether the
> same information can be composed from the "little blocks").
> That's why I mentioned it among the consistency "tasks" inquired about by
> Sean.
> 
> 
> Best regards,
> Gilles
> 
> [1] Several times, I found myself obliged to use an ad-hoc version of CM
>     (containing tediously added "print" statements) in order to trace the
>     behaviour of the optimizers.
> [2] With added features in Java 7.
> 
> 
>> > > [...]
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

>>>> [...]
>>
>> What I suggested is to try and see whether the "ExceptionContext"
>> can be used more.
>
> Exceptioncontext has been add more than two years ago (revision 
> 1099771,
> 2011-05-05). Since then its setValue method is called in only two 
> places
> in regular code (not counting test code): in SymmLQ.java and in
> ConjugateGradient, so this key/value features seems a failure to me. 
> Its
> message part on the other hand is used everywhere, and I even use it
> outside of Commons Math since I introduced the 
> ExceptionContextProvider
> interface in September 2001 (we had a short discussion about it 
> there:
> <http://markmail.org/message/q7gv5mqx62w735pc>).

There is a bit of confusion here; the ExceptionContextProider was part 
of
the initial batch of features:
   https://issues.apache.org/jira/browse/MATH-566
What you added a few months later (not in 2001 :-) was a method
   Exception getException()
to the "ExceptionContextProvider" interface.

And by the same token as you used above, this could count as a 
"failure"
too since it is used only once in the code 
("o.a.c.m.ode.events.EventState").

I remind that the "setValue" feature was primarily intended for _user_ 
code
(and suggested in a discussion, with the broader Commons community, as 
a
service to users). [I was pretty sure at the time already that it 
wouldn't
have much use inside CM.]

Moreover, the sole uses would probably be superseded by logging 
statements
if the possibility existed in CM.  This is really a pure annoyance[1] 
for a
library that purports to be used in scientific applications, where one
sometimes needs to recover how an algorithm arrived to a solution (e.g. 
in
order to assess whether the chosen algorithm is appropriate for the 
problem
at hand).

>
> So I agree ExceptionContext is a useful class, and perhaps it would 
> need
> some changes. As per the comments above, I would suggest to further
> improve the used part, typically allowing to retrieve the patterns 
> and
> arguments which I really miss,

> as the patterns are already set
> everywhere and can be used externally to also identify a particular
> exception.

IMO, this is a very bad (TM) idea since it assumes that a given pattern
is here to stay forever. [Whereas it surely counts as "unstable" or
"implementation detail", not to be counted on more than the value
returned by the "toString" method.]

If you actually have use cases for catching a specific exception, this
is a positive argument for implementing specific exceptions!
It is beyond me that you can argue to use unspecific instances of
"MathIllegalArgumentException" and then add code that digs into the
internals of those.
Java enables one to catch specific exception types[2]; our job is to
provide those types (i.e. define more exception types, not less).
Everything points to the same direction, except perhaps old habits
(old, but by no means standard).

> I have no real idea about improving the currently almost
> unused key/value feature, but fear it would imply a tremendous work

 From an internal POV, my opinion is that it can be removed.
I don't know what you mean by improving here. The feature is there, and
can be used. [But we can decide that we don't want to maintain a tool
which nobody sees how it can be useful inside CM, and which nobody uses
outside CM.]

>>
>>> For the code
>>> I work on, I will continue to do that, whatever contortions are
>>> required to create them.  I personally don't mind working with the
>>> message pattern setup we have.  The one improvement I would
>>> appreciate / incrementally help with is an attempt at organizing
>>> them a little better.
>>
>> That was the purpose of my suggestions, which came up to 
>> implementing
>> the "ExceptionContext".
>> Again, this may not be the answer to all wishes in matter of 
>> conveying
>> error information, but how will we know if we don't even try?
>>
>> It is quite possible that many duplicates ended up in the error
>> message list just because it was easy to miss the one that should 
>> have
>> been reused or that the existing ones where not quite appropriate to
>> the exception being thrown.
>
> This may be true, but finding it is also a lot of work, with little
> benefit. Feel free to do it if you want, though.

Yes, it's a repetitive work (a "grep" for each pattern and see whether 
the
same information can be composed from the "little blocks").
That's why I mentioned it among the consistency "tasks" inquired about 
by
Sean.


Best regards,
Gilles

[1] Several times, I found myself obliged to use an ad-hoc version of 
CM
     (containing tediously added "print" statements) in order to trace 
the
     behaviour of the optimizers.
[2] With added features in Java 7.


> > > [...]


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Luc Maisonobe <lu...@spaceroots.org>]

Le 04/11/2013 00:59, Gilles a écrit :
> On Sun, 03 Nov 2013 15:33:12 -0800, Phil Steitz wrote:
>> On 11/3/13 2:57 PM, Gilles wrote:
>>> On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
>>>> Le 03/11/2013 20:17, Ted Dunning a écrit :
>>>>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe
>>>>> <lu...@spaceroots.org> wrote:
>>>>>
>>>>>>> I had proposed that error messages be incrementally built from
>>>>>>> simple
>>>>>>> "base" patterns, to be assembled either at the point where the
>>>>>>> exception
>>>>>>> is going to be thrown or inside specific exceptions[2] (or a
>>>>>>> combination
>>>>>>> of both).
>>>>>>
>>>>>> It often doesn't work. Sentences constructions are completely
>>>>>> different
>>>>>> in different languages, and it is impossible to simply buid up
>>>>>> from
>>>>>> elementary components that are individually translated and
>>>>>> assembled
>>>>>> later. See all the documentation about the ancient gettext for
>>>>>> example.
>>>
>>> I think that a message good enough to convey the cause of the
>>> failure can
>>> in many cases be built from such blocks. I concede that it may not be
>>> perfectly formulated in a natural language, but even if it where,
>>> the error
>>> message are _rarely_ if ever clear,
>>
>> I disagree with that statement.  I think we have in general done a
>> pretty good job producing clear, understandable exception error
>> messages, which are very useful to users of a library.
> 
> My statement is not that the error message is not a well written
> English (or French) sentence. It is that being in a correct natural
> language does not help in figuring out what the caller code did to
> trigger the error.
> The low level cause can be conveyed with the "little blocks" too.
> 
> It seems we go in circles; every time I have to assure that I do
> not, and never did, propose to suppress error messages!
> 
> What I suggested is to try and see whether the "ExceptionContext"
> can be used more.

Exceptioncontext has been add more than two years ago (revision 1099771,
2011-05-05). Since then its setValue method is called in only two places
in regular code (not counting test code): in SymmLQ.java and in
ConjugateGradient, so this key/value features seems a failure to me. Its
message part on the other hand is used everywhere, and I even use it
outside of Commons Math since I introduced the ExceptionContextProvider
interface in September 2001 (we had a short discussion about it there:
<http://markmail.org/message/q7gv5mqx62w735pc>).

So I agree ExceptionContext is a useful class, and perhaps it would need
some changes. As per the comments above, I would suggest to further
improve the used part, typically allowing to retrieve the patterns and
arguments which I really miss, as the patterns are already set
everywhere and can be used externally to also identify a particular
exception. I have no real idea about improving the currently almost
unused key/value feature, but fear it would imply a tremendous work.


> 
>> For the code
>> I work on, I will continue to do that, whatever contortions are
>> required to create them.  I personally don't mind working with the
>> message pattern setup we have.  The one improvement I would
>> appreciate / incrementally help with is an attempt at organizing
>> them a little better.
> 
> That was the purpose of my suggestions, which came up to implementing
> the "ExceptionContext".
> Again, this may not be the answer to all wishes in matter of conveying
> error information, but how will we know if we don't even try?
> 
> It is quite possible that many duplicates ended up in the error
> message list just because it was easy to miss the one that should have
> been reused or that the existing ones where not quite appropriate to
> the exception being thrown.

This may be true, but finding it is also a lot of work, with little
benefit. Feel free to do it if you want, though.

best regards,
Luc

> 
> Gilles
> 
>>
>> Phil
>>> an one needs to refer to the apidocs
>>> in order to understand what caused the exception. There the error
>>> condition
>>> is hopefully (as it should) explained in a nice and clear sentences.
>>>
>>> There are drawbacks in having these hundreds of messages patterns.
>>>
>>>>>
>>>>>
>>>>> Modern printf implementations deal with this by numbered
>>>>> arguments.  This
>>>>> is not a problem any more.
>>>>
>>>> Which means you have a complete message with a sentence that
>>>> simply has
>>>> placeholders for variables parts. What I understand about Gilles
>>>> proposal is to go further in the way of small blocks like
>>>> COLUMN_INDEX,
>>>> CONSTRAINT, EVALUATION, INDEX, NORM, and build up from there. Is
>>>> it what
>>>> was initially meant?
>>>
>>> Not sure I understand what you wrote. But, yes; where a
>>> juxtaposition of
>>> the blocks is good enough, we could readily use it. That will
>>> reduce the list
>>> quite substantially. When we get rid of all the similar patterns,
>>> it will be
>>> more acceptable to keep a few longer patters where really necessary.
>>>
>>>
>>> Regards,
>>> Gilles
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 03 Nov 2013 15:33:12 -0800, Phil Steitz wrote:
> On 11/3/13 2:57 PM, Gilles wrote:
>> On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
>>> Le 03/11/2013 20:17, Ted Dunning a écrit :
>>>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe
>>>> <lu...@spaceroots.org> wrote:
>>>>
>>>>>> I had proposed that error messages be incrementally built from
>>>>>> simple
>>>>>> "base" patterns, to be assembled either at the point where the
>>>>>> exception
>>>>>> is going to be thrown or inside specific exceptions[2] (or a
>>>>>> combination
>>>>>> of both).
>>>>>
>>>>> It often doesn't work. Sentences constructions are completely
>>>>> different
>>>>> in different languages, and it is impossible to simply buid up
>>>>> from
>>>>> elementary components that are individually translated and
>>>>> assembled
>>>>> later. See all the documentation about the ancient gettext for
>>>>> example.
>>
>> I think that a message good enough to convey the cause of the
>> failure can
>> in many cases be built from such blocks. I concede that it may not 
>> be
>> perfectly formulated in a natural language, but even if it where,
>> the error
>> message are _rarely_ if ever clear,
>
> I disagree with that statement.  I think we have in general done a
> pretty good job producing clear, understandable exception error
> messages, which are very useful to users of a library.

My statement is not that the error message is not a well written
English (or French) sentence. It is that being in a correct natural
language does not help in figuring out what the caller code did to
trigger the error.
The low level cause can be conveyed with the "little blocks" too.

It seems we go in circles; every time I have to assure that I do
not, and never did, propose to suppress error messages!

What I suggested is to try and see whether the "ExceptionContext"
can be used more.

> For the code
> I work on, I will continue to do that, whatever contortions are
> required to create them.  I personally don't mind working with the
> message pattern setup we have.  The one improvement I would
> appreciate / incrementally help with is an attempt at organizing
> them a little better.

That was the purpose of my suggestions, which came up to implementing
the "ExceptionContext".
Again, this may not be the answer to all wishes in matter of conveying
error information, but how will we know if we don't even try?

It is quite possible that many duplicates ended up in the error
message list just because it was easy to miss the one that should have
been reused or that the existing ones where not quite appropriate to
the exception being thrown.

Gilles

>
> Phil
>> an one needs to refer to the apidocs
>> in order to understand what caused the exception. There the error
>> condition
>> is hopefully (as it should) explained in a nice and clear sentences.
>>
>> There are drawbacks in having these hundreds of messages patterns.
>>
>>>>
>>>>
>>>> Modern printf implementations deal with this by numbered
>>>> arguments.  This
>>>> is not a problem any more.
>>>
>>> Which means you have a complete message with a sentence that
>>> simply has
>>> placeholders for variables parts. What I understand about Gilles
>>> proposal is to go further in the way of small blocks like
>>> COLUMN_INDEX,
>>> CONSTRAINT, EVALUATION, INDEX, NORM, and build up from there. Is
>>> it what
>>> was initially meant?
>>
>> Not sure I understand what you wrote. But, yes; where a
>> juxtaposition of
>> the blocks is good enough, we could readily use it. That will
>> reduce the list
>> quite substantially. When we get rid of all the similar patterns,
>> it will be
>> more acceptable to keep a few longer patters where really necessary.
>>
>>
>> Regards,
>> Gilles
>>
>>
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/3/13 2:57 PM, Gilles wrote:
> On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
>> Le 03/11/2013 20:17, Ted Dunning a écrit :
>>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe
>>> <lu...@spaceroots.org> wrote:
>>>
>>>>> I had proposed that error messages be incrementally built from
>>>>> simple
>>>>> "base" patterns, to be assembled either at the point where the
>>>>> exception
>>>>> is going to be thrown or inside specific exceptions[2] (or a
>>>>> combination
>>>>> of both).
>>>>
>>>> It often doesn't work. Sentences constructions are completely
>>>> different
>>>> in different languages, and it is impossible to simply buid up
>>>> from
>>>> elementary components that are individually translated and
>>>> assembled
>>>> later. See all the documentation about the ancient gettext for
>>>> example.
>
> I think that a message good enough to convey the cause of the
> failure can
> in many cases be built from such blocks. I concede that it may not be
> perfectly formulated in a natural language, but even if it where,
> the error
> message are _rarely_ if ever clear,

I disagree with that statement.  I think we have in general done a
pretty good job producing clear, understandable exception error
messages, which are very useful to users of a library.  For the code
I work on, I will continue to do that, whatever contortions are
required to create them.  I personally don't mind working with the
message pattern setup we have.  The one improvement I would
appreciate / incrementally help with is an attempt at organizing
them a little better.

Phil
> an one needs to refer to the apidocs
> in order to understand what caused the exception. There the error
> condition
> is hopefully (as it should) explained in a nice and clear sentences.
>
> There are drawbacks in having these hundreds of messages patterns.
>
>>>
>>>
>>> Modern printf implementations deal with this by numbered
>>> arguments.  This
>>> is not a problem any more.
>>
>> Which means you have a complete message with a sentence that
>> simply has
>> placeholders for variables parts. What I understand about Gilles
>> proposal is to go further in the way of small blocks like
>> COLUMN_INDEX,
>> CONSTRAINT, EVALUATION, INDEX, NORM, and build up from there. Is
>> it what
>> was initially meant?
>
> Not sure I understand what you wrote. But, yes; where a
> juxtaposition of
> the blocks is good enough, we could readily use it. That will
> reduce the list
> quite substantially. When we get rid of all the similar patterns,
> it will be
> more acceptable to keep a few longer patters where really necessary.
>
>
> Regards,
> Gilles
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sun, 03 Nov 2013 21:03:02 +0100, Luc Maisonobe wrote:
> Le 03/11/2013 20:17, Ted Dunning a écrit :
>> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe <lu...@spaceroots.org> 
>> wrote:
>>
>>>> I had proposed that error messages be incrementally built from 
>>>> simple
>>>> "base" patterns, to be assembled either at the point where the 
>>>> exception
>>>> is going to be thrown or inside specific exceptions[2] (or a 
>>>> combination
>>>> of both).
>>>
>>> It often doesn't work. Sentences constructions are completely 
>>> different
>>> in different languages, and it is impossible to simply buid up from
>>> elementary components that are individually translated and 
>>> assembled
>>> later. See all the documentation about the ancient gettext for 
>>> example.

I think that a message good enough to convey the cause of the failure 
can
in many cases be built from such blocks. I concede that it may not be
perfectly formulated in a natural language, but even if it where, the 
error
message are _rarely_ if ever clear, an one needs to refer to the 
apidocs
in order to understand what caused the exception. There the error 
condition
is hopefully (as it should) explained in a nice and clear sentences.

There are drawbacks in having these hundreds of messages patterns.

>>
>>
>> Modern printf implementations deal with this by numbered arguments.  
>> This
>> is not a problem any more.
>
> Which means you have a complete message with a sentence that simply 
> has
> placeholders for variables parts. What I understand about Gilles
> proposal is to go further in the way of small blocks like 
> COLUMN_INDEX,
> CONSTRAINT, EVALUATION, INDEX, NORM, and build up from there. Is it 
> what
> was initially meant?

Not sure I understand what you wrote. But, yes; where a juxtaposition 
of
the blocks is good enough, we could readily use it. That will reduce 
the list
quite substantially. When we get rid of all the similar patterns, it 
will be
more acceptable to keep a few longer patters where really necessary.


Regards,
Gilles


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Luc Maisonobe <lu...@spaceroots.org>]

Le 03/11/2013 20:17, Ted Dunning a écrit :
> On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe <lu...@spaceroots.org> wrote:
> 
>>> I had proposed that error messages be incrementally built from simple
>>> "base" patterns, to be assembled either at the point where the exception
>>> is going to be thrown or inside specific exceptions[2] (or a combination
>>> of both).
>>
>> It often doesn't work. Sentences constructions are completely different
>> in different languages, and it is impossible to simply buid up from
>> elementary components that are individually translated and assembled
>> later. See all the documentation about the ancient gettext for example.
> 
> 
> Modern printf implementations deal with this by numbered arguments.  This
> is not a problem any more.

Which means you have a complete message with a sentence that simply has
placeholders for variables parts. What I understand about Gilles
proposal is to go further in the way of small blocks like COLUMN_INDEX,
CONSTRAINT, EVALUATION, INDEX, NORM, and build up from there. Is it what
was initially meant?

Luc

> 
> See http://docs.oracle.com/javase/7/docs/api/java/util/Formatter.html#syntax
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Ted Dunning <te...@gmail.com>]

On Sun, Nov 3, 2013 at 10:56 AM, Luc Maisonobe <lu...@spaceroots.org> wrote:

> > I had proposed that error messages be incrementally built from simple
> > "base" patterns, to be assembled either at the point where the exception
> > is going to be thrown or inside specific exceptions[2] (or a combination
> > of both).
>
> It often doesn't work. Sentences constructions are completely different
> in different languages, and it is impossible to simply buid up from
> elementary components that are individually translated and assembled
> later. See all the documentation about the ancient gettext for example.


Modern printf implementations deal with this by numbered arguments.  This
is not a problem any more.

See http://docs.oracle.com/javase/7/docs/api/java/util/Formatter.html#syntax

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Luc Maisonobe <lu...@spaceroots.org>]

Hi Gilles,

Le 03/11/2013 14:33, Gilles a écrit :
> On Sat, 2 Nov 2013 17:35:17 +0000, Sean Owen wrote:
>> On Sat, Nov 2, 2013 at 4:49 PM, Phil Steitz <ph...@gmail.com>
>> wrote:
>>> If the improvements make a material difference [1], by all means
>>> open tickets and submit patches.  If they are just cosmetic, my
>>> personal opinion is this should be done in small commits
>>> incrementally if at all [2].  It is OK to open master tickets to
>>> track style / optimization changes and do the work to review and
>>> commit them incrementally.  The master ticket should be preceded by
>>> discussion here to make sure we are all in agreement that the
>>> changes are appropriate.
>>>
>>> You mention quite a few different kinds of thing above.  It is
>>> probably best to break up into different discussion threads by
>>> topical area and then open master tickets for the ones we agree to
>>> fix / standardize.  Regarding the javadoc errors (broken javadoc
>>> references, missing / incorrect @deprecates), there is no need for
>>> discussion - just open tickets and we can commit the fixes.
>>
>> I agree with that. I'm going to file two JIRAs and probably leave it
>> for now. No point in getting into minutiae.
>>
>> Let me turn it around: are there general clean-up tasks that people
>> have wanted to do for a while but not gotten to? Maybe I could take a
>> request or two and run with them.
>>
> 
> One issue[1] is the unnecessarily high number of error patterns (defined
> the "org.apache.commons.math3.exception.util.LocalizedFormats" class).
> If one agrees that is is not necessary that every "throw" statement
> must have its own pattern, it is possible to reduce the enormous
> redundancy that exists in that list of patterns.
> 
> I had proposed that error messages be incrementally built from simple
> "base" patterns, to be assembled either at the point where the exception
> is going to be thrown or inside specific exceptions[2] (or a combination
> of both).

It often doesn't work. Sentences constructions are completely different
in different languages, and it is impossible to simply buid up from
elementary components that are individually translated and assembled
later. See all the documentation about the ancient gettext for example.

best regards,
Luc

> Thus, the error message will be explicitly tied to a specific code
> context, where we know all the information about the error and can add
> it to the message that will be carried up the calling stack.
> 
> 
> Best regards,
> Gilles
> 
> [1] Whether it is trivial or minutiae or a possible improvement, is again
>     a matter of taste.
> [2] If a similar error condition occurs at different places in the code.
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
> 
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sat, 2 Nov 2013 17:35:17 +0000, Sean Owen wrote:
> On Sat, Nov 2, 2013 at 4:49 PM, Phil Steitz <ph...@gmail.com> 
> wrote:
>> If the improvements make a material difference [1], by all means
>> open tickets and submit patches.  If they are just cosmetic, my
>> personal opinion is this should be done in small commits
>> incrementally if at all [2].  It is OK to open master tickets to
>> track style / optimization changes and do the work to review and
>> commit them incrementally.  The master ticket should be preceded by
>> discussion here to make sure we are all in agreement that the
>> changes are appropriate.
>>
>> You mention quite a few different kinds of thing above.  It is
>> probably best to break up into different discussion threads by
>> topical area and then open master tickets for the ones we agree to
>> fix / standardize.  Regarding the javadoc errors (broken javadoc
>> references, missing / incorrect @deprecates), there is no need for
>> discussion - just open tickets and we can commit the fixes.
>
> I agree with that. I'm going to file two JIRAs and probably leave it
> for now. No point in getting into minutiae.
>
> Let me turn it around: are there general clean-up tasks that people
> have wanted to do for a while but not gotten to? Maybe I could take a
> request or two and run with them.
>

One issue[1] is the unnecessarily high number of error patterns 
(defined
the "org.apache.commons.math3.exception.util.LocalizedFormats" class).
If one agrees that is is not necessary that every "throw" statement
must have its own pattern, it is possible to reduce the enormous
redundancy that exists in that list of patterns.

I had proposed that error messages be incrementally built from simple
"base" patterns, to be assembled either at the point where the 
exception
is going to be thrown or inside specific exceptions[2] (or a 
combination
of both).
Thus, the error message will be explicitly tied to a specific code
context, where we know all the information about the error and can add
it to the message that will be carried up the calling stack.


Best regards,
Gilles

[1] Whether it is trivial or minutiae or a possible improvement, is 
again
     a matter of taste.
[2] If a similar error condition occurs at different places in the 
code.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Sean Owen <sr...@gmail.com>]

On Sat, Nov 2, 2013 at 4:49 PM, Phil Steitz <ph...@gmail.com> wrote:
> If the improvements make a material difference [1], by all means
> open tickets and submit patches.  If they are just cosmetic, my
> personal opinion is this should be done in small commits
> incrementally if at all [2].  It is OK to open master tickets to
> track style / optimization changes and do the work to review and
> commit them incrementally.  The master ticket should be preceded by
> discussion here to make sure we are all in agreement that the
> changes are appropriate.
>
> You mention quite a few different kinds of thing above.  It is
> probably best to break up into different discussion threads by
> topical area and then open master tickets for the ones we agree to
> fix / standardize.  Regarding the javadoc errors (broken javadoc
> references, missing / incorrect @deprecates), there is no need for
> discussion - just open tickets and we can commit the fixes.

I agree with that. I'm going to file two JIRAs and probably leave it
for now. No point in getting into minutiae.

Let me turn it around: are there general clean-up tasks that people
have wanted to do for a while but not gotten to? Maybe I could take a
request or two and run with them.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Phil Steitz <ph...@gmail.com>]

On 11/2/13 8:19 AM, Sean Owen wrote:
> There are tens of possible types of changes here, each affecting tens
> or even 100+ files. I wondered if it would be irritating to open,
> potentially, tens of JIRAs.
>
> Maybe I can start with a JIRA or two, each addressing one or a few
> closely-related changes. If it's going well, I can propose more until
> it's clear we're getting too fine-grained.
>
> I'll do this over time since some changes will affect the same files
> or sections of code.

If the improvements make a material difference [1], by all means
open tickets and submit patches.  If they are just cosmetic, my
personal opinion is this should be done in small commits
incrementally if at all [2].  It is OK to open master tickets to
track style / optimization changes and do the work to review and
commit them incrementally.  The master ticket should be preceded by
discussion here to make sure we are all in agreement that the
changes are appropriate.

You mention quite a few different kinds of thing above.  It is
probably best to break up into different discussion threads by
topical area and then open master tickets for the ones we agree to
fix / standardize.  Regarding the javadoc errors (broken javadoc
references, missing / incorrect @deprecates), there is no need for
discussion - just open tickets and we can commit the fixes.

Phil

[1] I know what "material difference" means is inherently
subjective.  For me it means either material performance improvement
or significantly improved readability / approachability.  I have
consistently argued against too many "style rules" because they
create barriers to entry for those trying to comply with them. 
Think about all of the examples you mention above in terms of new
contributors, some of whom may be top notch applied mathematicians
coming from a primarily C or other-language background.  Why burden
them (or us - which will include you soon enough - hehe) with the
tedious work to make their code comply with every Java coding best
practice we can imagine - *unless it makes a real difference to
performance or readability*?

[2] See "Value laziness as a virtue" and "Small, reversible steps"
starting about slide 167 in
http://www.betaversion.org/~stefano/papers/ac2006.2.pdf
>
> On Sat, Nov 2, 2013 at 3:03 PM, Gilles <gi...@harfang.homelinux.org> wrote:
>> On Sat, 2 Nov 2013 14:52:34 +0000, Sean Owen wrote:
>>> In Math, is there any appetite for large patches containing many
>>> instances of particular micro-optimizations? Examples:
>>>
>>> - Replace:
>>>     a[i][j] = a[i][j] + foo;
>>>   with:
>>>     a[i][j] += foo;
>>>   … which is faster/leaner in the byte code by a little bit. It might
>>> make a difference in many nested, tight loops.
>>> - Inefficient toArray() calls with 0-length arg
>>> - Using Map.entrySet() instead of keySet() + get()s
>>> - Unnecessarily non-static private methods/classes
>>> - StringBuffer vs StringBuilder
>>> - etc.
>>>
>>> There are some non-functional but still possibly useful, simplifying
>>> changes too:
>>>
>>> - Add @Deprecated annotations
>>> - Fix broken method refs in javadoc
>>> - Removing unnecessary boxing/unboxing
>>> - Using foreach where possible
>>> - etc.
>>>
>>> I could go on for a while. Most of what I might otherwise suggest is
>>> style-level stuff like fixing C-style array declarations which is
>>> unlikely to be useful enough to justify. Or, it would involve changing
>>> signatures somewhere, which is probably not cool.
>>>
>>> If there’s interest in these sorts of things I can generate one or
>>> more patches easily. The downside is the disruption of large patches,
>>> perhaps breaking existing patches.
>>
>> I think that is good to have a uniform style.
>> But why would a big patch be unavoidable? Reviewing a few changes is easier.
>>
>> Thanks for your help with this,
>> Gilles
>>
>>
>> [1] If just for better readability. But not the majority shares this view
>>     around here.
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Sean Owen <sr...@gmail.com>]

There are tens of possible types of changes here, each affecting tens
or even 100+ files. I wondered if it would be irritating to open,
potentially, tens of JIRAs.

Maybe I can start with a JIRA or two, each addressing one or a few
closely-related changes. If it's going well, I can propose more until
it's clear we're getting too fine-grained.

I'll do this over time since some changes will affect the same files
or sections of code.

On Sat, Nov 2, 2013 at 3:03 PM, Gilles <gi...@harfang.homelinux.org> wrote:
> On Sat, 2 Nov 2013 14:52:34 +0000, Sean Owen wrote:
>>
>> In Math, is there any appetite for large patches containing many
>> instances of particular micro-optimizations? Examples:
>>
>> - Replace:
>>     a[i][j] = a[i][j] + foo;
>>   with:
>>     a[i][j] += foo;
>>   … which is faster/leaner in the byte code by a little bit. It might
>> make a difference in many nested, tight loops.
>> - Inefficient toArray() calls with 0-length arg
>> - Using Map.entrySet() instead of keySet() + get()s
>> - Unnecessarily non-static private methods/classes
>> - StringBuffer vs StringBuilder
>> - etc.
>>
>> There are some non-functional but still possibly useful, simplifying
>> changes too:
>>
>> - Add @Deprecated annotations
>> - Fix broken method refs in javadoc
>> - Removing unnecessary boxing/unboxing
>> - Using foreach where possible
>> - etc.
>>
>> I could go on for a while. Most of what I might otherwise suggest is
>> style-level stuff like fixing C-style array declarations which is
>> unlikely to be useful enough to justify. Or, it would involve changing
>> signatures somewhere, which is probably not cool.
>>
>> If there’s interest in these sorts of things I can generate one or
>> more patches easily. The downside is the disruption of large patches,
>> perhaps breaking existing patches.
>
>
> I think that is good to have a uniform style.
> But why would a big patch be unavoidable? Reviewing a few changes is easier.
>
> Thanks for your help with this,
> Gilles
>
>
> [1] If just for better readability. But not the majority shares this view
>     around here.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[Gilles <gi...@harfang.homelinux.org>]

On Sat, 2 Nov 2013 14:52:34 +0000, Sean Owen wrote:
> In Math, is there any appetite for large patches containing many
> instances of particular micro-optimizations? Examples:
>
> - Replace:
>     a[i][j] = a[i][j] + foo;
>   with:
>     a[i][j] += foo;
>   … which is faster/leaner in the byte code by a little bit. It might
> make a difference in many nested, tight loops.
> - Inefficient toArray() calls with 0-length arg
> - Using Map.entrySet() instead of keySet() + get()s
> - Unnecessarily non-static private methods/classes
> - StringBuffer vs StringBuilder
> - etc.
>
> There are some non-functional but still possibly useful, simplifying
> changes too:
>
> - Add @Deprecated annotations
> - Fix broken method refs in javadoc
> - Removing unnecessary boxing/unboxing
> - Using foreach where possible
> - etc.
>
> I could go on for a while. Most of what I might otherwise suggest is
> style-level stuff like fixing C-style array declarations which is
> unlikely to be useful enough to justify. Or, it would involve 
> changing
> signatures somewhere, which is probably not cool.
>
> If there’s interest in these sorts of things I can generate one or
> more patches easily. The downside is the disruption of large patches,
> perhaps breaking existing patches.

I think that is good to have a uniform style.
But why would a big patch be unavoidable? Reviewing a few changes is 
easier.

Thanks for your help with this,
Gilles


[1] If just for better readability. But not the majority shares this 
view
     around here.



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [MATH] Interest in large patches for small cleanup / performance changes?

[sebb <se...@gmail.com>]

On 2 November 2013 14:52, Sean Owen <sr...@apache.org> wrote:
> In Math, is there any appetite for large patches containing many
> instances of particular micro-optimizations? Examples:
>
> - Replace:
>     a[i][j] = a[i][j] + foo;
>   with:
>     a[i][j] += foo;
>   … which is faster/leaner in the byte code by a little bit. It might
> make a difference in many nested, tight loops.

Regardless of any speedup, it's safer not to repeat the value.

a[i][j] = a[i][j] + foo;
looks a lot like
a[i][j] = a[i][i] + foo;
or
a[i][j] = a[j][i] + foo;
at first glance, whereas
a[i][j] += foo;
is unambiguous, and much clearer.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org