You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Gilles Sadowski <gi...@harfang.homelinux.org> on 2011/09/29 18:04:31 UTC
[Math] Issues 650, 675, 676
Hello.
Referring to:
https://issues.apache.org/jira/browse/MATH-650
https://issues.apache.org/jira/browse/MATH-675
https://issues.apache.org/jira/browse/MATH-676
In the absence of additional comments I'm proposing to resolve those issues.
Please let me know whether there are pending objections to my suggested
solutions.
Thanks,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 09/10/2011 13:56, sebb a écrit :
> On 30 September 2011 23:03, Gilles Sadowski
> <gi...@harfang.homelinux.org> wrote:
>>>>
>>>> Referring to:
>>>> https://issues.apache.org/jira/browse/MATH-650
>>>> https://issues.apache.org/jira/browse/MATH-675
>>>> https://issues.apache.org/jira/browse/MATH-676
>>>>
>>>> In the absence of additional comments I'm proposing to resolve those issues.
>>>> Please let me know whether there are pending objections to my suggested
>>>> solutions.
>>>
>>> Regarding MATH-650 (FastMath)
>>>
>>> If the proposal is to use resources instead of static arrays, then I
>>> disagree, because resources are significantly slower than static
>>> arrays.
>>>
>>> I am opposed to storing the arrays in package protected classes,
>>> because that exposes the data unnecessarily.
>>>
>>> However, if you really cannot agree to the including the static arrays
>>> within the FastMath class itself, then I suggest using separate
>>> classes with private arrays and package-protected getters.
>>
>> It has been shown that literal arrays are faster than resources in one
>> micro-benchmark instance: Namely, an application that consists of a single
>> function call. We still wait for a real use-case where the difference will
>> significantly matter.
>
> The original reason the JIRA was raised is probably solved by moving
> to on demand init.
> It's not clear if that alone is sufficient to deal with the OP's use case.
I think both way are fine for this use case. I met Alexis last week and
he was happy with the improvements.
Luc
>
>> Independently, I think that the right way to load this kind of data is
>> indeed via the "resources" functionality. If this is so, one should rather
>> ask whether the loading time can be improved. In fact, I wonder why loading
>> resources should be "so much" slower than arrays (18 ms vs 6 ms).
>
> I wonder too. If you can make it go as fast as the preset arrays, then
> I would be happy with that.
>
>> Hence, if there is still no consensus on that issue,
>
> I think you are the only person who appears to be vetoing the local arrays.
>
> I have yet to see a proper objection to leaving the arrays within the
> FastMath class.
> The code still generates the same results, and is proven faster in startup.
> What is wrong with that?
>
>> you are welcome to
>> create the helper classes as you suggest (which, IIRC, is what I had also
>> suggested from the outset).
>
> This is slightly different from the original suggestion; IIRC that did
> not include using getters.
>
>> That way, both alternatives will be on an equal footing for further testing.
>
> I don't follow that.
>
>>
>> Gilles
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Mon, Oct 10, 2011 at 03:26:01AM +0100, sebb wrote:
> On 9 October 2011 18:16, Gilles Sadowski <gi...@harfang.homelinux.org> wrote:
>
> >> This is slightly different from the original suggestion; IIRC that did
> >> not include using getters.
> >
> > I put getters because you asked them. But, indeed, there is no difference
> > with direct accessing the arrays since the getters do not return a copy: it
> > is possible to modify the "original" array.
>
> Those are *not* the getters I was referring to.
>
> I intended the getter to return an array element, not the array, so as
> to protect the array contents.
Don't you think that it will make the access much slower?
> As you point out, there's absolutely no point having the getter otherwise.
>
> The current implementation is not particularly secure against
> malicious or accidental tampering.
The only tampering could come from a bug in classes in the same package. OR
do I miss something?
> That is why I left the arrays in the same source file originally, as
> it allows the arrays to be automatically protected, without needing a
> getter.
Yes I understand the argument. Mine is that this data should not be stored
there.
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by sebb <se...@gmail.com>.
On 9 October 2011 18:16, Gilles Sadowski <gi...@harfang.homelinux.org> wrote:
>> This is slightly different from the original suggestion; IIRC that did
>> not include using getters.
>
> I put getters because you asked them. But, indeed, there is no difference
> with direct accessing the arrays since the getters do not return a copy: it
> is possible to modify the "original" array.
Those are *not* the getters I was referring to.
I intended the getter to return an array element, not the array, so as
to protect the array contents.
As you point out, there's absolutely no point having the getter otherwise.
The current implementation is not particularly secure against
malicious or accidental tampering.
That is why I left the arrays in the same source file originally, as
it allows the arrays to be automatically protected, without needing a
getter.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Mon, Oct 10, 2011 at 02:03:26AM +0100, sebb wrote:
> On 9 October 2011 22:44, Gilles Sadowski <gi...@harfang.homelinux.org> wrote:
> >> >>>>> Referring to:
> >> >>>>> https://issues.apache.org/jira/browse/MATH-650
> >> >>>>> https://issues.apache.org/jira/browse/MATH-675
> >> >>>>> https://issues.apache.org/jira/browse/MATH-676
> >> >>>>>
> >> >>>>> In the absence of additional comments I'm proposing to resolve those issues.
> >> >>>>> Please let me know whether there are pending objections to my suggested
> >> >>>>> solutions.
> >> >>>> Regarding MATH-650 (FastMath)
> >> >>>>
> >> >>>> If the proposal is to use resources instead of static arrays, then I
> >> >>>> disagree, because resources are significantly slower than static
> >> >>>> arrays.
> >> >>>>
> >> >>>> I am opposed to storing the arrays in package protected classes,
> >> >>>> because that exposes the data unnecessarily.
> >> >>>>
> >> >>>> However, if you really cannot agree to the including the static arrays
> >> >>>> within the FastMath class itself, then I suggest using separate
> >> >>>> classes with private arrays and package-protected getters.
> >> >>> It has been shown that literal arrays are faster than resources in one
> >> >>> micro-benchmark instance: Namely, an application that consists of a single
> >> >>> function call. We still wait for a real use-case where the difference will
> >> >>> significantly matter.
> >> >> The original reason the JIRA was raised is probably solved by moving
> >> >> to on demand init.
> >> > I've been saying that same thing more than a few times, in various ways.
> >> >
> >> >> It's not clear if that alone is sufficient to deal with the OP's use case.
> >> > At one point, Alexis wrote (on JIRA):
> >> > "A response time with the magnitude of 100ms is really perfect !"
> >> >
> >> > Then I provided a test case, following Luc's indication of the typical
> >> > functions called in the original use case.
> >> > On my machine, the timing for all alternatives was below 100 ms.
> >>
> >> 100ms on a fast machine doing nothing else is not awe-inspiring.
> >
> > I don't understand whether you agree with my position i.e. that all
> > alternatives having times below 100 ms can be considered sufficiently good.
>
> AIUI, the 100 ms was for loading time on the Android system, not a
> desktop or laptop.
The numbers last posted by Alexis where of the order of 100 ms. AIUI this
was already satisfactory to him, and with the current, there is an
additional slight improvement.
> > I recall that the sole worry clearly expressed by the original poster was
> > the psychological effect of waiting "too long" for the result. [For an
> > interactive application, waiting ~100 ms is not noticeable; it's far less
> > than the time we routinely wait between a click on an icon, and the opening
> > of the application window.]
> >
> >> Holding a static class lock for that long is in fact at *bad
> >> thing*.
> >
> > I don't understand. What do you mean? Which class, which lock? Can you
> > illustrate the items using the current issue?
> >
> >> People way smarter than me have come up with great ways to
> >> work around delays in app startup times. I prefer not to have to
> >> use those things, and, more importantly, not to force others using
> >> my code to use them.
> >
> > The point is that there are no delays anymore (solved by IOD).
>
> No, IOD only solves the delays for method calls that don't use the arrays.
But *that* was the original issue; from the JIRA description:
"[...] I've discovered that a simple FastMath.floor() takes about 4 to 5
secs [...]"
> >> I favor the static arrays solution, IIUC what the timing tests have
> >> shown. Specifically, if static arrays can reduce class loading time
> >> by more than 50ms, I think we should use them. If there is some
> >> other workaround that allows the resource loading and does not
> >> increase loading time more than 50-100ms, I am OK with the resources
> >> approach as well.
> >
> > [I guess that we are using the timings on my machine as a baseline.]
> > There is no reduction of more than 50 ms by using literal arrays: at most 12
> > ms, with "FastMath" bloated with the literal arrays.
> >
> > A strange thing is that moving the arrays to their own class increased their
> > loading time by ~6ms, while decreasing the resource loading time by about
> > the same amount.[1] I mean that it is as if, somehow, the literal arrays[2]
> > were incurring a penalty to other functionalities (the resource loading in
> > this case) called from within "FastMath"!
>
> I don't think you can conclude that without further evidence.
I state what I observed. I do not have an explanation for it but this is a
fact.
You are welcome to experiment, and I would be happy if you can extract the
root of this seeming strangeness.
>
> > In conclusion, with the current code in trunk, the difference between the
> > "literal arrays" and "resources" alternatives is less than ~0.5 millisecond.
>
> On your system.
>
> On an Android phone it will be a considerably bigger difference
Numbers, please.
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by sebb <se...@gmail.com>.
On 9 October 2011 22:44, Gilles Sadowski <gi...@harfang.homelinux.org> wrote:
>> >>>>> Referring to:
>> >>>>> https://issues.apache.org/jira/browse/MATH-650
>> >>>>> https://issues.apache.org/jira/browse/MATH-675
>> >>>>> https://issues.apache.org/jira/browse/MATH-676
>> >>>>>
>> >>>>> In the absence of additional comments I'm proposing to resolve those issues.
>> >>>>> Please let me know whether there are pending objections to my suggested
>> >>>>> solutions.
>> >>>> Regarding MATH-650 (FastMath)
>> >>>>
>> >>>> If the proposal is to use resources instead of static arrays, then I
>> >>>> disagree, because resources are significantly slower than static
>> >>>> arrays.
>> >>>>
>> >>>> I am opposed to storing the arrays in package protected classes,
>> >>>> because that exposes the data unnecessarily.
>> >>>>
>> >>>> However, if you really cannot agree to the including the static arrays
>> >>>> within the FastMath class itself, then I suggest using separate
>> >>>> classes with private arrays and package-protected getters.
>> >>> It has been shown that literal arrays are faster than resources in one
>> >>> micro-benchmark instance: Namely, an application that consists of a single
>> >>> function call. We still wait for a real use-case where the difference will
>> >>> significantly matter.
>> >> The original reason the JIRA was raised is probably solved by moving
>> >> to on demand init.
>> > I've been saying that same thing more than a few times, in various ways.
>> >
>> >> It's not clear if that alone is sufficient to deal with the OP's use case.
>> > At one point, Alexis wrote (on JIRA):
>> > "A response time with the magnitude of 100ms is really perfect !"
>> >
>> > Then I provided a test case, following Luc's indication of the typical
>> > functions called in the original use case.
>> > On my machine, the timing for all alternatives was below 100 ms.
>>
>> 100ms on a fast machine doing nothing else is not awe-inspiring.
>
> I don't understand whether you agree with my position i.e. that all
> alternatives having times below 100 ms can be considered sufficiently good.
AIUI, the 100 ms was for loading time on the Android system, not a
desktop or laptop.
> I recall that the sole worry clearly expressed by the original poster was
> the psychological effect of waiting "too long" for the result. [For an
> interactive application, waiting ~100 ms is not noticeable; it's far less
> than the time we routinely wait between a click on an icon, and the opening
> of the application window.]
>
>> Holding a static class lock for that long is in fact at *bad
>> thing*.
>
> I don't understand. What do you mean? Which class, which lock? Can you
> illustrate the items using the current issue?
>
>> People way smarter than me have come up with great ways to
>> work around delays in app startup times. I prefer not to have to
>> use those things, and, more importantly, not to force others using
>> my code to use them.
>
> The point is that there are no delays anymore (solved by IOD).
No, IOD only solves the delays for method calls that don't use the arrays.
>> I favor the static arrays solution, IIUC what the timing tests have
>> shown. Specifically, if static arrays can reduce class loading time
>> by more than 50ms, I think we should use them. If there is some
>> other workaround that allows the resource loading and does not
>> increase loading time more than 50-100ms, I am OK with the resources
>> approach as well.
>
> [I guess that we are using the timings on my machine as a baseline.]
> There is no reduction of more than 50 ms by using literal arrays: at most 12
> ms, with "FastMath" bloated with the literal arrays.
>
> A strange thing is that moving the arrays to their own class increased their
> loading time by ~6ms, while decreasing the resource loading time by about
> the same amount.[1] I mean that it is as if, somehow, the literal arrays[2]
> were incurring a penalty to other functionalities (the resource loading in
> this case) called from within "FastMath"!
I don't think you can conclude that without further evidence.
> In conclusion, with the current code in trunk, the difference between the
> "literal arrays" and "resources" alternatives is less than ~0.5 millisecond.
On your system.
On an Android phone it will be a considerably bigger difference
>
> Gilles
>
> [1] I also gained some 1-2 ms using "nio" and reducing the number of I/O
> method calls.
> [2] Even when they were not used, e.g. when "LOAD_RESOURCES" was set to
> "true" in order to use the resource loading alternative.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
> [...]. I can see we are getting nowhere in this discussion,
> so I am not going to comment further.
It's getting nowhere, because there is no evidence that there is an
identified problem. I don't get what you are aiming at with piling
hypothetical considerations of thread contention. Some posts ago, Ted
already pointed out that this was all dreams in the context of a web
application.
The original problem has been solved in a _sufficiently_ satisfactory way
with IOD. Period.
The rest of the dicussion is about a tiny amount of millisenconds, which
the original poster said he does not care about (reponse time below 100 ms
is OK).
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Phil Steitz <ph...@gmail.com>.
On 10/9/11 2:44 PM, Gilles Sadowski wrote:
>>>>>>> Referring to:
>>>>>>> https://issues.apache.org/jira/browse/MATH-650
>>>>>>> https://issues.apache.org/jira/browse/MATH-675
>>>>>>> https://issues.apache.org/jira/browse/MATH-676
>>>>>>>
>>>>>>> In the absence of additional comments I'm proposing to resolve those issues.
>>>>>>> Please let me know whether there are pending objections to my suggested
>>>>>>> solutions.
>>>>>> Regarding MATH-650 (FastMath)
>>>>>>
>>>>>> If the proposal is to use resources instead of static arrays, then I
>>>>>> disagree, because resources are significantly slower than static
>>>>>> arrays.
>>>>>>
>>>>>> I am opposed to storing the arrays in package protected classes,
>>>>>> because that exposes the data unnecessarily.
>>>>>>
>>>>>> However, if you really cannot agree to the including the static arrays
>>>>>> within the FastMath class itself, then I suggest using separate
>>>>>> classes with private arrays and package-protected getters.
>>>>> It has been shown that literal arrays are faster than resources in one
>>>>> micro-benchmark instance: Namely, an application that consists of a single
>>>>> function call. We still wait for a real use-case where the difference will
>>>>> significantly matter.
>>>> The original reason the JIRA was raised is probably solved by moving
>>>> to on demand init.
>>> I've been saying that same thing more than a few times, in various ways.
>>>
>>>> It's not clear if that alone is sufficient to deal with the OP's use case.
>>> At one point, Alexis wrote (on JIRA):
>>> "A response time with the magnitude of 100ms is really perfect !"
>>>
>>> Then I provided a test case, following Luc's indication of the typical
>>> functions called in the original use case.
>>> On my machine, the timing for all alternatives was below 100 ms.
>> 100ms on a fast machine doing nothing else is not awe-inspiring.
> I don't understand whether you agree with my position i.e. that all
> alternatives having times below 100 ms can be considered sufficiently good.
> I recall that the sole worry clearly expressed by the original poster was
> the psychological effect of waiting "too long" for the result. [For an
> interactive application, waiting ~100 ms is not noticeable; it's far less
> than the time we routinely wait between a click on an icon, and the opening
> of the application window.]
Even 50ms holding a class lock is bad if it can be avoided. In web
applications, or anything multi-threaded, you have to worry about
concurrency. Holding an exclusive lock on a resource that lots of
threads need for 100ms can cause threads to pile up with the end
result being way longer than 100ms for unlucky threads and, yes,
causing noticeable problems for the application. As has been
pointed out, there are workarounds to allow slow-starting web apps
not to get overwhelmed as they park threads waiting on locks; but
needlessly creating / contributing to this is not a good thing for a
library to do. I can see we are getting nowhere in this discussion,
so I am not going to comment further.
Phil
>
>> Holding a static class lock for that long is in fact at *bad
>> thing*.
> I don't understand. What do you mean? Which class, which lock? Can you
> illustrate the items using the current issue?
>
>> People way smarter than me have come up with great ways to
>> work around delays in app startup times. I prefer not to have to
>> use those things, and, more importantly, not to force others using
>> my code to use them.
> The point is that there are no delays anymore (solved by IOD).
>
>> I favor the static arrays solution, IIUC what the timing tests have
>> shown. Specifically, if static arrays can reduce class loading time
>> by more than 50ms, I think we should use them. If there is some
>> other workaround that allows the resource loading and does not
>> increase loading time more than 50-100ms, I am OK with the resources
>> approach as well.
> [I guess that we are using the timings on my machine as a baseline.]
> There is no reduction of more than 50 ms by using literal arrays: at most 12
> ms, with "FastMath" bloated with the literal arrays.
>
> A strange thing is that moving the arrays to their own class increased their
> loading time by ~6ms, while decreasing the resource loading time by about
> the same amount.[1] I mean that it is as if, somehow, the literal arrays[2]
> were incurring a penalty to other functionalities (the resource loading in
> this case) called from within "FastMath"!
>
> In conclusion, with the current code in trunk, the difference between the
> "literal arrays" and "resources" alternatives is less than ~0.5 millisecond.
>
>
> Gilles
>
> [1] I also gained some 1-2 ms using "nio" and reducing the number of I/O
> method calls.
> [2] Even when they were not used, e.g. when "LOAD_RESOURCES" was set to
> "true" in order to use the resource loading alternative.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
> >>>>> Referring to:
> >>>>> https://issues.apache.org/jira/browse/MATH-650
> >>>>> https://issues.apache.org/jira/browse/MATH-675
> >>>>> https://issues.apache.org/jira/browse/MATH-676
> >>>>>
> >>>>> In the absence of additional comments I'm proposing to resolve those issues.
> >>>>> Please let me know whether there are pending objections to my suggested
> >>>>> solutions.
> >>>> Regarding MATH-650 (FastMath)
> >>>>
> >>>> If the proposal is to use resources instead of static arrays, then I
> >>>> disagree, because resources are significantly slower than static
> >>>> arrays.
> >>>>
> >>>> I am opposed to storing the arrays in package protected classes,
> >>>> because that exposes the data unnecessarily.
> >>>>
> >>>> However, if you really cannot agree to the including the static arrays
> >>>> within the FastMath class itself, then I suggest using separate
> >>>> classes with private arrays and package-protected getters.
> >>> It has been shown that literal arrays are faster than resources in one
> >>> micro-benchmark instance: Namely, an application that consists of a single
> >>> function call. We still wait for a real use-case where the difference will
> >>> significantly matter.
> >> The original reason the JIRA was raised is probably solved by moving
> >> to on demand init.
> > I've been saying that same thing more than a few times, in various ways.
> >
> >> It's not clear if that alone is sufficient to deal with the OP's use case.
> > At one point, Alexis wrote (on JIRA):
> > "A response time with the magnitude of 100ms is really perfect !"
> >
> > Then I provided a test case, following Luc's indication of the typical
> > functions called in the original use case.
> > On my machine, the timing for all alternatives was below 100 ms.
>
> 100ms on a fast machine doing nothing else is not awe-inspiring.
I don't understand whether you agree with my position i.e. that all
alternatives having times below 100 ms can be considered sufficiently good.
I recall that the sole worry clearly expressed by the original poster was
the psychological effect of waiting "too long" for the result. [For an
interactive application, waiting ~100 ms is not noticeable; it's far less
than the time we routinely wait between a click on an icon, and the opening
of the application window.]
> Holding a static class lock for that long is in fact at *bad
> thing*.
I don't understand. What do you mean? Which class, which lock? Can you
illustrate the items using the current issue?
> People way smarter than me have come up with great ways to
> work around delays in app startup times. I prefer not to have to
> use those things, and, more importantly, not to force others using
> my code to use them.
The point is that there are no delays anymore (solved by IOD).
> I favor the static arrays solution, IIUC what the timing tests have
> shown. Specifically, if static arrays can reduce class loading time
> by more than 50ms, I think we should use them. If there is some
> other workaround that allows the resource loading and does not
> increase loading time more than 50-100ms, I am OK with the resources
> approach as well.
[I guess that we are using the timings on my machine as a baseline.]
There is no reduction of more than 50 ms by using literal arrays: at most 12
ms, with "FastMath" bloated with the literal arrays.
A strange thing is that moving the arrays to their own class increased their
loading time by ~6ms, while decreasing the resource loading time by about
the same amount.[1] I mean that it is as if, somehow, the literal arrays[2]
were incurring a penalty to other functionalities (the resource loading in
this case) called from within "FastMath"!
In conclusion, with the current code in trunk, the difference between the
"literal arrays" and "resources" alternatives is less than ~0.5 millisecond.
Gilles
[1] I also gained some 1-2 ms using "nio" and reducing the number of I/O
method calls.
[2] Even when they were not used, e.g. when "LOAD_RESOURCES" was set to
"true" in order to use the resource loading alternative.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Phil Steitz <ph...@gmail.com>.
On 10/9/11 10:16 AM, Gilles Sadowski wrote:
> On Sun, Oct 09, 2011 at 12:56:30PM +0100, sebb wrote:
>> On 30 September 2011 23:03, Gilles Sadowski
>> <gi...@harfang.homelinux.org> wrote:
>>>>> Referring to:
>>>>> https://issues.apache.org/jira/browse/MATH-650
>>>>> https://issues.apache.org/jira/browse/MATH-675
>>>>> https://issues.apache.org/jira/browse/MATH-676
>>>>>
>>>>> In the absence of additional comments I'm proposing to resolve those issues.
>>>>> Please let me know whether there are pending objections to my suggested
>>>>> solutions.
>>>> Regarding MATH-650 (FastMath)
>>>>
>>>> If the proposal is to use resources instead of static arrays, then I
>>>> disagree, because resources are significantly slower than static
>>>> arrays.
>>>>
>>>> I am opposed to storing the arrays in package protected classes,
>>>> because that exposes the data unnecessarily.
>>>>
>>>> However, if you really cannot agree to the including the static arrays
>>>> within the FastMath class itself, then I suggest using separate
>>>> classes with private arrays and package-protected getters.
>>> It has been shown that literal arrays are faster than resources in one
>>> micro-benchmark instance: Namely, an application that consists of a single
>>> function call. We still wait for a real use-case where the difference will
>>> significantly matter.
>> The original reason the JIRA was raised is probably solved by moving
>> to on demand init.
> I've been saying that same thing more than a few times, in various ways.
>
>> It's not clear if that alone is sufficient to deal with the OP's use case.
> At one point, Alexis wrote (on JIRA):
> "A response time with the magnitude of 100ms is really perfect !"
>
> Then I provided a test case, following Luc's indication of the typical
> functions called in the original use case.
> On my machine, the timing for all alternatives was below 100 ms.
100ms on a fast machine doing nothing else is not awe-inspiring.
Holding a static class lock for that long is in fact at *bad
thing*. People way smarter than me have come up with great ways to
work around delays in app startup times. I prefer not to have to
use those things, and, more importantly, not to force others using
my code to use them.
I favor the static arrays solution, IIUC what the timing tests have
shown. Specifically, if static arrays can reduce class loading time
by more than 50ms, I think we should use them. If there is some
other workaround that allows the resource loading and does not
increase loading time more than 50-100ms, I am OK with the resources
approach as well.
Phil
>
>>> Independently, I think that the right way to load this kind of data is
>>> indeed via the "resources" functionality. If this is so, one should rather
>>> ask whether the loading time can be improved. In fact, I wonder why loading
>>> resources should be "so much" slower than arrays (18 ms vs 6 ms).
>> I wonder too. If you can make it go as fast as the preset arrays, then
>> I would be happy with that.
> See my last comment on JIRA.
>
>>> Hence, if there is still no consensus on that issue,
>> I think you are the only person who appears to be vetoing the local arrays.
> Because, this is bad practice. In my opinion, the inconvenience is not worth
> it because, in the first place, over-optimizing is bad (TM): There was no
> _real_ use-case showing that initialization was still too slow after you
> implemented IOD. At that point, we should have removed the unnecessary
> over-optimization that was the literal arrays.
> If we want to optimize, we should lokk for the places where it matters.[1]
> I think of the matrix mulitplication issue, where the gain is additive,
> potentially saving seconds, minutes, hours of computation time, whereas this
> issue will at most gain a few dozen milliseconds.
>
>> I have yet to see a proper objection to leaving the arrays within the
>> FastMath class.
>> The code still generates the same results, and is proven faster in startup.
>> What is wrong with that?
> I think I've answered that in the above paragraphs.
> [Nothing wrong with the speed-up, if it didn't come at the expense of mixing
> code and data, increasing byte-code size, and other strange things that
> probably happen because a class file should not be arbitrarily large.]
>
>>> you are welcome to
>>> create the helper classes as you suggest (which, IIRC, is what I had also
>>> suggested from the outset).
>> This is slightly different from the original suggestion; IIRC that did
>> not include using getters.
> I put getters because you asked them. But, indeed, there is no difference
> with direct accessing the arrays since the getters do not return a copy: it
> is possible to modify the "original" array.
>
>>> That way, both alternatives will be on an equal footing for further testing.
>> I don't follow that.
> "resource" files are accessed from a helper class, and literal arrays are
> also now accessed from a helper class.
> When we want to suppress the arrays ;-), we just delete the class and 5
> lines in "FastMath" (instead of 6000+ lines).
>
> Regards,
> Gilles
>
> [1] That's what I had proposed when suggesting that we establish a set of
> benchmarks (real, not micro) using use-cases from the various users of
> CM.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Sun, Oct 09, 2011 at 12:56:30PM +0100, sebb wrote:
> On 30 September 2011 23:03, Gilles Sadowski
> <gi...@harfang.homelinux.org> wrote:
> >> >
> >> > Referring to:
> >> > https://issues.apache.org/jira/browse/MATH-650
> >> > https://issues.apache.org/jira/browse/MATH-675
> >> > https://issues.apache.org/jira/browse/MATH-676
> >> >
> >> > In the absence of additional comments I'm proposing to resolve those issues.
> >> > Please let me know whether there are pending objections to my suggested
> >> > solutions.
> >>
> >> Regarding MATH-650 (FastMath)
> >>
> >> If the proposal is to use resources instead of static arrays, then I
> >> disagree, because resources are significantly slower than static
> >> arrays.
> >>
> >> I am opposed to storing the arrays in package protected classes,
> >> because that exposes the data unnecessarily.
> >>
> >> However, if you really cannot agree to the including the static arrays
> >> within the FastMath class itself, then I suggest using separate
> >> classes with private arrays and package-protected getters.
> >
> > It has been shown that literal arrays are faster than resources in one
> > micro-benchmark instance: Namely, an application that consists of a single
> > function call. We still wait for a real use-case where the difference will
> > significantly matter.
>
> The original reason the JIRA was raised is probably solved by moving
> to on demand init.
I've been saying that same thing more than a few times, in various ways.
> It's not clear if that alone is sufficient to deal with the OP's use case.
At one point, Alexis wrote (on JIRA):
"A response time with the magnitude of 100ms is really perfect !"
Then I provided a test case, following Luc's indication of the typical
functions called in the original use case.
On my machine, the timing for all alternatives was below 100 ms.
>
> > Independently, I think that the right way to load this kind of data is
> > indeed via the "resources" functionality. If this is so, one should rather
> > ask whether the loading time can be improved. In fact, I wonder why loading
> > resources should be "so much" slower than arrays (18 ms vs 6 ms).
>
> I wonder too. If you can make it go as fast as the preset arrays, then
> I would be happy with that.
See my last comment on JIRA.
> > Hence, if there is still no consensus on that issue,
>
> I think you are the only person who appears to be vetoing the local arrays.
Because, this is bad practice. In my opinion, the inconvenience is not worth
it because, in the first place, over-optimizing is bad (TM): There was no
_real_ use-case showing that initialization was still too slow after you
implemented IOD. At that point, we should have removed the unnecessary
over-optimization that was the literal arrays.
If we want to optimize, we should lokk for the places where it matters.[1]
I think of the matrix mulitplication issue, where the gain is additive,
potentially saving seconds, minutes, hours of computation time, whereas this
issue will at most gain a few dozen milliseconds.
> I have yet to see a proper objection to leaving the arrays within the
> FastMath class.
> The code still generates the same results, and is proven faster in startup.
> What is wrong with that?
I think I've answered that in the above paragraphs.
[Nothing wrong with the speed-up, if it didn't come at the expense of mixing
code and data, increasing byte-code size, and other strange things that
probably happen because a class file should not be arbitrarily large.]
> > you are welcome to
> > create the helper classes as you suggest (which, IIRC, is what I had also
> > suggested from the outset).
>
> This is slightly different from the original suggestion; IIRC that did
> not include using getters.
I put getters because you asked them. But, indeed, there is no difference
with direct accessing the arrays since the getters do not return a copy: it
is possible to modify the "original" array.
> > That way, both alternatives will be on an equal footing for further testing.
>
> I don't follow that.
"resource" files are accessed from a helper class, and literal arrays are
also now accessed from a helper class.
When we want to suppress the arrays ;-), we just delete the class and 5
lines in "FastMath" (instead of 6000+ lines).
Regards,
Gilles
[1] That's what I had proposed when suggesting that we establish a set of
benchmarks (real, not micro) using use-cases from the various users of
CM.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by sebb <se...@gmail.com>.
On 30 September 2011 23:03, Gilles Sadowski
<gi...@harfang.homelinux.org> wrote:
>> >
>> > Referring to:
>> > https://issues.apache.org/jira/browse/MATH-650
>> > https://issues.apache.org/jira/browse/MATH-675
>> > https://issues.apache.org/jira/browse/MATH-676
>> >
>> > In the absence of additional comments I'm proposing to resolve those issues.
>> > Please let me know whether there are pending objections to my suggested
>> > solutions.
>>
>> Regarding MATH-650 (FastMath)
>>
>> If the proposal is to use resources instead of static arrays, then I
>> disagree, because resources are significantly slower than static
>> arrays.
>>
>> I am opposed to storing the arrays in package protected classes,
>> because that exposes the data unnecessarily.
>>
>> However, if you really cannot agree to the including the static arrays
>> within the FastMath class itself, then I suggest using separate
>> classes with private arrays and package-protected getters.
>
> It has been shown that literal arrays are faster than resources in one
> micro-benchmark instance: Namely, an application that consists of a single
> function call. We still wait for a real use-case where the difference will
> significantly matter.
The original reason the JIRA was raised is probably solved by moving
to on demand init.
It's not clear if that alone is sufficient to deal with the OP's use case.
> Independently, I think that the right way to load this kind of data is
> indeed via the "resources" functionality. If this is so, one should rather
> ask whether the loading time can be improved. In fact, I wonder why loading
> resources should be "so much" slower than arrays (18 ms vs 6 ms).
I wonder too. If you can make it go as fast as the preset arrays, then
I would be happy with that.
> Hence, if there is still no consensus on that issue,
I think you are the only person who appears to be vetoing the local arrays.
I have yet to see a proper objection to leaving the arrays within the
FastMath class.
The code still generates the same results, and is proven faster in startup.
What is wrong with that?
> you are welcome to
> create the helper classes as you suggest (which, IIRC, is what I had also
> suggested from the outset).
This is slightly different from the original suggestion; IIRC that did
not include using getters.
> That way, both alternatives will be on an equal footing for further testing.
I don't follow that.
>
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
> >
> > Referring to:
> > https://issues.apache.org/jira/browse/MATH-650
> > https://issues.apache.org/jira/browse/MATH-675
> > https://issues.apache.org/jira/browse/MATH-676
> >
> > In the absence of additional comments I'm proposing to resolve those issues.
> > Please let me know whether there are pending objections to my suggested
> > solutions.
>
> Regarding MATH-650 (FastMath)
>
> If the proposal is to use resources instead of static arrays, then I
> disagree, because resources are significantly slower than static
> arrays.
>
> I am opposed to storing the arrays in package protected classes,
> because that exposes the data unnecessarily.
>
> However, if you really cannot agree to the including the static arrays
> within the FastMath class itself, then I suggest using separate
> classes with private arrays and package-protected getters.
It has been shown that literal arrays are faster than resources in one
micro-benchmark instance: Namely, an application that consists of a single
function call. We still wait for a real use-case where the difference will
significantly matter.
I'm sorry to observe that in a serious project like this one, nobody is
willing to acknowledge that there should be a reasonably argued (and
constant) policy about changing code.
Indeed, I cannot agree on having changed "FastMath" in that way; at least,
the approach of storing the humongous arrays in other files/classes should
have been taken as a first step towards solving the reported problem.
Independently, I think that the right way to load this kind of data is
indeed via the "resources" functionality. If this is so, one should rather
ask whether the loading time can be improved. In fact, I wonder why loading
resources should be "so much" slower than arrays (18 ms vs 6 ms).
Hence, if there is still no consensus on that issue, you are welcome to
create the helper classes as you suggest (which, IIRC, is what I had also
suggested from the outset).
That way, both alternatives will be on an equal footing for further testing.
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by sebb <se...@gmail.com>.
On 29 September 2011 17:04, Gilles Sadowski
<gi...@harfang.homelinux.org> wrote:
> Hello.
>
> Referring to:
> https://issues.apache.org/jira/browse/MATH-650
> https://issues.apache.org/jira/browse/MATH-675
> https://issues.apache.org/jira/browse/MATH-676
>
> In the absence of additional comments I'm proposing to resolve those issues.
> Please let me know whether there are pending objections to my suggested
> solutions.
Regarding MATH-650 (FastMath)
If the proposal is to use resources instead of static arrays, then I
disagree, because resources are significantly slower than static
arrays.
I am opposed to storing the arrays in package protected classes,
because that exposes the data unnecessarily.
However, if you really cannot agree to the including the static arrays
within the FastMath class itself, then I suggest using separate
classes with private arrays and package-protected getters.
>
> Thanks,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Greg Sterijevski <gs...@gmail.com>.
Hold off on 675. I owe you a couple of changes, related to naming and some
other issues we discussed on the ML.
-Greg
On Thu, Sep 29, 2011 at 1:44 PM, Luc Maisonobe <Lu...@free.fr>wrote:
> Le 29/09/2011 18:04, Gilles Sadowski a écrit :
>
> Hello.
>>
>> Referring to:
>> https://issues.apache.org/**jira/browse/MATH-650<https://issues.apache.org/jira/browse/MATH-650>
>> https://issues.apache.org/**jira/browse/MATH-675<https://issues.apache.org/jira/browse/MATH-675>
>> https://issues.apache.org/**jira/browse/MATH-676<https://issues.apache.org/jira/browse/MATH-676>
>>
>> In the absence of additional comments I'm proposing to resolve those
>> issues.
>> Please let me know whether there are pending objections to my suggested
>> solutions.
>>
>
> Sorry, I was offline for a while, with scarce access to the net.
> What is the solution you propose for MATH-676, I only see the curves, but
> not how the faster one is achieved.
>
> I am OK with the two other issues.
>
> Luc
>
>
>
>>
>> Thanks,
>> Gilles
>>
>> ------------------------------**------------------------------**---------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.**apache.org<de...@commons.apache.org>
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ------------------------------**------------------------------**---------
> To unsubscribe, e-mail: dev-unsubscribe@commons.**apache.org<de...@commons.apache.org>
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [Math] Issues 650, 675, 676
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 30/09/2011 00:41, Gilles Sadowski a écrit :
> Hello Luc.
>
>>>
>>> Referring to:
>>> https://issues.apache.org/jira/browse/MATH-650
>>> https://issues.apache.org/jira/browse/MATH-675
>>> https://issues.apache.org/jira/browse/MATH-676
>>>
>>> In the absence of additional comments I'm proposing to resolve those issues.
>>> Please let me know whether there are pending objections to my suggested
>>> solutions.
>>
>> Sorry, I was offline for a while, with scarce access to the net.
>> What is the solution you propose for MATH-676, I only see the
>> curves, but not how the faster one is achieved.
>
> Well, I propose to commit the new "multiply" method. In fact, it is based
> on the Jama implementation.[1]
> But if you prefer to see it first, I can attach a patch to the report page.
> Let me know.
No, I will look at it once its committed, please go ahead.
Luc
>
>>
>> I am OK with the two other issues.
>
> Great!
>
> Regards,
> Gilles
>
> [1] Credits to E. Sarrazin for pointing it out. ;-)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
Hello Luc.
> >
> >Referring to:
> > https://issues.apache.org/jira/browse/MATH-650
> > https://issues.apache.org/jira/browse/MATH-675
> > https://issues.apache.org/jira/browse/MATH-676
> >
> >In the absence of additional comments I'm proposing to resolve those issues.
> >Please let me know whether there are pending objections to my suggested
> >solutions.
>
> Sorry, I was offline for a while, with scarce access to the net.
> What is the solution you propose for MATH-676, I only see the
> curves, but not how the faster one is achieved.
Well, I propose to commit the new "multiply" method. In fact, it is based
on the Jama implementation.[1]
But if you prefer to see it first, I can attach a patch to the report page.
Let me know.
>
> I am OK with the two other issues.
Great!
Regards,
Gilles
[1] Credits to E. Sarrazin for pointing it out. ;-)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [Math] Issues 650, 675, 676
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 29/09/2011 18:04, Gilles Sadowski a écrit :
> Hello.
>
> Referring to:
> https://issues.apache.org/jira/browse/MATH-650
> https://issues.apache.org/jira/browse/MATH-675
> https://issues.apache.org/jira/browse/MATH-676
>
> In the absence of additional comments I'm proposing to resolve those issues.
> Please let me know whether there are pending objections to my suggested
> solutions.
Sorry, I was offline for a while, with scarce access to the net.
What is the solution you propose for MATH-676, I only see the curves,
but not how the faster one is achieved.
I am OK with the two other issues.
Luc
>
>
> Thanks,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org