You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Sébastien Brisard <se...@m4x.org> on 2011/08/08 20:52:18 UTC
[math] Monitoring iterative algorithms
Hi,
a useful feature for *any* iterative algorithm would be the
possibility to call a "monitor" at the end of each iteration. What
would this monitor do would pretty much depend on the user's mood:
periodically backup the state of the algorithm (in order to restart it
in case of a crash), log a set of tabulated values (to plot the
convergence of the algorithm), ...
The idea would be to come up with a design as general as possible.
Gilles proposed (see MATH-581) to use java.util.Observable. This
sounds very attractive to me. Any comments?
Best regards,
Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
Hi.
> I think we do agree on most points:
>
> For very-limited functionality, this could be an option. The big drawback
> > is that, if it takes time to implement, the merge with the "official"
> > branch
> > may not be trivial. [IIUC, Ted suggested that "git" was superior to "svn"
> > in making this less of a pain.]
> >
> >
> Yes, smaller, "snap on" features are probably best addressed this way. The
> issues with merging a feature branch into trunk are well known. The more
> things a feature touches, the more merge conflicts present themselves.
> However, in the context of monitoring iterative algorithms the merge should
> be manageable. There are a lot of these types of minor to medium intensity
> feature needs in math commons.
Yes, because the feature is new: It will probably mostly add code, not
replace something existing which itself could evolve during the branch
work...
It was just a (maybe unnecessary) warning.
>
> > For functionality that encompasses a large part of the library, or a design
> > change, or a policy change, it will be even more difficult, even if your
> > proposal fits the initial bill as discussed on the ML.
> > I've attempted one such experiment. You could search the archive for
> > "cal10n". [Another project by Ceki Gulcu. Oops ;-).] I was asked to create
> > a
> > "sandbox" branch; I did all the proof-of-concept coding. In the end,
> > nobody[1] checked out that branch for actual testing and the proposal was
> > rejected because none of its merits could redeem the supposedly fundamental
> > flaw of being (optionally) based on an external library (although that had
> > been made very clear on the ML, also during a very long discussion).
> > So, for me, never again! And I wouldn't dare advise it to anyone.
> >
>
> Shame on the community in this case. If the community requests a feature and
> you do the work for no reason, that's an indictment of the participants and
> organizer. That's a risk that the developer always faces in these instances.
To be fair, it was not a "community" request; it was a request from me. [See
the discussion in the archives, if you are interested ;-).]
Basically, my position was that CM should focus on math code; for _nothing_
else should it try to reinvent the wheel. In this instance this clashed with
the current policy of not depending on external libraries.
The same problem arises with logging.
And with monitoring: If there would already exist a good (pure-Java) solution
somewhere, it should be re-used, not re-implemented.
> I don't quite see how that's argues against frequent feature branching.
It was just an example of unnecessary work because the decision was made
without regards to the branch code. It was a policy-based decision (no
external library).
>
> Practically, given the low human resources (w.r.t the amount of code) in CM,
> > I think that evolving the code together on a single branch is better.
> >
>
> I think if we clean up the log jam that the discussions sometimes generate,
> we can maximize the effectiveness of the human resources.
>
> As for the cost of branching, it is almost trivial. svn copy https://fromurl
> https://tourl
>
>
>
> > Nevertheless I totally agree with you, and I've said many times on this
> > list,
> > that
> > * we should not expect to solve all problems in one ultimate design
> > * we should release more often
> > * we should not stick to old ways
> >
>
> Who could argue against these points? I am only advancing the argument that
> a branch allows for trying more new ideas. As for sticking to the old ways.
> Not sure what you mean.
Of course, I'm not trying to prevent you from working on a branch. I just
said that I wouldn't recommend it myself. Also because it will require
people to explicitly check out your branch and they might not do it in a
timely fashion so that it might become so out of date that you'll have to do
more work on it to keep in sync with the evolving trunk etc.
This would not be a problem if "lazy consensus" were always working. As far
as I'm concerned it did not work for redesigning the use of exceptions. Cf.
the exception debate that goes on since issue MATH-195.
[As an example of "sticking to old ways".]
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
This is very much what I had in mind. The downside is that the committer is
saddled with a recurring task. -Greg
On Wed, Aug 10, 2011 at 1:39 PM, Ted Dunning <te...@gmail.com> wrote:
> I have been able to manage some long-lived branches in Zookeeper and Mahout
> using git.
>
> The key was to rebase the branch frequently. That allowed me to spread the
> complexity of the ultimate merge out into a lot of small decisions, each of
> which was pretty simple and could be tracked back to a small change in
> trunk.
>
> The branch in Zookeeper was a particularly interesting case since it
> involved changes to dozens of files with lots of commits from multiple
> people on the branch and even more changes on the trunk while the branch
> lasted. The branch was not even a very short-term thing ... from first
> commit to final merge was 6-7 months.
>
> The key here was to not get too far behind. During active periods of
> development, I rebased almost every day.
>
> On Wed, Aug 10, 2011 at 11:21 AM, Luc Maisonobe <Luc.Maisonobe@free.fr
> >wrote:
>
> > As for the cost of branching, it is almost trivial. svn copy
> >> https://fromurl
> >> https://tourl
> >>
> >
> > The cost of branching is trivial. The cost of merging back is huge.
>
Re: [math] Monitoring iterative algorithms
Posted by Ted Dunning <te...@gmail.com>.
I have been able to manage some long-lived branches in Zookeeper and Mahout
using git.
The key was to rebase the branch frequently. That allowed me to spread the
complexity of the ultimate merge out into a lot of small decisions, each of
which was pretty simple and could be tracked back to a small change in
trunk.
The branch in Zookeeper was a particularly interesting case since it
involved changes to dozens of files with lots of commits from multiple
people on the branch and even more changes on the trunk while the branch
lasted. The branch was not even a very short-term thing ... from first
commit to final merge was 6-7 months.
The key here was to not get too far behind. During active periods of
development, I rebased almost every day.
On Wed, Aug 10, 2011 at 11:21 AM, Luc Maisonobe <Lu...@free.fr>wrote:
> As for the cost of branching, it is almost trivial. svn copy
>> https://fromurl
>> https://tourl
>>
>
> The cost of branching is trivial. The cost of merging back is huge.
Re: [math] Monitoring iterative algorithms
Posted by Sébastien Brisard <se...@m4x.org>.
I forgot a very important question: would this classes be applicable
to ODEs (see Luc's explanations below). If not, I can probably forget
about them, since, I think this package makes the most extensive use
of events at the moment in CM...
Sébastien
>
> So here are the needs for ODE:
> - callbacks for regular steps when they complete, without
> any feedback with the algorithm. This is now implemented using
> StepHandler and FixedStepHandler and the handlerStep method is void,
> so from the algorithm point of view this monitoring is almost no-op.
> - callbacks for discrete events, which may provide feedback to the
> algorithm. This is now implemented using EventHandler and requires
> a function to define the event (the g function, events occurring at
> sign changes) and a function to be triggered when the events occurs
> (the eventOccurred function, which provides the current state and
> which return value is used by the algorithm to know what to do next,
> i.e. continue/stop/reset state/reset derivatives.
>
> These needs are really, really important and in fact are one of the main
> feature in Commons Math ODE (and prior to that from Mantissa) that was often
> praised by users.
>
> Do you think we should design something now for 3.0 or should it wait 4.0 ?
> I would love to have such a new feature soon, but fear it could delay 3.0
> too much.
>
> Luc
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Sébastien Brisard <se...@m4x.org>.
Hi,
I'm reviving this thread to let you know that I took the liberty to
create a new JIRA ticket on this issue (see JIRA MATH-655). A few
classes are attached to this ticket for you to review. These classes
provide support for managing the iteration count as well as event
occuring before, during and after the main iteration loop. These
classes do *not* provide yet support for stopping criterion. I am
willing to have a go at this, but it seems it will be difficult to get
it right (see discussions on MATH-413) [1], and I would like to keep
in mind Phil's recommendations
>
> I think if you are careful, you can likely set this up so that you
> can continue to add features to the event framework (or even add the
> framework itself) and stopping criteria without breaking backward
> compatibility in 3.x releases.
>
> I would like to see this in 3.0 if you can get something simple
> completed that can be enhanced incrementally in 3.x and possibly
> refactored in 4.0.
>
Do you think the proposed classes could be useful? Do you think the
above requirements are fulfilled? If the answers are yes, where should
they go?
Best regards,
Sébastien
[1] Whatever is done on stopping criteria, I think that one important
point made in MATH-413 is that a default stopping criterion should be
hard-coded in the algorithm, for two reasons: i. implementation is
usually straightforward (e.g. the residual is already available, and
does not need to be recomputed), and ii. end-users might well be very
happy with this default stopping criterion. So, the stopping criteria
we are talking about here should really be understood as *additional*
stopping criteria.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 11/08/2011 06:14, Phil Steitz a écrit :
> On 8/10/11 8:19 PM, Sébastien Brisard wrote:
>> Hello,
>> going back to the initial conversation. It seems to me that
>> formalizing Iterative Algorithms in a general way is very interesting,
>> but not a realistic target for 3.0 (or probably even 3.1). However, I
>> would very much like to have a satisfactory working version of linear
>> iterative solvers. I've already implemented them at work, because I
>> really need them. Including them in Commons Math seems to me highly
>> desirable, but if that too is unrealistic, let me know. As I'm doing
>> this last part in my spare time, I would become available to work on
>> other, more urgent open issues.
>> So here is what I propose. For the time being, iterative linear
>> solvers can live without observers, since I can reverse a previous
>> argument: convergence checkers can be seen as monitors... so I can
>> (dirtily) tweak a checker to monitor my solver (this would be a
>> temporary solution, waiting for something better). Regarding
>> convergence checkers themselves (the other major issue raised in the
>> code I have already proposed), I think the Object model is easier to
>> derive, and something like
>> StoppingCriterion<STATE>{
>> init(LinearIterativeSolver solver);
>>
>> boolean hasConverged(STATE s);
>> }
>> should work (will polish it a little bit). Making the stopping
>> criterion generic would allow later use in e.g. GA (STATE=Population)
>> and Optimization.
>>
>> On second (third? fourth?) thoughts, I've realized that
>> Obervable/Observer would probably not be flexible enough, even for
>> very simple observers. A real event listener should probably be
>> preferred. For example, in many iterative solvers, there is an
>> initialization phase, during which the loop has not started, but the
>> counter (function evaluations, matrix-vector products) has. So a
>> different event should probably be fired, depending on whether or not
>> we are in the main loop.
>>
>> OK, so, to sum up: I get rid of any Observer/Listener in the existing
>> code (this will come up later, once Iterative Algorithms at large have
>> been discussed). I write a generic enough stopping criterion
>> interface. This could be done rather quickly. Question is: would it be
>> agreeable to the Commons Math community?
>
> Sounds reasonable to me; though I would personally be fine with
> adding some small classes along the lines of what I outlined above
> to support events. They could even be package-scoped if we want to
> keep them out of the public API and replace later with more general
> constructs used elsewhere in [math]. We are not talking about much
> code at all here.
>
> I think if you are careful, you can likely set this up so that you
> can continue to add features to the event framework (or even add the
> framework itself) and stopping criteria without breaking backward
> compatibility in 3.x releases.
>
> I would like to see this in 3.0 if you can get something simple
> completed that can be enhanced incrementally in 3.x and possibly
> refactored in 4.0.
This seems a good path to go.
Luc
>
> Phil
>>
>> Best regards,
>> Sebastien
>>
>> 2011/8/10 Greg Sterijevski<gs...@gmail.com>:
>>> Luc,
>>>
>>> I think we misunderstood each other, in the snippet below, my intention was
>>> to agree with him. Everything Gilles was agreeable and hard to argue
>>> against.
>>>
>>> -Greg
>>>
>>>> Luc
>>>>
>>>>
>>>> Not sure what you mean.
>>>>>
>>>>> In my opinion, discussions should serve to solve real problems of a
>>>>>> software
>>>>>> library: bugs, design inconsistencies, efficiency improvements (in that
>>>>>> order).
>>>>>>
>>>>>>
>>>>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Sébastien Brisard <se...@m4x.org>.
>
> Sounds reasonable to me; though I would personally be fine with
> adding some small classes along the lines of what I outlined above
> to support events. They could even be package-scoped if we want to
> keep them out of the public API and replace later with more general
> constructs used elsewhere in [math]. We are not talking about much
> code at all here.
>
> I think if you are careful, you can likely set this up so that you
> can continue to add features to the event framework (or even add the
> framework itself) and stopping criteria without breaking backward
> compatibility in 3.x releases.
>
> I would like to see this in 3.0 if you can get something simple
> completed that can be enhanced incrementally in 3.x and possibly
> refactored in 4.0.
>
> Phil
>
Of course, I'm all for it! I didn't really like this fall-back
solution, which was more aimed at efficiency. I'll try to set up
something and submit it to you all (drawing inspiration from the event
handling in ode).
Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Phil Steitz <ph...@gmail.com>.
On 8/10/11 8:19 PM, Sébastien Brisard wrote:
> Hello,
> going back to the initial conversation. It seems to me that
> formalizing Iterative Algorithms in a general way is very interesting,
> but not a realistic target for 3.0 (or probably even 3.1). However, I
> would very much like to have a satisfactory working version of linear
> iterative solvers. I've already implemented them at work, because I
> really need them. Including them in Commons Math seems to me highly
> desirable, but if that too is unrealistic, let me know. As I'm doing
> this last part in my spare time, I would become available to work on
> other, more urgent open issues.
> So here is what I propose. For the time being, iterative linear
> solvers can live without observers, since I can reverse a previous
> argument: convergence checkers can be seen as monitors... so I can
> (dirtily) tweak a checker to monitor my solver (this would be a
> temporary solution, waiting for something better). Regarding
> convergence checkers themselves (the other major issue raised in the
> code I have already proposed), I think the Object model is easier to
> derive, and something like
> StoppingCriterion<STATE>{
> init(LinearIterativeSolver solver);
>
> boolean hasConverged(STATE s);
> }
> should work (will polish it a little bit). Making the stopping
> criterion generic would allow later use in e.g. GA (STATE=Population)
> and Optimization.
>
> On second (third? fourth?) thoughts, I've realized that
> Obervable/Observer would probably not be flexible enough, even for
> very simple observers. A real event listener should probably be
> preferred. For example, in many iterative solvers, there is an
> initialization phase, during which the loop has not started, but the
> counter (function evaluations, matrix-vector products) has. So a
> different event should probably be fired, depending on whether or not
> we are in the main loop.
>
> OK, so, to sum up: I get rid of any Observer/Listener in the existing
> code (this will come up later, once Iterative Algorithms at large have
> been discussed). I write a generic enough stopping criterion
> interface. This could be done rather quickly. Question is: would it be
> agreeable to the Commons Math community?
Sounds reasonable to me; though I would personally be fine with
adding some small classes along the lines of what I outlined above
to support events. They could even be package-scoped if we want to
keep them out of the public API and replace later with more general
constructs used elsewhere in [math]. We are not talking about much
code at all here.
I think if you are careful, you can likely set this up so that you
can continue to add features to the event framework (or even add the
framework itself) and stopping criteria without breaking backward
compatibility in 3.x releases.
I would like to see this in 3.0 if you can get something simple
completed that can be enhanced incrementally in 3.x and possibly
refactored in 4.0.
Phil
>
> Best regards,
> Sebastien
>
> 2011/8/10 Greg Sterijevski <gs...@gmail.com>:
>> Luc,
>>
>> I think we misunderstood each other, in the snippet below, my intention was
>> to agree with him. Everything Gilles was agreeable and hard to argue
>> against.
>>
>> -Greg
>>
>>> Luc
>>>
>>>
>>> Not sure what you mean.
>>>>
>>>> In my opinion, discussions should serve to solve real problems of a
>>>>> software
>>>>> library: bugs, design inconsistencies, efficiency improvements (in that
>>>>> order).
>>>>>
>>>>>
>>>>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Sébastien Brisard <se...@m4x.org>.
Hello,
going back to the initial conversation. It seems to me that
formalizing Iterative Algorithms in a general way is very interesting,
but not a realistic target for 3.0 (or probably even 3.1). However, I
would very much like to have a satisfactory working version of linear
iterative solvers. I've already implemented them at work, because I
really need them. Including them in Commons Math seems to me highly
desirable, but if that too is unrealistic, let me know. As I'm doing
this last part in my spare time, I would become available to work on
other, more urgent open issues.
So here is what I propose. For the time being, iterative linear
solvers can live without observers, since I can reverse a previous
argument: convergence checkers can be seen as monitors... so I can
(dirtily) tweak a checker to monitor my solver (this would be a
temporary solution, waiting for something better). Regarding
convergence checkers themselves (the other major issue raised in the
code I have already proposed), I think the Object model is easier to
derive, and something like
StoppingCriterion<STATE>{
init(LinearIterativeSolver solver);
boolean hasConverged(STATE s);
}
should work (will polish it a little bit). Making the stopping
criterion generic would allow later use in e.g. GA (STATE=Population)
and Optimization.
On second (third? fourth?) thoughts, I've realized that
Obervable/Observer would probably not be flexible enough, even for
very simple observers. A real event listener should probably be
preferred. For example, in many iterative solvers, there is an
initialization phase, during which the loop has not started, but the
counter (function evaluations, matrix-vector products) has. So a
different event should probably be fired, depending on whether or not
we are in the main loop.
OK, so, to sum up: I get rid of any Observer/Listener in the existing
code (this will come up later, once Iterative Algorithms at large have
been discussed). I write a generic enough stopping criterion
interface. This could be done rather quickly. Question is: would it be
agreeable to the Commons Math community?
Best regards,
Sebastien
2011/8/10 Greg Sterijevski <gs...@gmail.com>:
> Luc,
>
> I think we misunderstood each other, in the snippet below, my intention was
> to agree with him. Everything Gilles was agreeable and hard to argue
> against.
>
> -Greg
>
>>
>> Luc
>>
>>
>> Not sure what you mean.
>>>
>>>
>>> In my opinion, discussions should serve to solve real problems of a
>>>> software
>>>> library: bugs, design inconsistencies, efficiency improvements (in that
>>>> order).
>>>>
>>>>
>>>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
Luc,
I think we misunderstood each other, in the snippet below, my intention was
to agree with him. Everything Gilles was agreeable and hard to argue
against.
-Greg
>
> Luc
>
>
> Not sure what you mean.
>>
>>
>> In my opinion, discussions should serve to solve real problems of a
>>> software
>>> library: bugs, design inconsistencies, efficiency improvements (in that
>>> order).
>>>
>>>
>>>
Re: [math] Monitoring iterative algorithms
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 10/08/2011 16:30, Greg Sterijevski a écrit :
> Gilles,
>
> I think we do agree on most points:
>
> For very-limited functionality, this could be an option. The big drawback
>> is that, if it takes time to implement, the merge with the "official"
>> branch
>> may not be trivial. [IIUC, Ted suggested that "git" was superior to "svn"
>> in making this less of a pain.]
>>
>>
> Yes, smaller, "snap on" features are probably best addressed this way. The
> issues with merging a feature branch into trunk are well known. The more
> things a feature touches, the more merge conflicts present themselves.
> However, in the context of monitoring iterative algorithms the merge should
> be manageable. There are a lot of these types of minor to medium intensity
> feature needs in math commons.
We already used branches and we can use them again.
Gilles is perfectly right, it is difficult to maintain them and this
becomes more and more difficult as time passes. Short-lived branches are
OK, long-lasting branches are a nigthmare. I remember the pain prior to
release 2.2, which is on a branch. Synchronizing what we wanted from
trunk and removing it again after having found unexpected problems was
really difficult.
>
>
>
>> For functionality that encompasses a large part of the library, or a design
>> change, or a policy change, it will be even more difficult, even if your
>> proposal fits the initial bill as discussed on the ML.
>> I've attempted one such experiment. You could search the archive for
>> "cal10n". [Another project by Ceki Gulcu. Oops ;-).] I was asked to create
>> a
>> "sandbox" branch; I did all the proof-of-concept coding. In the end,
>> nobody[1] checked out that branch for actual testing and the proposal was
>> rejected because none of its merits could redeem the supposedly fundamental
>> flaw of being (optionally) based on an external library (although that had
>> been made very clear on the ML, also during a very long discussion).
>> So, for me, never again! And I wouldn't dare advise it to anyone.
>>
>
> Shame on the community in this case. If the community requests a feature and
> you do the work for no reason, that's an indictment of the participants and
> organizer.
Please, have a look at the archives.
> That's a risk that the developer always faces in these instances.
> I don't quite see how that's argues against frequent feature branching.
>
> Practically, given the low human resources (w.r.t the amount of code) in CM,
>> I think that evolving the code together on a single branch is better.
>>
>
> I think if we clean up the log jam that the discussions sometimes generate,
> we can maximize the effectiveness of the human resources.
>
> As for the cost of branching, it is almost trivial. svn copy https://fromurl
> https://tourl
The cost of branching is trivial. The cost of merging back is huge.
>
>
>
>> Nevertheless I totally agree with you, and I've said many times on this
>> list,
>> that
>> * we should not expect to solve all problems in one ultimate design
>> * we should release more often
>> * we should not stick to old ways
>>
>
> Who could argue against these points? I am only advancing the argument that
> a branch allows for trying more new ideas. As for sticking to the old ways.
I'm puzzled. There is nothing wrong with branches and committers can use
them. What Gilles says and I agree is that people should be aware of it
that the merge part can be difficult. So they should be prepared to do it.
Luc
> Not sure what you mean.
>
>
>> In my opinion, discussions should serve to solve real problems of a
>> software
>> library: bugs, design inconsistencies, efficiency improvements (in that
>> order).
>>
>>
>> Regards,
>> Gilles
>>
>> [1] Please correct me if I'm wrong, and I apologize if so.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
> Overall, thank you. The comments were good.
>
> -Greg
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
Gilles,
I think we do agree on most points:
For very-limited functionality, this could be an option. The big drawback
> is that, if it takes time to implement, the merge with the "official"
> branch
> may not be trivial. [IIUC, Ted suggested that "git" was superior to "svn"
> in making this less of a pain.]
>
>
Yes, smaller, "snap on" features are probably best addressed this way. The
issues with merging a feature branch into trunk are well known. The more
things a feature touches, the more merge conflicts present themselves.
However, in the context of monitoring iterative algorithms the merge should
be manageable. There are a lot of these types of minor to medium intensity
feature needs in math commons.
> For functionality that encompasses a large part of the library, or a design
> change, or a policy change, it will be even more difficult, even if your
> proposal fits the initial bill as discussed on the ML.
> I've attempted one such experiment. You could search the archive for
> "cal10n". [Another project by Ceki Gulcu. Oops ;-).] I was asked to create
> a
> "sandbox" branch; I did all the proof-of-concept coding. In the end,
> nobody[1] checked out that branch for actual testing and the proposal was
> rejected because none of its merits could redeem the supposedly fundamental
> flaw of being (optionally) based on an external library (although that had
> been made very clear on the ML, also during a very long discussion).
> So, for me, never again! And I wouldn't dare advise it to anyone.
>
Shame on the community in this case. If the community requests a feature and
you do the work for no reason, that's an indictment of the participants and
organizer. That's a risk that the developer always faces in these instances.
I don't quite see how that's argues against frequent feature branching.
Practically, given the low human resources (w.r.t the amount of code) in CM,
> I think that evolving the code together on a single branch is better.
>
I think if we clean up the log jam that the discussions sometimes generate,
we can maximize the effectiveness of the human resources.
As for the cost of branching, it is almost trivial. svn copy https://fromurl
https://tourl
> Nevertheless I totally agree with you, and I've said many times on this
> list,
> that
> * we should not expect to solve all problems in one ultimate design
> * we should release more often
> * we should not stick to old ways
>
Who could argue against these points? I am only advancing the argument that
a branch allows for trying more new ideas. As for sticking to the old ways.
Not sure what you mean.
> In my opinion, discussions should serve to solve real problems of a
> software
> library: bugs, design inconsistencies, efficiency improvements (in that
> order).
>
>
> Regards,
> Gilles
>
> [1] Please correct me if I'm wrong, and I apologize if so.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Overall, thank you. The comments were good.
-Greg
Re: [math] Monitoring iterative algorithms
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
Hello.
> I think you hit the nail on the head with the comment:
>
> "... but also the time to
> experiment. Only the latter will be able to tell if the design is good.
> And this must take time so that all the potential pitfalls can be ..."
>
> Maybe this is chumming the water with more flotsam and jetsam, but a lot of
> the issues that arise in working out the best design ultimately revert to
> trying out a bunch of deadends [design by monte carlo ;)] The discussion on
> the list [from my rather limited history] seems to try to foresee all
> possible drawbacks with a design. This inevitably devolves into heated
> discussion and diminishing progress.
>
> Would it be possible to fork the trunk of the source tree to an
> "experimental branch"? Whether its monitoring the progress of ODE solvers,
> other solvers or any other feature extension, work could proceed on this
> experimental branch unmolested. There would be no concern about whether is
> this a 4.0 or 3.0 feature. The feature would be folded into trunk once it is
> ready (as evidenced by a vote of all concerned parties). Furthermore, all of
> us would have something very concrete to discuss. It would allow all
> interested parties to submit competing versions-instead of arguing over
> snippets of code. Phil has stressed to me that once an API change is made,
> that change is almost set in stone. An experimental branch would allow us to
> be more flexible and make mistakes without fear of supporting a garbage
> interface.
For very-limited functionality, this could be an option. The big drawback
is that, if it takes time to implement, the merge with the "official" branch
may not be trivial. [IIUC, Ted suggested that "git" was superior to "svn"
in making this less of a pain.]
For functionality that encompasses a large part of the library, or a design
change, or a policy change, it will be even more difficult, even if your
proposal fits the initial bill as discussed on the ML.
I've attempted one such experiment. You could search the archive for
"cal10n". [Another project by Ceki Gulcu. Oops ;-).] I was asked to create a
"sandbox" branch; I did all the proof-of-concept coding. In the end,
nobody[1] checked out that branch for actual testing and the proposal was
rejected because none of its merits could redeem the supposedly fundamental
flaw of being (optionally) based on an external library (although that had
been made very clear on the ML, also during a very long discussion).
So, for me, never again! And I wouldn't dare advise it to anyone.
Practically, given the low human resources (w.r.t the amount of code) in CM,
I think that evolving the code together on a single branch is better.
Nevertheless I totally agree with you, and I've said many times on this list,
that
* we should not expect to solve all problems in one ultimate design
* we should release more often
* we should not stick to old ways
In my opinion, discussions should serve to solve real problems of a software
library: bugs, design inconsistencies, efficiency improvements (in that
order).
Regards,
Gilles
[1] Please correct me if I'm wrong, and I apologize if so.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Ted Dunning <te...@gmail.com>.
This is a great idea.
And it is something you can just do on github.
On Tue, Aug 9, 2011 at 10:50 AM, Greg Sterijevski <gs...@gmail.com>wrote:
> Would it be possible to fork the trunk of the source tree to an
> "experimental branch"? Whether its monitoring the progress of ODE solvers,
> other solvers or any other feature extension, work could proceed on this
> experimental branch unmolested
>
Re: [math] Monitoring iterative algorithms
Posted by Sébastien Brisard <se...@m4x.org>.
In that case, I'm quite happy to test one of the options which have
been suggested within the framework of linear solvers.
I could for example try and implement the Obervable/Observer option,
and see how it goes.
Sébastien
2011/8/9 Mark Thomas <ma...@apache.org>:
> On 09/08/2011 18:50, Greg Sterijevski wrote:
>> I think you hit the nail on the head with the comment:
>>
>> "... but also the time to
>> experiment. Only the latter will be able to tell if the design is good.
>> And this must take time so that all the potential pitfalls can be ..."
>>
>> Maybe this is chumming the water with more flotsam and jetsam, but a lot of
>> the issues that arise in working out the best design ultimately revert to
>> trying out a bunch of deadends [design by monte carlo ;)] The discussion on
>> the list [from my rather limited history] seems to try to foresee all
>> possible drawbacks with a design. This inevitably devolves into heated
>> discussion and diminishing progress.
>>
>> Would it be possible to fork the trunk of the source tree to an
>> "experimental branch"?
>
> The idea has been around almost as long as the ASF has:
>
> http://incubator.apache.org/learn/rules-for-revolutionaries.html
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
I was not aware that someone has codified this. The "Rules for
Revolutionaries" is very very good. There is an elegant order out of
disorder theme that permeates it. Thank you. -Greg
On Tue, Aug 9, 2011 at 1:04 PM, Mark Thomas <ma...@apache.org> wrote:
> On 09/08/2011 18:50, Greg Sterijevski wrote:
> > I think you hit the nail on the head with the comment:
> >
> > "... but also the time to
> > experiment. Only the latter will be able to tell if the design is good.
> > And this must take time so that all the potential pitfalls can be ..."
> >
> > Maybe this is chumming the water with more flotsam and jetsam, but a lot
> of
> > the issues that arise in working out the best design ultimately revert to
> > trying out a bunch of deadends [design by monte carlo ;)] The discussion
> on
> > the list [from my rather limited history] seems to try to foresee all
> > possible drawbacks with a design. This inevitably devolves into heated
> > discussion and diminishing progress.
> >
> > Would it be possible to fork the trunk of the source tree to an
> > "experimental branch"?
>
> The idea has been around almost as long as the ASF has:
>
> http://incubator.apache.org/learn/rules-for-revolutionaries.html
>
> Mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Monitoring iterative algorithms
Posted by Mark Thomas <ma...@apache.org>.
On 09/08/2011 18:50, Greg Sterijevski wrote:
> I think you hit the nail on the head with the comment:
>
> "... but also the time to
> experiment. Only the latter will be able to tell if the design is good.
> And this must take time so that all the potential pitfalls can be ..."
>
> Maybe this is chumming the water with more flotsam and jetsam, but a lot of
> the issues that arise in working out the best design ultimately revert to
> trying out a bunch of deadends [design by monte carlo ;)] The discussion on
> the list [from my rather limited history] seems to try to foresee all
> possible drawbacks with a design. This inevitably devolves into heated
> discussion and diminishing progress.
>
> Would it be possible to fork the trunk of the source tree to an
> "experimental branch"?
The idea has been around almost as long as the ASF has:
http://incubator.apache.org/learn/rules-for-revolutionaries.html
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
I think you hit the nail on the head with the comment:
"... but also the time to
experiment. Only the latter will be able to tell if the design is good.
And this must take time so that all the potential pitfalls can be ..."
Maybe this is chumming the water with more flotsam and jetsam, but a lot of
the issues that arise in working out the best design ultimately revert to
trying out a bunch of deadends [design by monte carlo ;)] The discussion on
the list [from my rather limited history] seems to try to foresee all
possible drawbacks with a design. This inevitably devolves into heated
discussion and diminishing progress.
Would it be possible to fork the trunk of the source tree to an
"experimental branch"? Whether its monitoring the progress of ODE solvers,
other solvers or any other feature extension, work could proceed on this
experimental branch unmolested. There would be no concern about whether is
this a 4.0 or 3.0 feature. The feature would be folded into trunk once it is
ready (as evidenced by a vote of all concerned parties). Furthermore, all of
us would have something very concrete to discuss. It would allow all
interested parties to submit competing versions-instead of arguing over
snippets of code. Phil has stressed to me that once an API change is made,
that change is almost set in stone. An experimental branch would allow us to
be more flexible and make mistakes without fear of supporting a garbage
interface.
-Greg
It's not only the time to think, implement and test, but also the time to
> experiment. Only the latter will be able to tell if the design is good.
> And this must take time so that all the potential pitfalls can be
> encountered...
>
> Also, for what it's worth, I'd prefer everyone to focus on the open issues
> and advance towards 3.0. It's seemed that the thread which you started
> about
> a release plan was rather short-lived. Most unfortunately.
>
>
> Regards,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Monitoring iterative algorithms
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
On Tue, Aug 09, 2011 at 05:46:47PM +0200, Luc Maisonobe wrote:
> Le 09/08/2011 09:28, Phil Steitz a écrit :
> >On 8/8/11 10:46 PM, Sébastien Brisard wrote:
> >>2011/8/8 Phil Steitz<ph...@gmail.com>:
> >>>+1 to the idea of using the Observer pattern; but -0 for
> >>>Observable. I would favor defining Events and Listeners because a)
> >>>Observable is concrete, so effectively forces you to create an
> >>>Observable delegate
> >>>
> >>I'm not sure I understand. Components in java.awt also need to
> >>implement quite a few methods like addListener, fireEvent, and so on.
> >>Maybe we could write a DefaultObservable, which would implement most
> >>of the methods required by Observable, and could be inherited by some
> >>(probably not all, unfortunately) of the already implemented iterative
> >>algorithms ?
> >
> >I was suggesting that we dispense with having anything inherit from
> >Observable and just define events and listeners, similarly to what
> >you find in the awt components, but also elsewhere like javax.sql or
> >javax.servlet (and I am sure lots of other places too). Observable
> >just gives you hasChanged and a primitive, non-type-safe
> >notifyListeners. To do proper event propagation you need to define
> >custom events anyway and register listeners, so unless all you need
> >to raise are "some state has changed" events (like what a GUI
> >presentation layer needs from the model in MVC), you are better off
> >implementing a proper event framework. This does not have to be
> >heavyweight. Just define
> >
> >0) Event class(es) extending j.u.EventObject
> >1) a listener interface extending j.u.EventListener
> >2) implement addXxxListener, removeXxxListener and
> >fireInterstingEvent1...n in the class that you want to be observable
> >(i.e. source events)
> >
> >Items 0) and 1) and most of 2) are going to be necessary anyway if
> >you want to propagate typesafe events. All that you really save by
> >bringing in Observable is maintenance of the listener collection,
> >which is trivial.
> >
> ><snip/>
> >
> >>Browsing through the JavaDoc, I realized that o.a.c.m.ode has some
> >>event handling facilities. Is the implementer still around?
> >
> >That would be Luc.
>
> I'm here.
> Sorry for the delay, I am on holidays for three weeks and working in
> my house.
>
> When I read the first mail in this thread a few minutes ago, I
> thought: this is close to what we have in ODE. Then I remembered a
> discussion with Dietmar and Nikolaus when we met at an optimization
> workshop in Toulouse: they also raised the need for some
> monitoring/logging facility in optimization algorithms. Gilles also
> raise this problem in this thread.
>
> So I would now say: we do have both an event framework *and* a step
> handler framework in ODE, but they are both tailored for ODE.
> If we can think of a more general framework, I would say this
> framework should be used for all our iterative algorithms and the
> ODE framework should be changed to use it too.
>
> So here are the needs for ODE:
> - callbacks for regular steps when they complete, without
> any feedback with the algorithm. This is now implemented using
> StepHandler and FixedStepHandler and the handlerStep method is void,
> so from the algorithm point of view this monitoring is almost no-op.
> - callbacks for discrete events, which may provide feedback to the
> algorithm. This is now implemented using EventHandler and requires
> a function to define the event (the g function, events occurring at
> sign changes) and a function to be triggered when the events occurs
> (the eventOccurred function, which provides the current state and
> which return value is used by the algorithm to know what to do next,
> i.e. continue/stop/reset state/reset derivatives.
>
> These needs are really, really important and in fact are one of the main
> feature in Commons Math ODE (and prior to that from Mantissa) that
> was often praised by users.
>
> Do you think we should design something now for 3.0 or should it
> wait 4.0 ? I would love to have such a new feature soon, but fear it
> could delay 3.0 too much.
It's not only the time to think, implement and test, but also the time to
experiment. Only the latter will be able to tell if the design is good.
And this must take time so that all the potential pitfalls can be
encountered...
Also, for what it's worth, I'd prefer everyone to focus on the open issues
and advance towards 3.0. It's seemed that the thread which you started about
a release plan was rather short-lived. Most unfortunately.
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Luc Maisonobe <Lu...@free.fr>.
Le 09/08/2011 09:28, Phil Steitz a écrit :
> On 8/8/11 10:46 PM, Sébastien Brisard wrote:
>> 2011/8/8 Phil Steitz<ph...@gmail.com>:
>>> +1 to the idea of using the Observer pattern; but -0 for
>>> Observable. I would favor defining Events and Listeners because a)
>>> Observable is concrete, so effectively forces you to create an
>>> Observable delegate
>>>
>> I'm not sure I understand. Components in java.awt also need to
>> implement quite a few methods like addListener, fireEvent, and so on.
>> Maybe we could write a DefaultObservable, which would implement most
>> of the methods required by Observable, and could be inherited by some
>> (probably not all, unfortunately) of the already implemented iterative
>> algorithms ?
>
> I was suggesting that we dispense with having anything inherit from
> Observable and just define events and listeners, similarly to what
> you find in the awt components, but also elsewhere like javax.sql or
> javax.servlet (and I am sure lots of other places too). Observable
> just gives you hasChanged and a primitive, non-type-safe
> notifyListeners. To do proper event propagation you need to define
> custom events anyway and register listeners, so unless all you need
> to raise are "some state has changed" events (like what a GUI
> presentation layer needs from the model in MVC), you are better off
> implementing a proper event framework. This does not have to be
> heavyweight. Just define
>
> 0) Event class(es) extending j.u.EventObject
> 1) a listener interface extending j.u.EventListener
> 2) implement addXxxListener, removeXxxListener and
> fireInterstingEvent1...n in the class that you want to be observable
> (i.e. source events)
>
> Items 0) and 1) and most of 2) are going to be necessary anyway if
> you want to propagate typesafe events. All that you really save by
> bringing in Observable is maintenance of the listener collection,
> which is trivial.
>
> <snip/>
>
>> Browsing through the JavaDoc, I realized that o.a.c.m.ode has some
>> event handling facilities. Is the implementer still around?
>
> That would be Luc.
I'm here.
Sorry for the delay, I am on holidays for three weeks and working in my
house.
When I read the first mail in this thread a few minutes ago, I thought:
this is close to what we have in ODE. Then I remembered a discussion
with Dietmar and Nikolaus when we met at an optimization workshop in
Toulouse: they also raised the need for some monitoring/logging facility
in optimization algorithms. Gilles also raise this problem in this thread.
So I would now say: we do have both an event framework *and* a step
handler framework in ODE, but they are both tailored for ODE.
If we can think of a more general framework, I would say this framework
should be used for all our iterative algorithms and the ODE framework
should be changed to use it too.
So here are the needs for ODE:
- callbacks for regular steps when they complete, without
any feedback with the algorithm. This is now implemented using
StepHandler and FixedStepHandler and the handlerStep method is void,
so from the algorithm point of view this monitoring is almost no-op.
- callbacks for discrete events, which may provide feedback to the
algorithm. This is now implemented using EventHandler and requires
a function to define the event (the g function, events occurring at
sign changes) and a function to be triggered when the events occurs
(the eventOccurred function, which provides the current state and
which return value is used by the algorithm to know what to do next,
i.e. continue/stop/reset state/reset derivatives.
These needs are really, really important and in fact are one of the main
feature in Commons Math ODE (and prior to that from Mantissa) that was
often praised by users.
Do you think we should design something now for 3.0 or should it wait
4.0 ? I would love to have such a new feature soon, but fear it could
delay 3.0 too much.
Luc
>
> Phil
>> Sebastien
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
> >> +1 to the idea of using the Observer pattern; but -0 for
> >> Observable. I would favor defining Events and Listeners because a)
> >> Observable is concrete, so effectively forces you to create an
> >> Observable delegate
> >>
> > I'm not sure I understand. Components in java.awt also need to
> > implement quite a few methods like addListener, fireEvent, and so on.
> > Maybe we could write a DefaultObservable, which would implement most
> > of the methods required by Observable, and could be inherited by some
> > (probably not all, unfortunately) of the already implemented iterative
> > algorithms ?
>
> I was suggesting that we dispense with having anything inherit from
> Observable and just define events and listeners, similarly to what
> you find in the awt components, but also elsewhere like javax.sql or
> javax.servlet (and I am sure lots of other places too). Observable
> just gives you hasChanged and a primitive, non-type-safe
> notifyListeners. To do proper event propagation you need to define
> custom events anyway and register listeners, so unless all you need
> to raise are "some state has changed" events (like what a GUI
> presentation layer needs from the model in MVC), you are better off
> implementing a proper event framework. This does not have to be
> heavyweight. Just define
>
> 0) Event class(es) extending j.u.EventObject
> 1) a listener interface extending j.u.EventListener
> 2) implement addXxxListener, removeXxxListener and
> fireInterstingEvent1...n in the class that you want to be observable
> (i.e. source events)
I don't object that it would be more elegant or type-safe but this is
already more than the one method that would need to be delegated
to an "Obervable" instance stored in the class to be monitored.
For the kind of monitoring that is envisioned here (i.e. call the monitor
after each iteration, I feel that the direct/simple/crude approach may be
enough: It would use a standard class, it might be sufficient for
Sébastien's current need, it would be fairly easy to implement.
Later on (4.0), we can always switch to a more featureful monitoring setup.
> Items 0) and 1) and most of 2) are going to be necessary anyway if
> you want to propagate typesafe events. All that you really save by
> bringing in Observable is maintenance of the listener collection,
> which is trivial.
As I don't know anything about JMX, I have no idea of what it would take to
get to a workable proof-of-concept, i.e. something which we can test with
various classes that could be monitored, so that we can be convinced that
the design is fine.
This additional work-load is a consequence of not willing to depend on
external libraries.
Is there someone willing to work on this, i.e. search for similar tools, and
duplicate the work so that CM can continue to not depend on other libraries?
I also think that before embarking on the monitoring issue using concepts
like "Event", "Listener" and JMX, it might be wise to (re-)discuss a
seemingly less complex but related issue issue: logging.
Depending on the conclusion reached on that topic, the approach to the other
issue might be different.
> [...]
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Phil Steitz <ph...@gmail.com>.
On 8/8/11 10:46 PM, Sébastien Brisard wrote:
> 2011/8/8 Phil Steitz <ph...@gmail.com>:
>> +1 to the idea of using the Observer pattern; but -0 for
>> Observable. I would favor defining Events and Listeners because a)
>> Observable is concrete, so effectively forces you to create an
>> Observable delegate
>>
> I'm not sure I understand. Components in java.awt also need to
> implement quite a few methods like addListener, fireEvent, and so on.
> Maybe we could write a DefaultObservable, which would implement most
> of the methods required by Observable, and could be inherited by some
> (probably not all, unfortunately) of the already implemented iterative
> algorithms ?
I was suggesting that we dispense with having anything inherit from
Observable and just define events and listeners, similarly to what
you find in the awt components, but also elsewhere like javax.sql or
javax.servlet (and I am sure lots of other places too). Observable
just gives you hasChanged and a primitive, non-type-safe
notifyListeners. To do proper event propagation you need to define
custom events anyway and register listeners, so unless all you need
to raise are "some state has changed" events (like what a GUI
presentation layer needs from the model in MVC), you are better off
implementing a proper event framework. This does not have to be
heavyweight. Just define
0) Event class(es) extending j.u.EventObject
1) a listener interface extending j.u.EventListener
2) implement addXxxListener, removeXxxListener and
fireInterstingEvent1...n in the class that you want to be observable
(i.e. source events)
Items 0) and 1) and most of 2) are going to be necessary anyway if
you want to propagate typesafe events. All that you really save by
bringing in Observable is maintenance of the listener collection,
which is trivial.
<snip/>
> Browsing through the JavaDoc, I realized that o.a.c.m.ode has some
> event handling facilities. Is the implementer still around?
That would be Luc.
Phil
> Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Ole Ersoy <ol...@gmail.com>.
One approach that will minimize the burden on commons math developers is to point users to an AspectJ approach as is discussed here:
http://www.ibm.com/developerworks/java/library/j-aopwork6/index.html
The article starts out with a discussion of how to implement the observer design pattern the classic way, along with the associated design and maintenance costs, and then explains how to do it with AspectJ which leaves the class of the objects being observed in its original form.
Cheers,
- Ole
On 08/09/2011 12:46 AM, Sébastien Brisard wrote:
> 2011/8/8 Phil Steitz<ph...@gmail.com>:
>>
>> +1 to the idea of using the Observer pattern; but -0 for
>> Observable. I would favor defining Events and Listeners because a)
>> Observable is concrete, so effectively forces you to create an
>> Observable delegate
>>
> I'm not sure I understand. Components in java.awt also need to
> implement quite a few methods like addListener, fireEvent, and so on.
> Maybe we could write a DefaultObservable, which would implement most
> of the methods required by Observable, and could be inherited by some
> (probably not all, unfortunately) of the already implemented iterative
> algorithms ?
>>
>> and b) the interface is crude and is really
>> designed for GUI scenarios where observers are holding references to
>> the Observable and just being notified that something has changed.
>> Events are more flexible and can be implemented using interfaces.
>> Has anyone ever actually used Observable outside of a GUI m/v
>> environment? Does anyone know of event frameworks suitable for
>> monitoring long-running computations that we might adapt?
>>
>> In any case, I think it is a good idea to develop an event framework
>> for [math]. We should probably also think about doing this in a way
>> that is at least JMX-friendly.
>>
>> Phil
>>
> Browsing through the JavaDoc, I realized that o.a.c.m.ode has some
> event handling facilities. Is the implementer still around?
> Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Sébastien Brisard <se...@m4x.org>.
2011/8/8 Phil Steitz <ph...@gmail.com>:
>
> +1 to the idea of using the Observer pattern; but -0 for
> Observable. I would favor defining Events and Listeners because a)
> Observable is concrete, so effectively forces you to create an
> Observable delegate
>
I'm not sure I understand. Components in java.awt also need to
implement quite a few methods like addListener, fireEvent, and so on.
Maybe we could write a DefaultObservable, which would implement most
of the methods required by Observable, and could be inherited by some
(probably not all, unfortunately) of the already implemented iterative
algorithms ?
>
> and b) the interface is crude and is really
> designed for GUI scenarios where observers are holding references to
> the Observable and just being notified that something has changed.
> Events are more flexible and can be implemented using interfaces.
> Has anyone ever actually used Observable outside of a GUI m/v
> environment? Does anyone know of event frameworks suitable for
> monitoring long-running computations that we might adapt?
>
> In any case, I think it is a good idea to develop an event framework
> for [math]. We should probably also think about doing this in a way
> that is at least JMX-friendly.
>
> Phil
>
Browsing through the JavaDoc, I realized that o.a.c.m.ode has some
event handling facilities. Is the implementer still around?
Sebastien
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
+1 on JMX framework.
On Mon, Aug 8, 2011 at 3:41 PM, Phil Steitz <ph...@gmail.com> wrote:
> On 8/8/11 11:52 AM, Sébastien Brisard wrote:
> > Hi,
> > a useful feature for *any* iterative algorithm would be the
> > possibility to call a "monitor" at the end of each iteration. What
> > would this monitor do would pretty much depend on the user's mood:
> > periodically backup the state of the algorithm (in order to restart it
> > in case of a crash), log a set of tabulated values (to plot the
> > convergence of the algorithm), ...
> > The idea would be to come up with a design as general as possible.
> > Gilles proposed (see MATH-581) to use java.util.Observable. This
> > sounds very attractive to me. Any comments?
>
> +1 to the idea of using the Observer pattern; but -0 for
> Observable. I would favor defining Events and Listeners because a)
> Observable is concrete, so effectively forces you to create an
> Observable delegate and b) the interface is crude and is really
> designed for GUI scenarios where observers are holding references to
> the Observable and just being notified that something has changed.
> Events are more flexible and can be implemented using interfaces.
> Has anyone ever actually used Observable outside of a GUI m/v
> environment? Does anyone know of event frameworks suitable for
> monitoring long-running computations that we might adapt?
>
> In any case, I think it is a good idea to develop an event framework
> for [math]. We should probably also think about doing this in a way
> that is at least JMX-friendly.
>
> Phil
> > Best regards,
> > Sebastien
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] Monitoring iterative algorithms
Posted by Gilles Sadowski <gi...@harfang.homelinux.org>.
> > a useful feature for *any* iterative algorithm would be the
> > possibility to call a "monitor" at the end of each iteration. What
> > would this monitor do would pretty much depend on the user's mood:
> > periodically backup the state of the algorithm (in order to restart it
> > in case of a crash), log a set of tabulated values (to plot the
> > convergence of the algorithm), ...
> > The idea would be to come up with a design as general as possible.
> > Gilles proposed (see MATH-581) to use java.util.Observable. This
> > sounds very attractive to me. Any comments?
>
> +1 to the idea of using the Observer pattern; but -0 for
> Observable. I would favor defining Events and Listeners because a)
> Observable is concrete, so effectively forces you to create an
> Observable delegate and b) the interface is crude and is really
> designed for GUI scenarios where observers are holding references to
> the Observable and just being notified that something has changed.
Indeed, that would be all that is necessary for a simple home-made
monitoring solution.
I fear that a more complex functionality would entail reinventing the wheel.
> Events are more flexible and can be implemented using interfaces.
> Has anyone ever actually used Observable outside of a GUI m/v
> environment?
Hmm, that would be me.
> Does anyone know of event frameworks suitable for
> monitoring long-running computations that we might adapt?
I must raise the issue again:
s/adapt/use/
> In any case, I think it is a good idea to develop an event framework
> for [math]. We should probably also think about doing this in a way
> that is at least JMX-friendly.
I don't know anything about JMX.
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Phil Steitz <ph...@gmail.com>.
On 8/8/11 11:52 AM, Sébastien Brisard wrote:
> Hi,
> a useful feature for *any* iterative algorithm would be the
> possibility to call a "monitor" at the end of each iteration. What
> would this monitor do would pretty much depend on the user's mood:
> periodically backup the state of the algorithm (in order to restart it
> in case of a crash), log a set of tabulated values (to plot the
> convergence of the algorithm), ...
> The idea would be to come up with a design as general as possible.
> Gilles proposed (see MATH-581) to use java.util.Observable. This
> sounds very attractive to me. Any comments?
+1 to the idea of using the Observer pattern; but -0 for
Observable. I would favor defining Events and Listeners because a)
Observable is concrete, so effectively forces you to create an
Observable delegate and b) the interface is crude and is really
designed for GUI scenarios where observers are holding references to
the Observable and just being notified that something has changed.
Events are more flexible and can be implemented using interfaces.
Has anyone ever actually used Observable outside of a GUI m/v
environment? Does anyone know of event frameworks suitable for
monitoring long-running computations that we might adapt?
In any case, I think it is a good idea to develop an event framework
for [math]. We should probably also think about doing this in a way
that is at least JMX-friendly.
Phil
> Best regards,
> Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Monitoring iterative algorithms
Posted by Greg Sterijevski <gs...@gmail.com>.
How about just serializing the object? -Greg
2011/8/8 Sébastien Brisard <se...@m4x.org>
> Hi,
> a useful feature for *any* iterative algorithm would be the
> possibility to call a "monitor" at the end of each iteration. What
> would this monitor do would pretty much depend on the user's mood:
> periodically backup the state of the algorithm (in order to restart it
> in case of a crash), log a set of tabulated values (to plot the
> convergence of the algorithm), ...
> The idea would be to come up with a design as general as possible.
> Gilles proposed (see MATH-581) to use java.util.Observable. This
> sounds very attractive to me. Any comments?
> Best regards,
> Sebastien
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>