You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Phil Steitz <ph...@gmail.com> on 2015/04/01 19:27:52 UTC
[math] Apachecon slides
Starting with Thomas and Gilles' tex setup from FOSDEM 2013, I have
started creating slides for my talk in a couple of weeks at
Apachecon NA [1]. The basic idea is to combine an overview and
quick examples with some discussion of the many design and
implementation challenges that we have faced. I have put the tex
sources on github [2] and would appreciate any comments or just PRs
there. There is a make file (thanks, Thomas!) that creates a pdf
from the sources and also has a clean target to clean up cruft. You
need to have beamer installed for the build to work.
The content looks like too much for 50 mins, but I can fly through
most of it, so don't be discouraged to provide more examples. I can
also swap out some of the ones that are there. I have also
obviously not made it look pretty yet. Patches of all kinds welcome
:) My goal is to have the slides complete and uploaded to LF by the
end of this weekend.
Thanks in advance for review / feedback. Its probably best to
provide the feedback on github. If we get into any code
discussions, we can take things back here.
Phil
[1] http://sched.co/2P9O
[2] https://github.com/psteitz/apachecon-2015-commons-math.git
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Apachecon slides
Posted by Phil Steitz <ph...@gmail.com>.
On 4/2/15 7:43 AM, Gilles wrote:
> Hi Phil.
>
> On Wed, 01 Apr 2015 10:27:52 -0700, Phil Steitz wrote:
>> Starting with Thomas and Gilles' tex setup from FOSDEM 2013, I have
>> started creating slides for my talk in a couple of weeks at
>> Apachecon NA [1]. The basic idea is to combine an overview and
>> quick examples with some discussion of the many design and
>> implementation challenges that we have faced. I have put the tex
>> sources on github [2] and would appreciate any comments or just PRs
>> there. There is a make file (thanks, Thomas!) that creates a pdf
>> from the sources and also has a clean target to clean up cruft. You
>> need to have beamer installed for the build to work.
>>
>> The content looks like too much for 50 mins, but I can fly through
>> most of it, so don't be discouraged to provide more examples. I can
>> also swap out some of the ones that are there. I have also
>> obviously not made it look pretty yet. Patches of all kinds welcome
>> :) My goal is to have the slides complete and uploaded to LF by the
>> end of this weekend.
>>
>> Thanks in advance for review / feedback. Its probably best to
>> provide the feedback on github. If we get into any code
>> discussions, we can take things back here.
>>
>> Phil
>> [1] http://sched.co/2P9O
>> [2] https://github.com/psteitz/apachecon-2015-commons-math.git
>
> Depending on your goal(s) and expected audience, you might want[1]
> to make it transparent that there are conflicting approaches to
> the development of Commons Math (sometimes along the lines of what
> you mention as "problems"). As an example, the statement
> ---
> Distribution classes also have sample() methods, but that makes
> them dependent on generators
> ---
> could be expanded to show that the problem _can_ be solved, that
> a patch exists, but that no consensus can be reached due to the
> non reconcilable opinions: stability of the (naming) API[2] vs
> evolving the API whenever such a problem is identified.
Exactly. The point of all of these "problems" slides is to call out
the challenges. I will make it very clear that there are not
obvious, uniformly agreed upon answers to some (actually most) of
them. I will also start with the disclaimer that what I am
presenting is one contributors' view.
>
> That was a concrete example; a more general aspect that leads
> to tension is that it is not easy or obvious to delimit the
> scope of Commons Math. Some like to add features, others fear
> that a feature may become orphan (when the contributor goes
> away) when it is realized that major refactoring is needed
> (example: "BOBYQAOptimizer").
++1 - the not yet created second slide on optimization was going to
mention this and I was also going to call it (abandonment) out as a
core challenge in the (not yet created) summary slide. Patches most
welcome describing BOBYQAOptimizer and its challenges.
>
> Even more pervasive is the less and less true statement of
> Commons Math being "state-of-the-art" (whether programming-like,
> library-like or performance-like): one aspect that you mention
> is indeed the inherently parallel nature of some algorithms
> and the decision to maintain codes that cannot benefit from the
> now ubiquitous multi-CPU machines (example: genetic algorithm).
> Another is the whether to (be allowed to) keep up with the Java
> language (also as a means to, perhaps, attract people who might
> take the challenge to make Commons Math state-of-the-art again).
That is what I meant by calling out the multi-threading decision
point. My personal opinion is that we will be better served making
our stuff easier to use in distributed execution environments than
trying to manage threads ourselves; but if I make that statement
(which I may not) I will make it clear it is just my opinion.
>
> It might also be interesting to update the status of the
> "FastMath" comparisons (also with Java 8).
I thought about that. Patches welcome if time exists. I don't have
time to do it myself right now and the presentation is going to be
pretty packed as it is.
Thanks for the feedback!
Phil
>
>
> Regards,
> Gilles
>
> [1] Or not...
> [2] Even though the top-level package is changed anyways (so that
> existing code is never broken anymore).
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Apachecon slides
Posted by Gilles <gi...@harfang.homelinux.org>.
Hi Phil.
On Wed, 01 Apr 2015 10:27:52 -0700, Phil Steitz wrote:
> Starting with Thomas and Gilles' tex setup from FOSDEM 2013, I have
> started creating slides for my talk in a couple of weeks at
> Apachecon NA [1]. The basic idea is to combine an overview and
> quick examples with some discussion of the many design and
> implementation challenges that we have faced. I have put the tex
> sources on github [2] and would appreciate any comments or just PRs
> there. There is a make file (thanks, Thomas!) that creates a pdf
> from the sources and also has a clean target to clean up cruft. You
> need to have beamer installed for the build to work.
>
> The content looks like too much for 50 mins, but I can fly through
> most of it, so don't be discouraged to provide more examples. I can
> also swap out some of the ones that are there. I have also
> obviously not made it look pretty yet. Patches of all kinds welcome
> :) My goal is to have the slides complete and uploaded to LF by the
> end of this weekend.
>
> Thanks in advance for review / feedback. Its probably best to
> provide the feedback on github. If we get into any code
> discussions, we can take things back here.
>
> Phil
> [1] http://sched.co/2P9O
> [2] https://github.com/psteitz/apachecon-2015-commons-math.git
Depending on your goal(s) and expected audience, you might want[1]
to make it transparent that there are conflicting approaches to
the development of Commons Math (sometimes along the lines of what
you mention as "problems"). As an example, the statement
---
Distribution classes also have sample() methods, but that makes
them dependent on generators
---
could be expanded to show that the problem _can_ be solved, that
a patch exists, but that no consensus can be reached due to the
non reconcilable opinions: stability of the (naming) API[2] vs
evolving the API whenever such a problem is identified.
That was a concrete example; a more general aspect that leads
to tension is that it is not easy or obvious to delimit the
scope of Commons Math. Some like to add features, others fear
that a feature may become orphan (when the contributor goes
away) when it is realized that major refactoring is needed
(example: "BOBYQAOptimizer").
Even more pervasive is the less and less true statement of
Commons Math being "state-of-the-art" (whether programming-like,
library-like or performance-like): one aspect that you mention
is indeed the inherently parallel nature of some algorithms
and the decision to maintain codes that cannot benefit from the
now ubiquitous multi-CPU machines (example: genetic algorithm).
Another is the whether to (be allowed to) keep up with the Java
language (also as a means to, perhaps, attract people who might
take the challenge to make Commons Math state-of-the-art again).
It might also be interesting to update the status of the
"FastMath" comparisons (also with Java 8).
Regards,
Gilles
[1] Or not...
[2] Even though the top-level package is changed anyways (so that
existing code is never broken anymore).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Apachecon slides
Posted by Phil Steitz <ph...@gmail.com>.
On 4/6/15 6:19 AM, Gilles wrote:
> Hi Phil.
>
> On Sun, 05 Apr 2015 13:25:02 -0700, Phil Steitz wrote:
>> Thanks for the feedback and contributions!
>>
>> I added a disclaimer making it clear that the challenges / lessons
>> learned stuff was just one contributor's view. The content is now
>> pretty much complete (unless someone has some awesome better example
>> I can swap in). I would appreciate a review of the completed
>> optimization slides and both the final challenges and lessons
>> learned slides. I will wait a few days to upload the slides so I
>> can clean them up some more and incorporate any feedback I get.
>>
>> Anyone who wants to review but does not want to fuss with
>> tex/beamer, shoot me a note and I will cut a pdf based on master
>> when I get the request.
>
> After this:
>
> ---CUT---
> public PAIR optimize(OptimizationData... optData)
> ---CUT---
>
> you could perhaps add one or two slides (to indicate the current
> state of affairs and what remains to be implemented for the next
> major release); something along the following lines:
>
> ---CUT---
> Decision: Refactor of all classes in "o.a.c.m.optim"
>
> * Phase out "OptimizationData"
> * Use "fluent" API
> * Separate optimization problem from optimization procedure
>
> New ideas (brought forward by our most recent committer, Evan Ward)
> experimented on least-squares optimizers:
> * LevenbergMarquardtOptimizer
> * GaussNewtonOptimizer
>
> New classes/interfaces in "o.a.c.m.fitting.leastsquares":
> * LeastSquaresProblem and LeastSquaresProblem.Evaluation
> * LeastSquaresOptimizer and LeastSquaresOptimizer.Optimum
> * LeastSquaresFactory
> * LeastSquaresBuilder
> * LevenbergMarquardtOptimizer
> * GaussNewtonOptimizer
> * AbstractEvaluation
> * ...
>
> Usage:
>
> final LeastSquaresProblem problem = LeastSquaresFactory.create( /*
> ... */ );
> final LeastSquaresOptimizer optim = new
> LevenbergMarquardtOptimizer( /* ... */ );
> final Optimum result = optim.optimize(problem);
>
> Yet to be done (before next major release?): Refactor all the
> other optimizers.
> ---CUT---
>
>
> Regards,
> Gilles
Thanks, Gilles!
I added two slides with only slight editing of the above.
Thanks again for reviewing.
Phil
>
>> Phil
>>
>> On 4/1/15 10:27 AM, Phil Steitz wrote:
>>> Starting with Thomas and Gilles' tex setup from FOSDEM 2013, I have
>>> started creating slides for my talk in a couple of weeks at
>>> Apachecon NA [1]. The basic idea is to combine an overview and
>>> quick examples with some discussion of the many design and
>>> implementation challenges that we have faced. I have put the tex
>>> sources on github [2] and would appreciate any comments or just PRs
>>> there. There is a make file (thanks, Thomas!) that creates a pdf
>>> from the sources and also has a clean target to clean up cruft.
>>> You
>>> need to have beamer installed for the build to work.
>>>
>>> The content looks like too much for 50 mins, but I can fly through
>>> most of it, so don't be discouraged to provide more examples. I
>>> can
>>> also swap out some of the ones that are there. I have also
>>> obviously not made it look pretty yet. Patches of all kinds
>>> welcome
>>> :) My goal is to have the slides complete and uploaded to LF by
>>> the
>>> end of this weekend.
>>>
>>> Thanks in advance for review / feedback. Its probably best to
>>> provide the feedback on github. If we get into any code
>>> discussions, we can take things back here.
>>>
>>> Phil
>>> [1] http://sched.co/2P9O
>>> [2] https://github.com/psteitz/apachecon-2015-commons-math.git
>>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Apachecon slides
Posted by Gilles <gi...@harfang.homelinux.org>.
Hi Phil.
On Sun, 05 Apr 2015 13:25:02 -0700, Phil Steitz wrote:
> Thanks for the feedback and contributions!
>
> I added a disclaimer making it clear that the challenges / lessons
> learned stuff was just one contributor's view. The content is now
> pretty much complete (unless someone has some awesome better example
> I can swap in). I would appreciate a review of the completed
> optimization slides and both the final challenges and lessons
> learned slides. I will wait a few days to upload the slides so I
> can clean them up some more and incorporate any feedback I get.
>
> Anyone who wants to review but does not want to fuss with
> tex/beamer, shoot me a note and I will cut a pdf based on master
> when I get the request.
After this:
---CUT---
public PAIR optimize(OptimizationData... optData)
---CUT---
you could perhaps add one or two slides (to indicate the current
state of affairs and what remains to be implemented for the next
major release); something along the following lines:
---CUT---
Decision: Refactor of all classes in "o.a.c.m.optim"
* Phase out "OptimizationData"
* Use "fluent" API
* Separate optimization problem from optimization procedure
New ideas (brought forward by our most recent committer, Evan Ward)
experimented on least-squares optimizers:
* LevenbergMarquardtOptimizer
* GaussNewtonOptimizer
New classes/interfaces in "o.a.c.m.fitting.leastsquares":
* LeastSquaresProblem and LeastSquaresProblem.Evaluation
* LeastSquaresOptimizer and LeastSquaresOptimizer.Optimum
* LeastSquaresFactory
* LeastSquaresBuilder
* LevenbergMarquardtOptimizer
* GaussNewtonOptimizer
* AbstractEvaluation
* ...
Usage:
final LeastSquaresProblem problem = LeastSquaresFactory.create( /* ...
*/ );
final LeastSquaresOptimizer optim = new LevenbergMarquardtOptimizer( /*
... */ );
final Optimum result = optim.optimize(problem);
Yet to be done (before next major release?): Refactor all the other
optimizers.
---CUT---
Regards,
Gilles
> Phil
>
> On 4/1/15 10:27 AM, Phil Steitz wrote:
>> Starting with Thomas and Gilles' tex setup from FOSDEM 2013, I have
>> started creating slides for my talk in a couple of weeks at
>> Apachecon NA [1]. The basic idea is to combine an overview and
>> quick examples with some discussion of the many design and
>> implementation challenges that we have faced. I have put the tex
>> sources on github [2] and would appreciate any comments or just PRs
>> there. There is a make file (thanks, Thomas!) that creates a pdf
>> from the sources and also has a clean target to clean up cruft. You
>> need to have beamer installed for the build to work.
>>
>> The content looks like too much for 50 mins, but I can fly through
>> most of it, so don't be discouraged to provide more examples. I can
>> also swap out some of the ones that are there. I have also
>> obviously not made it look pretty yet. Patches of all kinds welcome
>> :) My goal is to have the slides complete and uploaded to LF by the
>> end of this weekend.
>>
>> Thanks in advance for review / feedback. Its probably best to
>> provide the feedback on github. If we get into any code
>> discussions, we can take things back here.
>>
>> Phil
>> [1] http://sched.co/2P9O
>> [2] https://github.com/psteitz/apachecon-2015-commons-math.git
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Apachecon slides
Posted by Phil Steitz <ph...@gmail.com>.
Thanks for the feedback and contributions!
I added a disclaimer making it clear that the challenges / lessons
learned stuff was just one contributor's view. The content is now
pretty much complete (unless someone has some awesome better example
I can swap in). I would appreciate a review of the completed
optimization slides and both the final challenges and lessons
learned slides. I will wait a few days to upload the slides so I
can clean them up some more and incorporate any feedback I get.
Anyone who wants to review but does not want to fuss with
tex/beamer, shoot me a note and I will cut a pdf based on master
when I get the request.
Phil
On 4/1/15 10:27 AM, Phil Steitz wrote:
> Starting with Thomas and Gilles' tex setup from FOSDEM 2013, I have
> started creating slides for my talk in a couple of weeks at
> Apachecon NA [1]. The basic idea is to combine an overview and
> quick examples with some discussion of the many design and
> implementation challenges that we have faced. I have put the tex
> sources on github [2] and would appreciate any comments or just PRs
> there. There is a make file (thanks, Thomas!) that creates a pdf
> from the sources and also has a clean target to clean up cruft. You
> need to have beamer installed for the build to work.
>
> The content looks like too much for 50 mins, but I can fly through
> most of it, so don't be discouraged to provide more examples. I can
> also swap out some of the ones that are there. I have also
> obviously not made it look pretty yet. Patches of all kinds welcome
> :) My goal is to have the slides complete and uploaded to LF by the
> end of this weekend.
>
> Thanks in advance for review / feedback. Its probably best to
> provide the feedback on github. If we get into any code
> discussions, we can take things back here.
>
> Phil
> [1] http://sched.co/2P9O
> [2] https://github.com/psteitz/apachecon-2015-commons-math.git
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org