You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Sam Halliday <sa...@gmail.com> on 2009/05/14 12:18:20 UTC
commons-math, matrix-toolkits-java and consolidation
Dear all,
I am a maintainer of the matrix-toolkits-java
http://code.google.com/p/matrix-toolkits-java/
which is a comprehensive collection of matrix data structures, linear
solvers, least squares methods, eigenvalue and singular value
decompositions.
This note is in regard to the commons-math library. It is clear that our
projects dovetail, especially when I look at "linear" in version 2.0 of the
API. It would be good if we could either complement or consolidate efforts,
rather than reproduce.
It would be excellent if all the functionality of matrix-toolkits-java were
available as part of commons-math. There is already too much diversity and
un-maintained maths code out there for Java!
As a start, I'd like to discourage the use of a solid implementation for
SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
implementations based on the Templates project:-
http://www.netlib.org/templates
The reason is that the storage implementation should be related to the type
of data being stored. For example, there are many well-known kinds of sparse
matrix that are well suited to particular kinds of calculations... consider
multiplying sparse matrices that you know to be diagonal!
In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
thinking about linear algebra and have set up unrivalled standard APIs which
have been implemented right down to the architecture level. It would be a
major mistake if commons-math didn't build on their good work.
I believe commons-math should move to a netlib-java backend (allowing the
use of machine optimised BLAS/LAPACK).
http://code.google.com/p/netlib-java/
The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
scalability to parallel architectures which use Parallel BLAS/LAPACK. The
former should be possible with some work within the current API, but I fear
major API changes would be needed for the latter. I do not want the
commons-math API to walk into this trap without having first considered
future architectures! MTJ has a distributed package, but I am not sure if
this is something that is completely future proof either.
What say ye'?
--
Sam
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I've asked Bjorn about an Apache license for MTJ and his reply was
"Yes, I don't see why not. The more users/developers, the better."
Ted Dunning wrote:
>
> On Thu, May 14, 2009 at 1:54 PM, Sam Halliday wrote:
>> I personally have no problems with my MTJ contributions being released
>> Apache. Bjorn-Ove is the person to talk to about the bulk of MTJ. I'll
>> ask
>> him!
>>
>
> Great.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23574199.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
On Thu, May 14, 2009 at 1:54 PM, Sam Halliday <sa...@gmail.com>wrote:
> I personally have no problems with my MTJ contributions being released
> Apache. Bjorn-Ove is the person to talk to about the bulk of MTJ. I'll ask
> him!
>
Great.
MTJ depends on netlib-java, which is technically a translation of the
> original netlib libraries. They are BSD license. I seriously doubt you'll
> get them to give you the right to redistribute as, so you'll have to decide
> if that's a blocker.
>
If they really are BSD, then there should be no problem. BSD allows
redistribution with attribution, preservation of the copyright notice and no
implication of endorsement.
> What would "adopting wholesale" mean? It would be a good opportunity to
> review/revise parts of the API and find duplication with the rest of the
> commons-math project.
>
There is a big issue with dependencies, but a much smaller issue with major
source code contributions. Essentially what I mean by "adopting wholesale"
would be for commons math to ingest MTJ. IN the best world, the contributor
communities would merge as well. There would still be plenty of issues such
as the conditional dependency on native libraries. I am not sure how that
should play out.
--
Ted Dunning, CTO
DeepDyve
111 West Evelyn Ave. Ste. 202
Sunnyvale, CA 94086
www.deepdyve.com
858-414-0013 (m)
408-773-0220 (fax)
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Replies inline:
Ted Dunning wrote:
>
>> As a start, I'd like to discourage the use of a solid implementation for
>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>> allowing
>> implementations based on the Templates project:-
>
> Can you say more about what aspects of the Templates project you feel are
> important? You mention one case of storage layout.
>
It's difficult to say which algorithms from Templates ones are the most
important ones, but in most cases reference implementations already exist
(usually in fortran) and should be preferred (e.g. by using f2j with a
wrapper layer): the theory can be quite involved. MTJ only touches the
surface! However, an important step is recognising that there are not just
"dense" and "sparse" matrices... but whole classes of structured sparse
matrices.
Ted Dunning wrote:
>
>> I believe commons-math should move to a netlib-java backend (allowing the
>> use of machine optimised BLAS/LAPACK).
>
> This is an interesting suggestion. Obviously adopting MTJ wholesale would
> accomplish that.
>
> Can you say something about the licensing issues if we were to explore,
> for
> discussion sake, MTJ being folded into commons-math? MTJ is LGPL while
> commons has to stay Apache licensed. This licensing issue has been the
> biggest sticking point in the past.
>
I personally have no problems with my MTJ contributions being released
Apache. Bjorn-Ove is the person to talk to about the bulk of MTJ. I'll ask
him!
MTJ depends on netlib-java, which is technically a translation of the
original netlib libraries. They are BSD license. I seriously doubt you'll
get them to give you the right to redistribute as, so you'll have to decide
if that's a blocker.
What would "adopting wholesale" mean? It would be a good opportunity to
review/revise parts of the API and find duplication with the rest of the
commons-math project.
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23548814.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Luc Maisonobe wrote:
>
> Ted Dunning a écrit :
>> As a start, I'd like to discourage the use of a solid implementation for
>>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>>> allowing
>>> implementations based on the Templates project:-
>
> Maybe SparseRealMatrix was a bad name and should have been
> SimpleSparseRealMatrix to avoid confusion with other sparse storage and
> dedicated algorithms.
>
I give a +1 for renaming SparseReal{Matrix, Vector}! These names should be
reserved for interfaces (which might be method-less) indicating that the
implementation storage needs to be sparse.
Luc Maisonobe wrote:
>
>> Can you say something about the licensing issues if we were to explore,
>> for
>> discussion sake, MTJ being folded into commons-math? MTJ is LGPL while
>> commons has to stay Apache licensed. This licensing issue has been the
>> biggest sticking point in the past.
>
> This is really an issue. Apache projects cannot use LGPL (or GPL) code.
> See http://www.apache.org/legal/resolved.html for the policy.
>
Solved! See other message. Both myself and (more importantly, because he
wrote MTJ) Bjorn are willing to use Apache license.
Luc Maisonobe wrote:
>
> Adding new dependencies, and especially dependencies that involve native
> libraries is a difficult decision that needs lots of discussion.
>
MTJ depends only on netlib-java, *which does not depend on any native libs*.
The option is there to add native optimised libs if the end user wants to.
Luc Maisonobe wrote:
>
> Some benchmarks I did a few weeks ago showed the new [math] linear
> package implementation was quite fast and compared very well with native
> fortran libraries
>
I'm going to call "foul" here :-)
The Java implementation of netlib-java is just as fast as machine-optimised
BLAS/LAPACK... but only for matrices smaller than roughly 1000 x 1000
elements AND ONLY FOR NORMAL DESKTOPS MACHINES! The important distinction
here is that hardware exists with crazy optimisations for the BLAS/LAPACK
API and having the option to use that architecture from within Java is a
great bonus. Consider, for example, a dedicated GPU (or FPGA) card which
comes with a BLAS/LAPACK binary.
Additionally, the BLAS/LAPACK API is universally accepted. It would be a
mistake to attempt to reproduce all the brain power and agreement that has
worked toward it.
Luc Maisonobe wrote:
>
> I am aware that we still lack lots of very efficient linear algebra
> algorithms. Joining efforts with you would be a real gain if we can
> solve the licensing issues and avoid new dependencies if possible.
>
I am very keen to consolidate efforts! I think the next step is perhaps for
you to have a look through the MTJ API and create a wish-list of everything
you think would make sense to appear in commons-math. Even if adopted
"wholesale", I would still strongly recommend a review of the API. e.g. some
interfaces extend Serializable (a mistake) ; I'm not entirely sure how
relevant the distributed package is nowadays; the Matrix Market IO is
difficult to understand/use ; there should perhaps be a "factory pattern" to
instantiating matrices/vectors.
In the meantime, I recommend holding off a 2.0 API release with any new
linear classes. That way we can stabilise the "new" merged API... releasing
that as part of 2.1.
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23574363.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> I suspected as much: that makes JAMA and Colt both deceased. Does anyone know
> how to contact the JAMA authors and ask them to make this explicit in the
> webpage? Asking to direct users to commons-math would be wise.
This fact appears at least here in this message:
http://cio.nist.gov/esd/emaildir/lists/jama/msg01384.html
Note that the project cited in this message also already uses
commons-math and other mathematical libraries too.
>
> I have attempted to contact the author/maintainer of Colt on many occasions,
> but I keep getting bounced mail. It would be good to be able to warn people
> away from using these dead projects. Do you know of any others?
At least mantissa is in the same situation, but the link with
commons-math is already in the small FAQ.
I would personally not try to urge people to switch to our library. They
may well be happy with the status of these projects and switching has a
time cost for adaptation to new API and validation.
>
>
> Luc Maisonobe wrote:
>> JAMA is a deceased project.
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I suspected as much: that makes JAMA and Colt both deceased. Does anyone know
how to contact the JAMA authors and ask them to make this explicit in the
webpage? Asking to direct users to commons-math would be wise.
I have attempted to contact the author/maintainer of Colt on many occasions,
but I keep getting bounced mail. It would be good to be able to warn people
away from using these dead projects. Do you know of any others?
Luc Maisonobe wrote:
>
> JAMA is a deceased project.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23575637.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> Just to let you know, I've contacted the author of this blog post... who has
> recently written a library called jblas. I've asked him if he wants to be
> involved with the initiative here, to consolidate efforts for Java Linear
> Algebra packages.
>
> Incidentally... this blog post references a very pervasive, yet abandoned,
> project named Colt. Colt was a brilliant library in its day (now numerically
> challenged), although riddled with license issues (depending on
> non-commercial and ill-defined not-for-military-use middleware). Colt is a
> reminder of what can happen when a great library is written but not
> maintained. There might be lessons to learn from their API... I know some
> projects that use it.
>
> It might be worthwhile contacting other Java Linear Algebra package authors,
> such as JAMA. JAMA is a very small library in comparison (no additional
> functionality over MTJ or commons-math)... but they might have a different
> take on APIs than we would have.
JAMA is a deceased project. We have based our API for decomposition
algorithms on the JAMA API, with some extensions we thought were useful
(see the class javadoc for the various decomposition interfaces). For
example, we have added interfaces instead of using only classes and we
have added solvers that could avoid building some large intermediate
matrices, thus greatly improving speed.
We have compared our results to JAMA. Here are some results I just rerun
for this message, with dimensions between 600 and 1000. Once again,
beware of the single case, single hardware, single trial effect: the
bench is most probably biased due to these characteristics.
multiplication: about 1.5 times faster with DenseRealMatrix
QR decomposition + solve: about 10 times faster at dimension 600, about
20-25 times faster at dimension 1000
eigen decomposition: about 3 times faster at dimension 600, about 6
times faster at dimension 1000
These results could have been expected: JAMA focused on the API, not on
the implementation and almost naive implementations have been used with
no consideration to the memory handling. So clearly, JAMA is superseded
on both the API, maintenance and efficiency aspects.
See for example these messages:
http://markmail.org/message/a4d454alexugur2m
http://markmail.org/message/h2rmq6wo7nwqzx4w
http://markmail.org/message/l4cvopambgumrwbs
Luc
>
>
> Ted Dunning wrote:
>> http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Just to let you know, I've contacted the author of this blog post... who has
recently written a library called jblas. I've asked him if he wants to be
involved with the initiative here, to consolidate efforts for Java Linear
Algebra packages.
Incidentally... this blog post references a very pervasive, yet abandoned,
project named Colt. Colt was a brilliant library in its day (now numerically
challenged), although riddled with license issues (depending on
non-commercial and ill-defined not-for-military-use middleware). Colt is a
reminder of what can happen when a great library is written but not
maintained. There might be lessons to learn from their API... I know some
projects that use it.
It might be worthwhile contacting other Java Linear Algebra package authors,
such as JAMA. JAMA is a very small library in comparison (no additional
functionality over MTJ or commons-math)... but they might have a different
take on APIs than we would have.
Ted Dunning wrote:
>
> http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23574736.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Ted, thanks for pointing this out... I'd never seen it before. Glad MTJ did
so well and I note that this isn't even with the optional native BLAS/LAPACK
:-)
Ted Dunning wrote:
>
> http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23574399.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Phil Steitz a écrit :
>
>> Ted Dunning wrote:
>>
>>> Phil, I think we have much of the same desires and motivations, but we
>>> seem
>>> to come to somewhat, but not entirely different conclusions.
>>>
>>> Assuming that (1) can be dealt with and assuming that (3) is already
>>> dealt
>>> with, do you still mind the inclusion of *optional*, automatically
>>> generated
>>> native code?
>>>
>>>
>> Part of 3) is having full code available with the package for
>> inspection. That is part of the reason that we have avoided external
>> dependencies. I would be open to making our fully self-contained, fully
>> documented, fully open source library extensible to use other libraries,
>> including native libraries, but I would not want to distribute anything
>> associated with external libraries. The reason for this is the
>> commitment made early on that all numerics and algorithms would be
>> immediately visible to the user - no chasing down external, possibly
>> incomplete or ambiguous docs to figure out what our code is doing.
>>
>
> I have an additional reason for avoiding native libraries. Pure Java can
> be processed by external tools for either inspection (think findbugs,
> cobertura, traceability, auditing) or modification (think Nabla!). The
> Nabla case is especially important to me, but I am aware this is a
> corner-case.
>
>
>>> This page has some useful speed comparisons. For matrix-matrix
>>> multiply up
>>> to size 50, java is competitive. If you get up to roughly n = 500 or
>>> 1000,
>>> then heavily optimized native code can be up to 3x faster. Note the
>>> line in
>>> the third graph for colt (a reasonably well written pure java
>>> implementation) and MTJ (which is running in pure java mode here). In my
>>> case, I will generally opt for portability, but I like to have a portable
>>> option for speed. It is also important to remember that numerical codes
>>> more often need blinding speed than most other applications.
>>>
>>>
>> As Luc said, commons-math aims to be a general-purpose applied math
>> package implementing good, well-documented, unencumbered numerical
>> algorithms. I think this can be done in Java and we are doing it. We
>> are never going to compete with optimized native code in speed, but
>> strong numerics, JRE improvements and Moore's law are rapidly shrinking
>> the class of real-world applications where the 3x difference above is
>> material.
>>
>
> Perhaps we should have some benchmarks including our new linear package.
> Something more serious than my little experiement with QR decomposition.
> Unfortunately, I clearly have no time for it now. My current priotity is
> to publish 2.0 as soon as possible and I am already late on my own schedule.
>
+1 for getting 2.0 released ASAP. This is long overdue and we need to
stay focussed on getting it out.
Phil
> Luc
>
>
>> Phil
>>
>>
>>
>>> http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
>>>
>>> Is that optional dependency really all that bad?
>>>
>>> On Fri, May 15, 2009 at 6:23 PM, Phil Steitz <ph...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>>> [math] also has currently zero dependencies. We had two dependencies on
>>>>
>>>>
>>>>> other commons components up to version 1.2 and removed them when we
>>>>> started work on version 2.0. Adding new dependencies, and especially
>>>>> dependencies that involve native libraries is a difficult decision that
>>>>> needs lots of discussion. We are currently trying to have 2.0 published
>>>>> very soon now, such a decision would delay the publication several
>>>>> months.
>>>>>
>>>>>
>>>>>
>>>>>
>>>> -1 for adding dependencies, especially on native code. Commons math
>>>> needs
>>>> to remain
>>>>
>>>> 1) ASL licensed
>>>> 2) self-contained
>>>> 3) fully documented, full open source
>>>>
>>>>
>>>
>>>
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
The output of f2j is almost unusable for normal mortals. There will
definitely be a wrapper layer.
Also, there is more than a wrapper in MTJ. There are also higher order
algorithms that use the lower level linear algebra provided by f2j.
On Tue, May 19, 2009 at 2:59 AM, Jin Mingjian <ji...@gmail.com> wrote:
> Can I ask a question? how about MTJ vs pure F2J? (If I don't use the native
> part of MTJ.)
> Can I get more must-have things? is it necessary to introduce a wrapper
> layer?
> Best regards,
> Jin
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
-1 on declaring things Serializable, especially if this is done everywhere.
Unless there is extensive testing and careful implementation, it is very,
very misleading to advertise this capability. In addition, there are a
number of formats which are reasonable candidates. Casting a particular
format in stone as the one true serialized format seems like a big mistake.
If you need to implement a serializable object, it is much better to mark
the field as transient and pick the wire format you really want to use from
a selection of alternatives.
On Thu, May 21, 2009 at 5:47 AM, sebb <se...@gmail.com> wrote:
> Merely implementing Serializable will prevent the warnings, however in
> general it's not trivial to ensure that serialization works correctly.
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Jin Mingjian <ji...@gmail.com>.
Can I ask a question? how about MTJ vs pure F2J? (If I don't use the native
part of MTJ.)
Can I get more must-have things? is it necessary to introduce a wrapper
layer?
Best regards,
Jin
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by James Carman <ja...@carmanconsulting.com>.
On Thu, May 21, 2009 at 11:31 AM, Sam Halliday <sa...@gmail.com> wrote:
>
> Luc... couldn't agree more regarding Serializable. Adding the Serializable
> interface instantly means you not only have to be API compatible with future
> releases but also binary Serializable compatible. This is what stung MTJ...
> it means you can't swap internal details of fields.
>
> I strongly recommend everybody read the Bloch chapters on Serialisation
> before ever implementing that interface.
I've always found that if you're going to do it, it's better to do
Externalizable (if you anticipate the stuff getting saved in long-term
storage and not just in remoting situations) and make sure you include
a version number in the output so you can handle different
serialization formats. I have no idea what Bloch said, that's just my
$0.02 (based on today's exchange rates of course).
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Oh.... and when something implements Serializable, the serial form must be
either documented in the Javadocs or the Javadocs must be quite clear and
say that the serial form might change in future releases. Either way, don't
forget serialVersionUID and to update it if the serial form ever changes.
Sam Halliday wrote:
>
> Absolutely remove from the interfaces... but leave on implementations :-)
> That way all future implementations aren't forced to be compatible to
> their first release.
>
> Hopefully we'll address transport of instances in 2.1 with Matrix Market
> IO, which should also encourage more efficient compression.
>
>
> Luc Maisonobe wrote:
>>
>> Sam Halliday a écrit :
>>> Luc... couldn't agree more regarding Serializable. Adding the
>>> Serializable
>>> interface instantly means you not only have to be API compatible with
>>> future
>>> releases but also binary Serializable compatible. This is what stung
>>> MTJ...
>>> it means you can't swap internal details of fields.
>>>
>>> I strongly recommend everybody read the Bloch chapters on Serialisation
>>> before ever implementing that interface.
>>
>> OK. What do you suggest now ? Completely remove Serializable (which
>> would really bother me as I do use it) or remove it from interfaces and
>> add it appropriately to some implementation classes ?
>>
>> Luc
>>
>>>
>>>
>>> sebb-2-2 wrote:
>>>> On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
>>>> Nevertheless, adding serialization needs to be considered carefully,
>>>> as pointed out here:
>>>>
>>>> http://www.javapractices.com/topic/TopicAction.do?Id=45
>>>>
>>>> Furthermore, readObject() cannot be used with final fields. Bloch
>>>> (Effective Java) suggests using readResolve() instead, but even this
>>>> has limitations.
>>>>
>>>> As the book points out, merely making a class Serializable is
>>>> equivalent to adding a public constructor which sets all the
>>>> non-transient fields without perfoming any validation.
>>>>
>>>> If there are any constraints on the fields, these have to be addressed
>>>> in readObject() and/or readResolve() methods.
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>>
>
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23656955.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Absolutely remove from the interfaces... but leave on implementations :-)
That way all future implementations aren't forced to be compatible to their
first release.
Hopefully we'll address transport of instances in 2.1 with Matrix Market IO,
which should also encourage more efficient compression.
Luc Maisonobe wrote:
>
> Sam Halliday a écrit :
>> Luc... couldn't agree more regarding Serializable. Adding the
>> Serializable
>> interface instantly means you not only have to be API compatible with
>> future
>> releases but also binary Serializable compatible. This is what stung
>> MTJ...
>> it means you can't swap internal details of fields.
>>
>> I strongly recommend everybody read the Bloch chapters on Serialisation
>> before ever implementing that interface.
>
> OK. What do you suggest now ? Completely remove Serializable (which
> would really bother me as I do use it) or remove it from interfaces and
> add it appropriately to some implementation classes ?
>
> Luc
>
>>
>>
>> sebb-2-2 wrote:
>>> On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
>>> Nevertheless, adding serialization needs to be considered carefully,
>>> as pointed out here:
>>>
>>> http://www.javapractices.com/topic/TopicAction.do?Id=45
>>>
>>> Furthermore, readObject() cannot be used with final fields. Bloch
>>> (Effective Java) suggests using readResolve() instead, but even this
>>> has limitations.
>>>
>>> As the book points out, merely making a class Serializable is
>>> equivalent to adding a public constructor which sets all the
>>> non-transient fields without perfoming any validation.
>>>
>>> If there are any constraints on the fields, these have to be addressed
>>> in readObject() and/or readResolve() methods.
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23656891.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> Luc... couldn't agree more regarding Serializable. Adding the Serializable
> interface instantly means you not only have to be API compatible with future
> releases but also binary Serializable compatible. This is what stung MTJ...
> it means you can't swap internal details of fields.
>
> I strongly recommend everybody read the Bloch chapters on Serialisation
> before ever implementing that interface.
OK. What do you suggest now ? Completely remove Serializable (which
would really bother me as I do use it) or remove it from interfaces and
add it appropriately to some implementation classes ?
Luc
>
>
> sebb-2-2 wrote:
>> On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
>> Nevertheless, adding serialization needs to be considered carefully,
>> as pointed out here:
>>
>> http://www.javapractices.com/topic/TopicAction.do?Id=45
>>
>> Furthermore, readObject() cannot be used with final fields. Bloch
>> (Effective Java) suggests using readResolve() instead, but even this
>> has limitations.
>>
>> As the book points out, merely making a class Serializable is
>> equivalent to adding a public constructor which sets all the
>> non-transient fields without perfoming any validation.
>>
>> If there are any constraints on the fields, these have to be addressed
>> in readObject() and/or readResolve() methods.
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Luc... couldn't agree more regarding Serializable. Adding the Serializable
interface instantly means you not only have to be API compatible with future
releases but also binary Serializable compatible. This is what stung MTJ...
it means you can't swap internal details of fields.
I strongly recommend everybody read the Bloch chapters on Serialisation
before ever implementing that interface.
sebb-2-2 wrote:
>
> On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
> Nevertheless, adding serialization needs to be considered carefully,
> as pointed out here:
>
> http://www.javapractices.com/topic/TopicAction.do?Id=45
>
> Furthermore, readObject() cannot be used with final fields. Bloch
> (Effective Java) suggests using readResolve() instead, but even this
> has limitations.
>
> As the book points out, merely making a class Serializable is
> equivalent to adding a public constructor which sets all the
> non-transient fields without perfoming any validation.
>
> If there are any constraints on the fields, these have to be addressed
> in readObject() and/or readResolve() methods.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23655263.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by sebb <se...@gmail.com>.
On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
> sebb a écrit :
>
> > On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
> >> Sam Halliday a écrit :
> >>
> >>> Bill, I've had a look at some of the recent changes... some comments,
> >> > including some new thoughts:-
> >> >
> >> > - changing the return type to be actual classes was only supposed to be for
> >> > copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
> >> > is no guarantee that is the best thing to do and it is likely this will
> >> > change in the future. This stands for all implementations. Great thought
> >> > needs to go into each method, not just by reading the current
> >> > implementation.
> >>
> >>
> >> It may help the compiler using directly the optimized versions in
> >> statements like:
> >>
> >> m1.add(m2.multiply(m3.subtract(m4))
> >>
> >>
> >> >
> >> > - I *strongly* urge you to remove Serializable from everything! Please, we
> >> > did this in MTJ and it turned out to be a major pain. A more appropriate
> >> > approach is to define a class for reading/writing Matrix Market files. This
> >> > can be a new feature in 2.1. If you're going to leave it, at least document
> >> > that the Serializable form is not guaranteed to remain compatible across
> >> > versions.
> >>
> >>
> >> I disagree with and I think its me who started adding this interface
> >> everywhere. The rationale is explained in the following use case:
> >>
> >> As a user, I want to implement an interface for some xyz library where
> >> the interface already extends Serializable. In my implementation, I want
> >> to have an instance field that is some RealMatrix. If RealMatrix does
> >> not declare it extends Serializable, I keep getting warnings about
> >> having a non-serializable field in a serializable class. Looking the
> >> other way round having serializable and not using it never caused me any
> >> pain because it was really a generalized pattern. Mixing serializable
> >> and non serializable is difficult, putting it everywhere and sticking to
> >> this policy is simple.
> >
> > One can use the "transient" marker for non-serializable fields.
>
>
> Transient is OK when the field state can be reconstructed on
> deserialization from other fields information. This is quite rare and is
> surely not the case with matrices which may be huge and therefore not
> duplicated in several different fields.
>
>
> >
> > Merely implementing Serializable will prevent the warnings, however in
> > general it's not trivial to ensure that serialization works correctly.
>
>
> This is probably simpler in math objects (except graph theory perhaps)
> than in other business domains. Our objects typically hold a primitive
> and arrays and some other lower level objects with similar simple
> structures.
>
Nevertheless, adding serialization needs to be considered carefully,
as pointed out here:
http://www.javapractices.com/topic/TopicAction.do?Id=45
Furthermore, readObject() cannot be used with final fields. Bloch
(Effective Java) suggests using readResolve() instead, but even this
has limitations.
As the book points out, merely making a class Serializable is
equivalent to adding a public constructor which sets all the
non-transient fields without perfoming any validation.
If there are any constraints on the fields, these have to be addressed
in readObject() and/or readResolve() methods.
> Luc
>
>
> >
> >> What kind of pain did Serializable cause in MTJ ?
> >>
> >> BTW, I'm really +1 to Matrix Market files, could you open a Jira issue
> >> asking for this enhancement for 2.1 ?
> >>
> >>
> >> >
> >> > - I discourage the use of the classes named *Impl. They will get very
> >> > confusing when other implementations are added later! Instead, I recommend
> >> > the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
> >> > backed implementation using row ordering). This allows a column-based or 1D
> >> > implementation in the future without names getting very confusing. These
> >> > implementations are hidden from users who just use the MatrixUtils help
> >>
> >>
> >> Phil suggested to change RealMatrixImpl to ArrayRealMatrix (and
> >> DenseRealMatrix to BlockRealMatrix). This sounds good to me.
> >>
> >>
> >> >
> >> > - -1 for inclusion of methods in the sparse interfaces and shape type enum.
> >> > This needs more thought and discussion before inclusion. I am happy enough
> >> > for marker interfaces to be included (as long as it is documented that these
> >> > interfaces are subject to update in the next release), but strongly against
> >> > including methods in them.
> >>
> >>
> >> +1 to further thoughts and discussion and postponing declaring methods
> >> in the interfaces after 2.0. Lets start with empty marker interfaces.
> >>
> >>
> >> Luc
> >>
> >>
> >> >
> >> > - could you please describe the hashing algorithm that OpenMap is using? I
> >> > have previously written a GNU Trove based sparse matrix implementation which
> >> > was a map from the primitive "long" to "double". It looks analagous. I broke
> >> > the row/column index into a long for the key using binary operations, but
> >> > the hashcode was tricky as it had to be an integer (not long). A naive
> >> > implementation can lead to collisions for typical or symmetric matrices. It
> >> > looks like the current implementation is using an int -> double backing map!
> >> > Wouldn't it make more sense to use a long?
> >> >
> >> > The following code might be useful to you (don't forget the L marker and
> >> > casts or the compiler will silently cast to an int), there are two options
> >> > for a hashcode calculator... I've not stress tested the second one but it
> >> > might be more appropriate as it gives more weight to lower bits. Remember
> >> > that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16 hex
> >> > characters in Java bitwise.
> >> >
> >> > int hash1(long val) {
> >> > int first = Integer.reverse(key2row(val));
> >> > int second = key2col(val);
> >> > return first + second;
> >> > }
> >> >
> >> > int hash2(long val) {
> >> > int first = key2row(val) & 0x0000FFFF;
> >> > int second = key2col(val) << 16;
> >> > return first + second;
> >> > }
> >> >
> >> > long key(int row, int column) {
> >> > return (((long) row) << 32) + (long) column;
> >> > }
> >> >
> >> > int key2col(long key) {
> >> > return (int) (key & 0x00000000FFFFFFFFL);
> >> > }
> >> >
> >> > int key2row(long key) {
> >> > return (int) (key >>> 32);
> >> > }
> >> >
> >> >
> >> > Bill Barker wrote:
> >> >> I have a slight preference for leaving the markers empty until 3.0, but I
> >> >> can remove them as well. But I can wait to see what the community
> >> >> consensus
> >> >> is before making changes.
> >> >>
> >> >
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
sebb a écrit :
> On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
>> Sam Halliday a écrit :
>>
>>> Bill, I've had a look at some of the recent changes... some comments,
>> > including some new thoughts:-
>> >
>> > - changing the return type to be actual classes was only supposed to be for
>> > copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
>> > is no guarantee that is the best thing to do and it is likely this will
>> > change in the future. This stands for all implementations. Great thought
>> > needs to go into each method, not just by reading the current
>> > implementation.
>>
>>
>> It may help the compiler using directly the optimized versions in
>> statements like:
>>
>> m1.add(m2.multiply(m3.subtract(m4))
>>
>>
>> >
>> > - I *strongly* urge you to remove Serializable from everything! Please, we
>> > did this in MTJ and it turned out to be a major pain. A more appropriate
>> > approach is to define a class for reading/writing Matrix Market files. This
>> > can be a new feature in 2.1. If you're going to leave it, at least document
>> > that the Serializable form is not guaranteed to remain compatible across
>> > versions.
>>
>>
>> I disagree with and I think its me who started adding this interface
>> everywhere. The rationale is explained in the following use case:
>>
>> As a user, I want to implement an interface for some xyz library where
>> the interface already extends Serializable. In my implementation, I want
>> to have an instance field that is some RealMatrix. If RealMatrix does
>> not declare it extends Serializable, I keep getting warnings about
>> having a non-serializable field in a serializable class. Looking the
>> other way round having serializable and not using it never caused me any
>> pain because it was really a generalized pattern. Mixing serializable
>> and non serializable is difficult, putting it everywhere and sticking to
>> this policy is simple.
>
> One can use the "transient" marker for non-serializable fields.
Transient is OK when the field state can be reconstructed on
deserialization from other fields information. This is quite rare and is
surely not the case with matrices which may be huge and therefore not
duplicated in several different fields.
>
> Merely implementing Serializable will prevent the warnings, however in
> general it's not trivial to ensure that serialization works correctly.
This is probably simpler in math objects (except graph theory perhaps)
than in other business domains. Our objects typically hold a primitive
and arrays and some other lower level objects with similar simple
structures.
Luc
>
>> What kind of pain did Serializable cause in MTJ ?
>>
>> BTW, I'm really +1 to Matrix Market files, could you open a Jira issue
>> asking for this enhancement for 2.1 ?
>>
>>
>> >
>> > - I discourage the use of the classes named *Impl. They will get very
>> > confusing when other implementations are added later! Instead, I recommend
>> > the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
>> > backed implementation using row ordering). This allows a column-based or 1D
>> > implementation in the future without names getting very confusing. These
>> > implementations are hidden from users who just use the MatrixUtils help
>>
>>
>> Phil suggested to change RealMatrixImpl to ArrayRealMatrix (and
>> DenseRealMatrix to BlockRealMatrix). This sounds good to me.
>>
>>
>> >
>> > - -1 for inclusion of methods in the sparse interfaces and shape type enum.
>> > This needs more thought and discussion before inclusion. I am happy enough
>> > for marker interfaces to be included (as long as it is documented that these
>> > interfaces are subject to update in the next release), but strongly against
>> > including methods in them.
>>
>>
>> +1 to further thoughts and discussion and postponing declaring methods
>> in the interfaces after 2.0. Lets start with empty marker interfaces.
>>
>>
>> Luc
>>
>>
>> >
>> > - could you please describe the hashing algorithm that OpenMap is using? I
>> > have previously written a GNU Trove based sparse matrix implementation which
>> > was a map from the primitive "long" to "double". It looks analagous. I broke
>> > the row/column index into a long for the key using binary operations, but
>> > the hashcode was tricky as it had to be an integer (not long). A naive
>> > implementation can lead to collisions for typical or symmetric matrices. It
>> > looks like the current implementation is using an int -> double backing map!
>> > Wouldn't it make more sense to use a long?
>> >
>> > The following code might be useful to you (don't forget the L marker and
>> > casts or the compiler will silently cast to an int), there are two options
>> > for a hashcode calculator... I've not stress tested the second one but it
>> > might be more appropriate as it gives more weight to lower bits. Remember
>> > that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16 hex
>> > characters in Java bitwise.
>> >
>> > int hash1(long val) {
>> > int first = Integer.reverse(key2row(val));
>> > int second = key2col(val);
>> > return first + second;
>> > }
>> >
>> > int hash2(long val) {
>> > int first = key2row(val) & 0x0000FFFF;
>> > int second = key2col(val) << 16;
>> > return first + second;
>> > }
>> >
>> > long key(int row, int column) {
>> > return (((long) row) << 32) + (long) column;
>> > }
>> >
>> > int key2col(long key) {
>> > return (int) (key & 0x00000000FFFFFFFFL);
>> > }
>> >
>> > int key2row(long key) {
>> > return (int) (key >>> 32);
>> > }
>> >
>> >
>> > Bill Barker wrote:
>> >> I have a slight preference for leaving the markers empty until 3.0, but I
>> >> can remove them as well. But I can wait to see what the community
>> >> consensus
>> >> is before making changes.
>> >>
>> >
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by sebb <se...@gmail.com>.
On 21/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
> Sam Halliday a écrit :
>
> > Bill, I've had a look at some of the recent changes... some comments,
> > including some new thoughts:-
> >
> > - changing the return type to be actual classes was only supposed to be for
> > copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
> > is no guarantee that is the best thing to do and it is likely this will
> > change in the future. This stands for all implementations. Great thought
> > needs to go into each method, not just by reading the current
> > implementation.
>
>
> It may help the compiler using directly the optimized versions in
> statements like:
>
> m1.add(m2.multiply(m3.subtract(m4))
>
>
> >
> > - I *strongly* urge you to remove Serializable from everything! Please, we
> > did this in MTJ and it turned out to be a major pain. A more appropriate
> > approach is to define a class for reading/writing Matrix Market files. This
> > can be a new feature in 2.1. If you're going to leave it, at least document
> > that the Serializable form is not guaranteed to remain compatible across
> > versions.
>
>
> I disagree with and I think its me who started adding this interface
> everywhere. The rationale is explained in the following use case:
>
> As a user, I want to implement an interface for some xyz library where
> the interface already extends Serializable. In my implementation, I want
> to have an instance field that is some RealMatrix. If RealMatrix does
> not declare it extends Serializable, I keep getting warnings about
> having a non-serializable field in a serializable class. Looking the
> other way round having serializable and not using it never caused me any
> pain because it was really a generalized pattern. Mixing serializable
> and non serializable is difficult, putting it everywhere and sticking to
> this policy is simple.
One can use the "transient" marker for non-serializable fields.
Merely implementing Serializable will prevent the warnings, however in
general it's not trivial to ensure that serialization works correctly.
> What kind of pain did Serializable cause in MTJ ?
>
> BTW, I'm really +1 to Matrix Market files, could you open a Jira issue
> asking for this enhancement for 2.1 ?
>
>
> >
> > - I discourage the use of the classes named *Impl. They will get very
> > confusing when other implementations are added later! Instead, I recommend
> > the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
> > backed implementation using row ordering). This allows a column-based or 1D
> > implementation in the future without names getting very confusing. These
> > implementations are hidden from users who just use the MatrixUtils help
>
>
> Phil suggested to change RealMatrixImpl to ArrayRealMatrix (and
> DenseRealMatrix to BlockRealMatrix). This sounds good to me.
>
>
> >
> > - -1 for inclusion of methods in the sparse interfaces and shape type enum.
> > This needs more thought and discussion before inclusion. I am happy enough
> > for marker interfaces to be included (as long as it is documented that these
> > interfaces are subject to update in the next release), but strongly against
> > including methods in them.
>
>
> +1 to further thoughts and discussion and postponing declaring methods
> in the interfaces after 2.0. Lets start with empty marker interfaces.
>
>
> Luc
>
>
> >
> > - could you please describe the hashing algorithm that OpenMap is using? I
> > have previously written a GNU Trove based sparse matrix implementation which
> > was a map from the primitive "long" to "double". It looks analagous. I broke
> > the row/column index into a long for the key using binary operations, but
> > the hashcode was tricky as it had to be an integer (not long). A naive
> > implementation can lead to collisions for typical or symmetric matrices. It
> > looks like the current implementation is using an int -> double backing map!
> > Wouldn't it make more sense to use a long?
> >
> > The following code might be useful to you (don't forget the L marker and
> > casts or the compiler will silently cast to an int), there are two options
> > for a hashcode calculator... I've not stress tested the second one but it
> > might be more appropriate as it gives more weight to lower bits. Remember
> > that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16 hex
> > characters in Java bitwise.
> >
> > int hash1(long val) {
> > int first = Integer.reverse(key2row(val));
> > int second = key2col(val);
> > return first + second;
> > }
> >
> > int hash2(long val) {
> > int first = key2row(val) & 0x0000FFFF;
> > int second = key2col(val) << 16;
> > return first + second;
> > }
> >
> > long key(int row, int column) {
> > return (((long) row) << 32) + (long) column;
> > }
> >
> > int key2col(long key) {
> > return (int) (key & 0x00000000FFFFFFFFL);
> > }
> >
> > int key2row(long key) {
> > return (int) (key >>> 32);
> > }
> >
> >
> > Bill Barker wrote:
> >> I have a slight preference for leaving the markers empty until 3.0, but I
> >> can remove them as well. But I can wait to see what the community
> >> consensus
> >> is before making changes.
> >>
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Luc Maisonobe <Lu...@free.fr>.
sebb a écrit :
> On 22/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
>> sebb a écrit :
>>
>>> On 22/05/2009, Phil Steitz <ph...@gmail.com> wrote:
>> >> Luc Maisonobe wrote:
>> >>
>> >>> Ted Dunning a écrit :
>> >>>
>> >>>
>> >>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>> >>>>
>> >>>> As an example, a Lucene index is a reasonable implementation of a sparse
>> >>>> matrix.
>> >>>>
>> >>>> Would you require that I have to figure out how to make it serializable
>> >> just
>> >>>> because I declare it as a Matrix?
>> >>>>
>> >>>> Do you imagine that most developers will do more than just punt in such
>> >> a
>> >>>> situation if the interface absolutely requires that the object be
>> >>>> serializable?
>> >>>>
>> >>>> Leave it to particular implementations to be serializable or not.
>> >> Please,
>> >>>> please, please don't force it into the contract for all implementations.
>> >>>>
>> >>>>
>> >>> So we have reached a consensus: remove Serializable from interfaces and
>> >>> push it down to implementations only.
>> >>>
>> >>>
>> >> +1
>> >>
>>
>>>>> Any volunteer to do this rather boring work ?
>> >>>
>> >>>
>> >
>>
>>> I can make a start on it.
>> >
>> > I suggest that the implementations are flagged (e.g. with TODOs) until
>> > the serialization implementation has been checked and documented.
>> >
>> > WDYT?
>>
>> It sounds good. Many thanks.
>>
>>
>
> Just started to look at this, and the first interface I checked is
> MultivariateMatrixFunction.
>
> This does not have any implementations, except for anonymous classes
> in some Test classes.
The interface is intended to be implemented by users of the optimization
package.
>
> In this case, presumably only the interface needs to be updated?
Yes.
>
> Would it be useful to record in the Javadoc is the interface used to
> have Serializable?
> [As well as in the release notes, of course]
In this case, I don't think so. It is a new interface not yet published.
>
> I think this might be useful, but if no-one else does, then there's no
> point doing it.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Luc Maisonobe <Lu...@free.fr>.
sebb a écrit :
> On 22/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
>> sebb a écrit :
>>
>>> On 22/05/2009, Phil Steitz <ph...@gmail.com> wrote:
>> >> Luc Maisonobe wrote:
>> >>
>> >>> Ted Dunning a écrit :
>> >>>
>> >>>
>> >>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>> >>>>
>> >>>> As an example, a Lucene index is a reasonable implementation of a sparse
>> >>>> matrix.
>> >>>>
>> >>>> Would you require that I have to figure out how to make it serializable
>> >> just
>> >>>> because I declare it as a Matrix?
>> >>>>
>> >>>> Do you imagine that most developers will do more than just punt in such
>> >> a
>> >>>> situation if the interface absolutely requires that the object be
>> >>>> serializable?
>> >>>>
>> >>>> Leave it to particular implementations to be serializable or not.
>> >> Please,
>> >>>> please, please don't force it into the contract for all implementations.
>> >>>>
>> >>>>
>> >>> So we have reached a consensus: remove Serializable from interfaces and
>> >>> push it down to implementations only.
>> >>>
>> >>>
>> >> +1
>> >>
>>
>>>>> Any volunteer to do this rather boring work ?
>> >>>
>> >>>
>> >
>>
>>> I can make a start on it.
>> >
>> > I suggest that the implementations are flagged (e.g. with TODOs) until
>> > the serialization implementation has been checked and documented.
>> >
>> > WDYT?
>>
>> It sounds good. Many thanks.
>>
>>
>
> Just started to look at this, and the first interface I checked is
> MultivariateMatrixFunction.
>
> This does not have any implementations, except for anonymous classes
> in some Test classes.
The interface is intended to be implemented by users of the optimization
package.
>
> In this case, presumably only the interface needs to be updated?
Yes.
>
> Would it be useful to record in the Javadoc is the interface used to
> have Serializable?
> [As well as in the release notes, of course]
In this case, I don't think so. It is a new interface not yet published.
>
> I think this might be useful, but if no-one else does, then there's no
> point doing it.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by sebb <se...@gmail.com>.
On 22/05/2009, Luc Maisonobe <Lu...@free.fr> wrote:
> sebb a écrit :
>
> > On 22/05/2009, Phil Steitz <ph...@gmail.com> wrote:
> >> Luc Maisonobe wrote:
> >>
> >>> Ted Dunning a écrit :
> >>>
> >>>
> >>>> In favor or not, Serializable shouldn't in in widely used interfaces.
> >>>>
> >>>> As an example, a Lucene index is a reasonable implementation of a sparse
> >>>> matrix.
> >>>>
> >>>> Would you require that I have to figure out how to make it serializable
> >> just
> >>>> because I declare it as a Matrix?
> >>>>
> >>>> Do you imagine that most developers will do more than just punt in such
> >> a
> >>>> situation if the interface absolutely requires that the object be
> >>>> serializable?
> >>>>
> >>>> Leave it to particular implementations to be serializable or not.
> >> Please,
> >>>> please, please don't force it into the contract for all implementations.
> >>>>
> >>>>
> >>> So we have reached a consensus: remove Serializable from interfaces and
> >>> push it down to implementations only.
> >>>
> >>>
> >> +1
> >>
>
> >>> Any volunteer to do this rather boring work ?
> >>>
> >>>
> >
>
> > I can make a start on it.
> >
> > I suggest that the implementations are flagged (e.g. with TODOs) until
> > the serialization implementation has been checked and documented.
> >
> > WDYT?
>
> It sounds good. Many thanks.
>
>
Just started to look at this, and the first interface I checked is
MultivariateMatrixFunction.
This does not have any implementations, except for anonymous classes
in some Test classes.
In this case, presumably only the interface needs to be updated?
Would it be useful to record in the Javadoc is the interface used to
have Serializable?
[As well as in the release notes, of course]
I think this might be useful, but if no-one else does, then there's no
point doing it.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Luc Maisonobe <Lu...@free.fr>.
sebb a écrit :
> On 22/05/2009, Phil Steitz <ph...@gmail.com> wrote:
>> Luc Maisonobe wrote:
>>
>>> Ted Dunning a écrit :
>>>
>>>
>>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>>
>>>> As an example, a Lucene index is a reasonable implementation of a sparse
>>>> matrix.
>>>>
>>>> Would you require that I have to figure out how to make it serializable
>> just
>>>> because I declare it as a Matrix?
>>>>
>>>> Do you imagine that most developers will do more than just punt in such
>> a
>>>> situation if the interface absolutely requires that the object be
>>>> serializable?
>>>>
>>>> Leave it to particular implementations to be serializable or not.
>> Please,
>>>> please, please don't force it into the contract for all implementations.
>>>>
>>>>
>>> So we have reached a consensus: remove Serializable from interfaces and
>>> push it down to implementations only.
>>>
>>>
>> +1
>>
>>> Any volunteer to do this rather boring work ?
>>>
>>>
>
> I can make a start on it.
>
> I suggest that the implementations are flagged (e.g. with TODOs) until
> the serialization implementation has been checked and documented.
>
> WDYT?
It sounds good. Many thanks.
Luc
>
>> I wish I could say yes, but I am running out of buffer space atm ;)
>>
>> Phil
>>
>>
>>>
>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>> <bi...@verizon.net>wrote:
>>>>
>>>>
>>>>> - I *strongly* urge you to remove Serializable from everything!
>> Please, we
>>>>>
>>>>>> did this in MTJ and it turned out to be a major pain. A more
>> appropriate
>>>>>> approach is to define a class for reading/writing Matrix Market
>> files.
>>>>>> This
>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>> document
>>>>>> that the Serializable form is not guaranteed to remain compatible
>> across
>>>>>> versions.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of the
>> posts
>>>>> on this thread have suggested problems with the current
>> implementation, I'll
>>>>> try and run some tests over the (what is here, long) weekend. Again,
>> no
>>>>> consensus so not doing anything immediately.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail:
>> dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by sebb <se...@gmail.com>.
On 22/05/2009, Phil Steitz <ph...@gmail.com> wrote:
> Luc Maisonobe wrote:
>
> > Ted Dunning a écrit :
> >
> >
> > > In favor or not, Serializable shouldn't in in widely used interfaces.
> > >
> > > As an example, a Lucene index is a reasonable implementation of a sparse
> > > matrix.
> > >
> > > Would you require that I have to figure out how to make it serializable
> just
> > > because I declare it as a Matrix?
> > >
> > > Do you imagine that most developers will do more than just punt in such
> a
> > > situation if the interface absolutely requires that the object be
> > > serializable?
> > >
> > > Leave it to particular implementations to be serializable or not.
> Please,
> > > please, please don't force it into the contract for all implementations.
> > >
> > >
> >
> > So we have reached a consensus: remove Serializable from interfaces and
> > push it down to implementations only.
> >
> >
> +1
>
> > Any volunteer to do this rather boring work ?
> >
> >
I can make a start on it.
I suggest that the implementations are flagged (e.g. with TODOs) until
the serialization implementation has been checked and documented.
WDYT?
> I wish I could say yes, but I am running out of buffer space atm ;)
>
> Phil
>
>
> >
> >
> > > On Thu, May 21, 2009 at 8:20 PM, Bill Barker
> <bi...@verizon.net>wrote:
> > >
> > >
> > >
> > > > - I *strongly* urge you to remove Serializable from everything!
> Please, we
> > > >
> > > >
> > > > > did this in MTJ and it turned out to be a major pain. A more
> appropriate
> > > > > approach is to define a class for reading/writing Matrix Market
> files.
> > > > > This
> > > > > can be a new feature in 2.1. If you're going to leave it, at least
> > > > > document
> > > > > that the Serializable form is not guaranteed to remain compatible
> across
> > > > > versions.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > Like Luc, I'm generallly in favor of Serializable. Since some of the
> posts
> > > > on this thread have suggested problems with the current
> implementation, I'll
> > > > try and run some tests over the (what is here, long) weekend. Again,
> no
> > > > consensus so not doing anything immediately.
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail:
> dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Ted Dunning" <te...@gmail.com>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Sunday, May 24, 2009 11:47 PM
Subject: Re: [math] Serialization
> Bill,
>
> Did you commit those changes?
>
Yes, but just for o.a.c.m.linear.
> I ask because I see similar issues with abstract and static classes.
> Gamma
> is an example of the static class issue. This class has only static
> methods, but it is marked as serializable (why?). ContinuedFraction is a
> good example of an abstract class that is marked as serializable. This
> latter case leads to code that is either strange or incorrect. The
> strange
> version has a version identifier even on anonymous classes as in this code
> here:
I didn't say that I was finished ;)
>
> // create continued fraction
> ContinuedFraction cf = new ContinuedFraction() {
>
> private static final long serialVersionUID =
> 5378525034886164398L;
>
> @Override
> protected double getA(int n, double x) {
> return ((2.0 * n) + 1.0) - a + x;
> }
>
> @Override
> protected double getB(int n, double x) {
> return n * (a - n);
> }
> };
>
> ret = 1.0 / cf.evaluate(x, epsilon, maxIterations);
> ret = Math.exp(-x + (a * Math.log(x)) - logGamma(a)) * ret;
>
> If the serialVersionUID is ommitted, the anonymous class would be
> incorrectly serialized because it would inherit it's parent UID (I think).
> More importantly, I can't imagine why an abstract class like
> ContinuedFraction should be serializable.
>
> On Sun, May 24, 2009 at 6:10 PM, Bill Barker
> <bi...@verizon.net>wrote:
>
>> Ok, most of the rest look like transient classes (e.g. all of
>> linear.decomposition). So asking for what needs to be Serialized, and
>> what
>> doesn't.
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Ted Dunning <te...@gmail.com>.
Bill,
Did you commit those changes?
I ask because I see similar issues with abstract and static classes. Gamma
is an example of the static class issue. This class has only static
methods, but it is marked as serializable (why?). ContinuedFraction is a
good example of an abstract class that is marked as serializable. This
latter case leads to code that is either strange or incorrect. The strange
version has a version identifier even on anonymous classes as in this code
here:
// create continued fraction
ContinuedFraction cf = new ContinuedFraction() {
private static final long serialVersionUID =
5378525034886164398L;
@Override
protected double getA(int n, double x) {
return ((2.0 * n) + 1.0) - a + x;
}
@Override
protected double getB(int n, double x) {
return n * (a - n);
}
};
ret = 1.0 / cf.evaluate(x, epsilon, maxIterations);
ret = Math.exp(-x + (a * Math.log(x)) - logGamma(a)) * ret;
If the serialVersionUID is ommitted, the anonymous class would be
incorrectly serialized because it would inherit it's parent UID (I think).
More importantly, I can't imagine why an abstract class like
ContinuedFraction should be serializable.
On Sun, May 24, 2009 at 6:10 PM, Bill Barker <bi...@verizon.net>wrote:
> Ok, most of the rest look like transient classes (e.g. all of
> linear.decomposition). So asking for what needs to be Serialized, and what
> doesn't.
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] Serialization
Posted by Phil Steitz <ph...@gmail.com>.
Bill Barker wrote:
>
> ----- Original Message ----- From: "Ted Dunning" <te...@gmail.com>
> To: "Commons Developers List" <de...@commons.apache.org>
> Sent: Monday, May 25, 2009 4:52 PM
> Subject: Re: [math] Serialization
>
>
>> This sounds like the germ of a repair process.
>>
>> Nuke all Serializable declarations without test cases. Then, add
>> them back
>> where people care enough to guarantee correctness by demonstrating a
>> correct
>> round trip.
>>
>
> Would get my +1, but for the moment stats.** is such a mess (UID
> inherited from an abstract base class 3-4 levels up) that I'd like a
> consensus before putting hours into fixing it.
+1 to fix. I was +1 in the early days when things were "overserialized"
(IMO) [1] and am +1 today.
[1] http://markmail.org/thread/ui47gz75ol5euxj4
Phil
>
>> On Mon, May 25, 2009 at 4:32 PM, Bill Barker
>> <bi...@verizon.net>wrote:
>>
>>> But again, want to ask advice before going terminator on those
>>> packages.
>>> IMHO, we'll need to write junit tests for the ones that we leave
>>> Serializable, since I already found (and fixed) two classes in
>>> linear that
>>> couldn't be de-serialized.
>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Ted Dunning" <te...@gmail.com>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Monday, May 25, 2009 4:52 PM
Subject: Re: [math] Serialization
> This sounds like the germ of a repair process.
>
> Nuke all Serializable declarations without test cases. Then, add them
> back
> where people care enough to guarantee correctness by demonstrating a
> correct
> round trip.
>
Would get my +1, but for the moment stats.** is such a mess (UID inherited
from an abstract base class 3-4 levels up) that I'd like a consensus before
putting hours into fixing it.
> On Mon, May 25, 2009 at 4:32 PM, Bill Barker
> <bi...@verizon.net>wrote:
>
>> But again, want to ask advice before going terminator on those packages.
>> IMHO, we'll need to write junit tests for the ones that we leave
>> Serializable, since I already found (and fixed) two classes in linear
>> that
>> couldn't be de-serialized.
>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Ted Dunning <te...@gmail.com>.
This sounds like the germ of a repair process.
Nuke all Serializable declarations without test cases. Then, add them back
where people care enough to guarantee correctness by demonstrating a correct
round trip.
On Mon, May 25, 2009 at 4:32 PM, Bill Barker <bi...@verizon.net>wrote:
> But again, want to ask advice before going terminator on those packages.
> IMHO, we'll need to write junit tests for the ones that we leave
> Serializable, since I already found (and fixed) two classes in linear that
> couldn't be de-serialized.
--
Ted Dunning, CTO
DeepDyve
Re: [math] Serialization
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Luc Maisonobe" <Lu...@free.fr>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Monday, May 25, 2009 12:39 PM
Subject: Re: [math] Serialization
> Bill Barker a écrit :
>> Ok, most of the rest look like transient classes (e.g. all of
>> linear.decomposition). So asking for what needs to be Serialized, and
>> what doesn't.
>
> As the main developer of the decomposition package, I can say the
> reasons for having these classes Serializable are the same bad reasons I
> had for all the other ones ... So as far as I am concerned you can
> safely remove Serializable for all these interfaces and also for the
> implementation classes.
>
> Sorry for having caused such a mess.
>
You should see the mess Phil caused in stats ;). Almost (or more likely
all) of the classes under stats and subpackages look transient to me (e.g.
why would anyone want to serialize Kurtosis?). But again, want to ask
advice before going terminator on those packages. IMHO, we'll need to write
junit tests for the ones that we leave Serializable, since I already found
(and fixed) two classes in linear that couldn't be de-serialized.
> Luc
>
>>
>> ----- Original Message ----- From: "Bill Barker"
>> <bi...@verizon.net>
>> To: "Commons Developers List" <de...@commons.apache.org>
>> Sent: Saturday, May 23, 2009 12:13 AM
>> Subject: Re: [math] Serialization
>>
>>
>>>
>>> ----- Original Message ----- From: "Luc Maisonobe"
>>> <Lu...@free.fr>
>>> To: "Commons Developers List" <de...@commons.apache.org>
>>> Sent: Friday, May 22, 2009 4:19 AM
>>> Subject: Re: [math] Serialization
>>>
>>>
>>>> [I changed the subject to help follow the thread]
>>>>
>>>> Phil Steitz a écrit :
>>>>> Luc Maisonobe wrote:
>>>>>> Ted Dunning a écrit :
>>>>>>
>>>>>>> In favor or not, Serializable shouldn't in in widely used
>>>>>>> interfaces.
>>>>>>>
>>>>>>> As an example, a Lucene index is a reasonable implementation of a
>>>>>>> sparse
>>>>>>> matrix.
>>>>>>>
>>>>>>> Would you require that I have to figure out how to make it
>>>>>>> serializable just
>>>>>>> because I declare it as a Matrix?
>>>>>>>
>>>>>>> Do you imagine that most developers will do more than just punt in
>>>>>>> such a
>>>>>>> situation if the interface absolutely requires that the object be
>>>>>>> serializable?
>>>>>>>
>>>>>>> Leave it to particular implementations to be serializable or not.
>>>>>>> Please,
>>>>>>> please, please don't force it into the contract for all
>>>>>>> implementations.
>>>>>>>
>>>>>>
>>>>>> So we have reached a consensus: remove Serializable from interfaces
>>>>>> and
>>>>>> push it down to implementations only.
>>>>>>
>>>>> +1
>>>>
>>>> There is one interface at least for which I ask to retain
>>>> Externalizable
>>>> (not really the same as Serializable, but in the same spirit). It is
>>>> the
>>>> StepInterpolator interface in the ode.sampling package. Externalization
>>>> for this interface is a desired and important feature used for example
>>>> in the ContinuousOutputModel class. The interface is not intended to be
>>>> implemented by users, and in fact even the class implementing it are
>>>> not
>>>> directly visible by users: instances are directly built by ODE
>>>> integrators (each integrator has its own interpolator).
>>>>
>>>>>> Any volunteer to do this rather boring work ?
>>>
>>> I can take a stab at it (but may have fewer spare cycles than sebb).
>>>
>>>>>>
>>>>> I wish I could say yes, but I am running out of buffer space atm ;)
>>>>>
>>>>> Phil
>>>>>>
>>>>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>>>>>> <bi...@verizon.net>wrote:
>>>>>>>
>>>>>>>
>>>>>>>> - I *strongly* urge you to remove Serializable from everything!
>>>>>>>> Please, we
>>>>>>>>
>>>>>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>>>>>> appropriate
>>>>>>>>> approach is to define a class for reading/writing Matrix Market
>>>>>>>>> files.
>>>>>>>>> This
>>>>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>>>>> document
>>>>>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>>>>>> across
>>>>>>>>> versions.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>>>>>> the posts
>>>>>>>> on this thread have suggested problems with the current
>>>>>>>> implementation, I'll
>>>>>>>> try and run some tests over the (what is here, long) weekend.
>>>>>>>> Again, no
>>>>>>>> consensus so not doing anything immediately.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Luc Maisonobe <Lu...@free.fr>.
Bill Barker a écrit :
> Ok, most of the rest look like transient classes (e.g. all of
> linear.decomposition). So asking for what needs to be Serialized, and
> what doesn't.
As the main developer of the decomposition package, I can say the
reasons for having these classes Serializable are the same bad reasons I
had for all the other ones ... So as far as I am concerned you can
safely remove Serializable for all these interfaces and also for the
implementation classes.
Sorry for having caused such a mess.
Luc
>
> ----- Original Message ----- From: "Bill Barker" <bi...@verizon.net>
> To: "Commons Developers List" <de...@commons.apache.org>
> Sent: Saturday, May 23, 2009 12:13 AM
> Subject: Re: [math] Serialization
>
>
>>
>> ----- Original Message ----- From: "Luc Maisonobe"
>> <Lu...@free.fr>
>> To: "Commons Developers List" <de...@commons.apache.org>
>> Sent: Friday, May 22, 2009 4:19 AM
>> Subject: Re: [math] Serialization
>>
>>
>>> [I changed the subject to help follow the thread]
>>>
>>> Phil Steitz a écrit :
>>>> Luc Maisonobe wrote:
>>>>> Ted Dunning a écrit :
>>>>>
>>>>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>>>>
>>>>>> As an example, a Lucene index is a reasonable implementation of a
>>>>>> sparse
>>>>>> matrix.
>>>>>>
>>>>>> Would you require that I have to figure out how to make it
>>>>>> serializable just
>>>>>> because I declare it as a Matrix?
>>>>>>
>>>>>> Do you imagine that most developers will do more than just punt in
>>>>>> such a
>>>>>> situation if the interface absolutely requires that the object be
>>>>>> serializable?
>>>>>>
>>>>>> Leave it to particular implementations to be serializable or not.
>>>>>> Please,
>>>>>> please, please don't force it into the contract for all
>>>>>> implementations.
>>>>>>
>>>>>
>>>>> So we have reached a consensus: remove Serializable from interfaces
>>>>> and
>>>>> push it down to implementations only.
>>>>>
>>>> +1
>>>
>>> There is one interface at least for which I ask to retain Externalizable
>>> (not really the same as Serializable, but in the same spirit). It is the
>>> StepInterpolator interface in the ode.sampling package. Externalization
>>> for this interface is a desired and important feature used for example
>>> in the ContinuousOutputModel class. The interface is not intended to be
>>> implemented by users, and in fact even the class implementing it are not
>>> directly visible by users: instances are directly built by ODE
>>> integrators (each integrator has its own interpolator).
>>>
>>>>> Any volunteer to do this rather boring work ?
>>
>> I can take a stab at it (but may have fewer spare cycles than sebb).
>>
>>>>>
>>>> I wish I could say yes, but I am running out of buffer space atm ;)
>>>>
>>>> Phil
>>>>>
>>>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>>>>> <bi...@verizon.net>wrote:
>>>>>>
>>>>>>
>>>>>>> - I *strongly* urge you to remove Serializable from everything!
>>>>>>> Please, we
>>>>>>>
>>>>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>>>>> appropriate
>>>>>>>> approach is to define a class for reading/writing Matrix Market
>>>>>>>> files.
>>>>>>>> This
>>>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>>>> document
>>>>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>>>>> across
>>>>>>>> versions.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>>>>> the posts
>>>>>>> on this thread have suggested problems with the current
>>>>>>> implementation, I'll
>>>>>>> try and run some tests over the (what is here, long) weekend.
>>>>>>> Again, no
>>>>>>> consensus so not doing anything immediately.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Bill Barker <bi...@verizon.net>.
Ok, most of the rest look like transient classes (e.g. all of
linear.decomposition). So asking for what needs to be Serialized, and what
doesn't.
----- Original Message -----
From: "Bill Barker" <bi...@verizon.net>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Saturday, May 23, 2009 12:13 AM
Subject: Re: [math] Serialization
>
> ----- Original Message -----
> From: "Luc Maisonobe" <Lu...@free.fr>
> To: "Commons Developers List" <de...@commons.apache.org>
> Sent: Friday, May 22, 2009 4:19 AM
> Subject: Re: [math] Serialization
>
>
>> [I changed the subject to help follow the thread]
>>
>> Phil Steitz a écrit :
>>> Luc Maisonobe wrote:
>>>> Ted Dunning a écrit :
>>>>
>>>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>>>
>>>>> As an example, a Lucene index is a reasonable implementation of a
>>>>> sparse
>>>>> matrix.
>>>>>
>>>>> Would you require that I have to figure out how to make it
>>>>> serializable just
>>>>> because I declare it as a Matrix?
>>>>>
>>>>> Do you imagine that most developers will do more than just punt in
>>>>> such a
>>>>> situation if the interface absolutely requires that the object be
>>>>> serializable?
>>>>>
>>>>> Leave it to particular implementations to be serializable or not.
>>>>> Please,
>>>>> please, please don't force it into the contract for all
>>>>> implementations.
>>>>>
>>>>
>>>> So we have reached a consensus: remove Serializable from interfaces and
>>>> push it down to implementations only.
>>>>
>>> +1
>>
>> There is one interface at least for which I ask to retain Externalizable
>> (not really the same as Serializable, but in the same spirit). It is the
>> StepInterpolator interface in the ode.sampling package. Externalization
>> for this interface is a desired and important feature used for example
>> in the ContinuousOutputModel class. The interface is not intended to be
>> implemented by users, and in fact even the class implementing it are not
>> directly visible by users: instances are directly built by ODE
>> integrators (each integrator has its own interpolator).
>>
>>>> Any volunteer to do this rather boring work ?
>
> I can take a stab at it (but may have fewer spare cycles than sebb).
>
>>>>
>>> I wish I could say yes, but I am running out of buffer space atm ;)
>>>
>>> Phil
>>>>
>>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>>>> <bi...@verizon.net>wrote:
>>>>>
>>>>>
>>>>>> - I *strongly* urge you to remove Serializable from everything!
>>>>>> Please, we
>>>>>>
>>>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>>>> appropriate
>>>>>>> approach is to define a class for reading/writing Matrix Market
>>>>>>> files.
>>>>>>> This
>>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>>> document
>>>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>>>> across
>>>>>>> versions.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>>>> the posts
>>>>>> on this thread have suggested problems with the current
>>>>>> implementation, I'll
>>>>>> try and run some tests over the (what is here, long) weekend.
>>>>>> Again, no
>>>>>> consensus so not doing anything immediately.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Luc Maisonobe" <Lu...@free.fr>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Friday, May 22, 2009 4:19 AM
Subject: Re: [math] Serialization
> [I changed the subject to help follow the thread]
>
> Phil Steitz a écrit :
>> Luc Maisonobe wrote:
>>> Ted Dunning a écrit :
>>>
>>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>>
>>>> As an example, a Lucene index is a reasonable implementation of a
>>>> sparse
>>>> matrix.
>>>>
>>>> Would you require that I have to figure out how to make it
>>>> serializable just
>>>> because I declare it as a Matrix?
>>>>
>>>> Do you imagine that most developers will do more than just punt in
>>>> such a
>>>> situation if the interface absolutely requires that the object be
>>>> serializable?
>>>>
>>>> Leave it to particular implementations to be serializable or not.
>>>> Please,
>>>> please, please don't force it into the contract for all
>>>> implementations.
>>>>
>>>
>>> So we have reached a consensus: remove Serializable from interfaces and
>>> push it down to implementations only.
>>>
>> +1
>
> There is one interface at least for which I ask to retain Externalizable
> (not really the same as Serializable, but in the same spirit). It is the
> StepInterpolator interface in the ode.sampling package. Externalization
> for this interface is a desired and important feature used for example
> in the ContinuousOutputModel class. The interface is not intended to be
> implemented by users, and in fact even the class implementing it are not
> directly visible by users: instances are directly built by ODE
> integrators (each integrator has its own interpolator).
>
>>> Any volunteer to do this rather boring work ?
I can take a stab at it (but may have fewer spare cycles than sebb).
>>>
>> I wish I could say yes, but I am running out of buffer space atm ;)
>>
>> Phil
>>>
>>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>>> <bi...@verizon.net>wrote:
>>>>
>>>>
>>>>> - I *strongly* urge you to remove Serializable from everything!
>>>>> Please, we
>>>>>
>>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>>> appropriate
>>>>>> approach is to define a class for reading/writing Matrix Market
>>>>>> files.
>>>>>> This
>>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>>> document
>>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>>> across
>>>>>> versions.
>>>>>>
>>>>>>
>>>>>>
>>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>>> the posts
>>>>> on this thread have suggested problems with the current
>>>>> implementation, I'll
>>>>> try and run some tests over the (what is here, long) weekend.
>>>>> Again, no
>>>>> consensus so not doing anything immediately.
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Serialization
Posted by Luc Maisonobe <Lu...@free.fr>.
[I changed the subject to help follow the thread]
Phil Steitz a écrit :
> Luc Maisonobe wrote:
>> Ted Dunning a écrit :
>>
>>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>>
>>> As an example, a Lucene index is a reasonable implementation of a sparse
>>> matrix.
>>>
>>> Would you require that I have to figure out how to make it
>>> serializable just
>>> because I declare it as a Matrix?
>>>
>>> Do you imagine that most developers will do more than just punt in
>>> such a
>>> situation if the interface absolutely requires that the object be
>>> serializable?
>>>
>>> Leave it to particular implementations to be serializable or not.
>>> Please,
>>> please, please don't force it into the contract for all implementations.
>>>
>>
>> So we have reached a consensus: remove Serializable from interfaces and
>> push it down to implementations only.
>>
> +1
There is one interface at least for which I ask to retain Externalizable
(not really the same as Serializable, but in the same spirit). It is the
StepInterpolator interface in the ode.sampling package. Externalization
for this interface is a desired and important feature used for example
in the ContinuousOutputModel class. The interface is not intended to be
implemented by users, and in fact even the class implementing it are not
directly visible by users: instances are directly built by ODE
integrators (each integrator has its own interpolator).
>> Any volunteer to do this rather boring work ?
>>
> I wish I could say yes, but I am running out of buffer space atm ;)
>
> Phil
>>
>>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker
>>> <bi...@verizon.net>wrote:
>>>
>>>
>>>> - I *strongly* urge you to remove Serializable from everything!
>>>> Please, we
>>>>
>>>>> did this in MTJ and it turned out to be a major pain. A more
>>>>> appropriate
>>>>> approach is to define a class for reading/writing Matrix Market files.
>>>>> This
>>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>>> document
>>>>> that the Serializable form is not guaranteed to remain compatible
>>>>> across
>>>>> versions.
>>>>>
>>>>>
>>>>>
>>>> Like Luc, I'm generallly in favor of Serializable. Since some of
>>>> the posts
>>>> on this thread have suggested problems with the current
>>>> implementation, I'll
>>>> try and run some tests over the (what is here, long) weekend.
>>>> Again, no
>>>> consensus so not doing anything immediately.
>>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Ted Dunning a écrit :
>
>> In favor or not, Serializable shouldn't in in widely used interfaces.
>>
>> As an example, a Lucene index is a reasonable implementation of a sparse
>> matrix.
>>
>> Would you require that I have to figure out how to make it serializable just
>> because I declare it as a Matrix?
>>
>> Do you imagine that most developers will do more than just punt in such a
>> situation if the interface absolutely requires that the object be
>> serializable?
>>
>> Leave it to particular implementations to be serializable or not. Please,
>> please, please don't force it into the contract for all implementations.
>>
>
> So we have reached a consensus: remove Serializable from interfaces and
> push it down to implementations only.
>
+1
> Any volunteer to do this rather boring work ?
>
I wish I could say yes, but I am running out of buffer space atm ;)
Phil
>
>> On Thu, May 21, 2009 at 8:20 PM, Bill Barker <bi...@verizon.net>wrote:
>>
>>
>>> - I *strongly* urge you to remove Serializable from everything! Please, we
>>>
>>>> did this in MTJ and it turned out to be a major pain. A more appropriate
>>>> approach is to define a class for reading/writing Matrix Market files.
>>>> This
>>>> can be a new feature in 2.1. If you're going to leave it, at least
>>>> document
>>>> that the Serializable form is not guaranteed to remain compatible across
>>>> versions.
>>>>
>>>>
>>>>
>>> Like Luc, I'm generallly in favor of Serializable. Since some of the posts
>>> on this thread have suggested problems with the current implementation, I'll
>>> try and run some tests over the (what is here, long) weekend. Again, no
>>> consensus so not doing anything immediately.
>>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Ted Dunning a écrit :
> In favor or not, Serializable shouldn't in in widely used interfaces.
>
> As an example, a Lucene index is a reasonable implementation of a sparse
> matrix.
>
> Would you require that I have to figure out how to make it serializable just
> because I declare it as a Matrix?
>
> Do you imagine that most developers will do more than just punt in such a
> situation if the interface absolutely requires that the object be
> serializable?
>
> Leave it to particular implementations to be serializable or not. Please,
> please, please don't force it into the contract for all implementations.
So we have reached a consensus: remove Serializable from interfaces and
push it down to implementations only.
Any volunteer to do this rather boring work ?
>
> On Thu, May 21, 2009 at 8:20 PM, Bill Barker <bi...@verizon.net>wrote:
>
>> - I *strongly* urge you to remove Serializable from everything! Please, we
>>> did this in MTJ and it turned out to be a major pain. A more appropriate
>>> approach is to define a class for reading/writing Matrix Market files.
>>> This
>>> can be a new feature in 2.1. If you're going to leave it, at least
>>> document
>>> that the Serializable form is not guaranteed to remain compatible across
>>> versions.
>>>
>>>
>> Like Luc, I'm generallly in favor of Serializable. Since some of the posts
>> on this thread have suggested problems with the current implementation, I'll
>> try and run some tests over the (what is here, long) weekend. Again, no
>> consensus so not doing anything immediately.
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
In favor or not, Serializable shouldn't in in widely used interfaces.
As an example, a Lucene index is a reasonable implementation of a sparse
matrix.
Would you require that I have to figure out how to make it serializable just
because I declare it as a Matrix?
Do you imagine that most developers will do more than just punt in such a
situation if the interface absolutely requires that the object be
serializable?
Leave it to particular implementations to be serializable or not. Please,
please, please don't force it into the contract for all implementations.
On Thu, May 21, 2009 at 8:20 PM, Bill Barker <bi...@verizon.net>wrote:
> - I *strongly* urge you to remove Serializable from everything! Please, we
>> did this in MTJ and it turned out to be a major pain. A more appropriate
>> approach is to define a class for reading/writing Matrix Market files.
>> This
>> can be a new feature in 2.1. If you're going to leave it, at least
>> document
>> that the Serializable form is not guaranteed to remain compatible across
>> versions.
>>
>>
> Like Luc, I'm generallly in favor of Serializable. Since some of the posts
> on this thread have suggested problems with the current implementation, I'll
> try and run some tests over the (what is here, long) weekend. Again, no
> consensus so not doing anything immediately.
--
Ted Dunning, CTO
DeepDyve
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Bill Barker a écrit :
>
>> ----- Original Message ----- From: "Ted Dunning" <te...@gmail.com>
>> To: "Commons Developers List" <de...@commons.apache.org>
>> Sent: Saturday, May 16, 2009 1:01 PM
>> Subject: Re: [math] Re: commons-math, matrix-toolkits-java and
>> consolidation
>>
>>
>>
>>> +1 on this change. Grand names like SparseRealMatrix should be
>>> reserved for
>>> the interfaces.
>>>
>>>
>> I'm +1 on these grounds (and, yes, that means I'm volunteering to do the
>> work). I could add the getSparcity method as well (the only one that is
>> useful for 2.0). Could do the getShape which would presumably return an
>> enum (which would only have one element in 2.0) to ease the pain of
>> upgrading to 2.1 when there are more.
>>
>
> Sounds good, I'll let you handle this and focus on my very late stiff
> ODE handling. Anyone volunteering for the return values of multiply, add
> and similar methods on RealMatrixImpl, DenseRealMatrix, ... ?
>
I am getting close to done on the stats stuff and will take a look if no
one else has picked this up by the time I get to it.
Phil
> Thanks,
> Luc
>
>
>>
>>
>>> On Sat, May 16, 2009 at 10:26 AM, Luc Maisonobe
>>> <Lu...@free.fr>wrote:
>>>
>>>
>>>>> - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix, >
>>>>>
>>>> Vector}.
>>>>
>>>>> Implement above interfaces.
>>>>>
>>>> I have no opinion on that. What do others think ?
>>>>
>>>
>>>
>>> --
>>> Ted Dunning, CTO
>>> DeepDyve
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Bill Barker a écrit :
>
> ----- Original Message ----- From: "Ted Dunning" <te...@gmail.com>
> To: "Commons Developers List" <de...@commons.apache.org>
> Sent: Saturday, May 16, 2009 1:01 PM
> Subject: Re: [math] Re: commons-math, matrix-toolkits-java and
> consolidation
>
>
>> +1 on this change. Grand names like SparseRealMatrix should be
>> reserved for
>> the interfaces.
>>
>
> I'm +1 on these grounds (and, yes, that means I'm volunteering to do the
> work). I could add the getSparcity method as well (the only one that is
> useful for 2.0). Could do the getShape which would presumably return an
> enum (which would only have one element in 2.0) to ease the pain of
> upgrading to 2.1 when there are more.
Sounds good, I'll let you handle this and focus on my very late stiff
ODE handling. Anyone volunteering for the return values of multiply, add
and similar methods on RealMatrixImpl, DenseRealMatrix, ... ?
Thanks,
Luc
>
>
>
>> On Sat, May 16, 2009 at 10:26 AM, Luc Maisonobe
>> <Lu...@free.fr>wrote:
>>
>>>
>>> > - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix, >
>>> Vector}.
>>> > Implement above interfaces.
>>>
>>> I have no opinion on that. What do others think ?
>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> Regarding the name of ArrayRealMatrix. Please don't forget to include the
> "2DRow" part to the name (indicating a 2D array which is Row ordered) to
> indicate the implementation type. Post 2.0 I'll convince you that a 1D Array
> approach is best as it will lead to more efficient use of BLAS and therefore
> reproducibility of reference algorithms from, e.g. the Templates project on
> netlib ;-)
>
> I've never seen a storage type like BlockRealMatrix before... I'm interested
> in seeing where it performs better than, say, a full 1D representation of a
> dense matrix. Breaking up the storage like this makes it very difficult to
> use BLAS/LAPACK internally.
This is a simple approach I tried based on cache considerations. It is a
compromise that works well when some algorithms are row oriented, others
are column oriented and still others are mixed (like multiplication). It
does improve multiplication time with respect to RealMatrixImpl for
example on matrices larger than 100x100, but it may need adapting block
size depending on target.
A more complex example is given in a paper by Siddhartha Chatterjee,
Alvin R. Lebeck, Praveen K. Patnala and Mithuna Thottethodi: Recursive
Array Layouts and Fast Matrix Multiplication
<http://www.cs.duke.edu/~alvy/papers/matrix-tpds.pdf>. I have tried this
approach too (using the Gray-Morton space-filling curve) but the gain
was not worth the pain. This attempt was not completed for this reason,
but it is still available in the experimental directory in the
repository, it is the RecursiveLayoutRealMatrix class. Maybe it can be
resurrected later, once completed and optimized.
>
>
> Luc Maisonobe wrote:
>>> - I discourage the use of the classes named *Impl. They will get very
>>> confusing when other implementations are added later! Instead, I
>>> recommend
>>> the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
>>> backed implementation using row ordering). This allows a column-based or
>>> 1D
>>> implementation in the future without names getting very confusing. These
>>> implementations are hidden from users who just use the MatrixUtils help
>> Phil suggested to change RealMatrixImpl to ArrayRealMatrix (and
>> DenseRealMatrix to BlockRealMatrix). This sounds good to me.
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Regarding the name of ArrayRealMatrix. Please don't forget to include the
"2DRow" part to the name (indicating a 2D array which is Row ordered) to
indicate the implementation type. Post 2.0 I'll convince you that a 1D Array
approach is best as it will lead to more efficient use of BLAS and therefore
reproducibility of reference algorithms from, e.g. the Templates project on
netlib ;-)
I've never seen a storage type like BlockRealMatrix before... I'm interested
in seeing where it performs better than, say, a full 1D representation of a
dense matrix. Breaking up the storage like this makes it very difficult to
use BLAS/LAPACK internally.
Luc Maisonobe wrote:
>
>> - I discourage the use of the classes named *Impl. They will get very
>> confusing when other implementations are added later! Instead, I
>> recommend
>> the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
>> backed implementation using row ordering). This allows a column-based or
>> 1D
>> implementation in the future without names getting very confusing. These
>> implementations are hidden from users who just use the MatrixUtils help
>
> Phil suggested to change RealMatrixImpl to ArrayRealMatrix (and
> DenseRealMatrix to BlockRealMatrix). This sounds good to me.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23655465.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Luc, it absolutely would help the compiler... but the point is that the
return types might not remain as they currently are in future releases. For
example, if you multiply a dense matrix and a diagonal matrix... does it
make sense to return a dense matrix? No. :-)
Incidentally, in MTJ we have a mechanism to allow the client to decide on
the storage type for return types. It's a bit clunky, and I'd prefer a
factory approach. I'd like to open up the debate on a mechanism in
commons-math for doing this post-2.0.
Luc Maisonobe wrote:
>
>> - changing the return type to be actual classes was only supposed to be
>> for
>> copy() for 2.0. Doing this on multiply, add, etc is a big mistake...
>> there
>> is no guarantee that is the best thing to do and it is likely this will
>> change in the future. This stands for all implementations. Great thought
>> needs to go into each method, not just by reading the current
>> implementation.
>
> It may help the compiler using directly the optimized versions in
> statements like:
>
> m1.add(m2.multiply(m3.subtract(m4))
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23655328.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> Bill, I've had a look at some of the recent changes... some comments,
> including some new thoughts:-
>
> - changing the return type to be actual classes was only supposed to be for
> copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
> is no guarantee that is the best thing to do and it is likely this will
> change in the future. This stands for all implementations. Great thought
> needs to go into each method, not just by reading the current
> implementation.
It may help the compiler using directly the optimized versions in
statements like:
m1.add(m2.multiply(m3.subtract(m4))
>
> - I *strongly* urge you to remove Serializable from everything! Please, we
> did this in MTJ and it turned out to be a major pain. A more appropriate
> approach is to define a class for reading/writing Matrix Market files. This
> can be a new feature in 2.1. If you're going to leave it, at least document
> that the Serializable form is not guaranteed to remain compatible across
> versions.
I disagree with and I think its me who started adding this interface
everywhere. The rationale is explained in the following use case:
As a user, I want to implement an interface for some xyz library where
the interface already extends Serializable. In my implementation, I want
to have an instance field that is some RealMatrix. If RealMatrix does
not declare it extends Serializable, I keep getting warnings about
having a non-serializable field in a serializable class. Looking the
other way round having serializable and not using it never caused me any
pain because it was really a generalized pattern. Mixing serializable
and non serializable is difficult, putting it everywhere and sticking to
this policy is simple.
What kind of pain did Serializable cause in MTJ ?
BTW, I'm really +1 to Matrix Market files, could you open a Jira issue
asking for this enhancement for 2.1 ?
>
> - I discourage the use of the classes named *Impl. They will get very
> confusing when other implementations are added later! Instead, I recommend
> the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
> backed implementation using row ordering). This allows a column-based or 1D
> implementation in the future without names getting very confusing. These
> implementations are hidden from users who just use the MatrixUtils help
Phil suggested to change RealMatrixImpl to ArrayRealMatrix (and
DenseRealMatrix to BlockRealMatrix). This sounds good to me.
>
> - -1 for inclusion of methods in the sparse interfaces and shape type enum.
> This needs more thought and discussion before inclusion. I am happy enough
> for marker interfaces to be included (as long as it is documented that these
> interfaces are subject to update in the next release), but strongly against
> including methods in them.
+1 to further thoughts and discussion and postponing declaring methods
in the interfaces after 2.0. Lets start with empty marker interfaces.
Luc
>
> - could you please describe the hashing algorithm that OpenMap is using? I
> have previously written a GNU Trove based sparse matrix implementation which
> was a map from the primitive "long" to "double". It looks analagous. I broke
> the row/column index into a long for the key using binary operations, but
> the hashcode was tricky as it had to be an integer (not long). A naive
> implementation can lead to collisions for typical or symmetric matrices. It
> looks like the current implementation is using an int -> double backing map!
> Wouldn't it make more sense to use a long?
>
> The following code might be useful to you (don't forget the L marker and
> casts or the compiler will silently cast to an int), there are two options
> for a hashcode calculator... I've not stress tested the second one but it
> might be more appropriate as it gives more weight to lower bits. Remember
> that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16 hex
> characters in Java bitwise.
>
> int hash1(long val) {
> int first = Integer.reverse(key2row(val));
> int second = key2col(val);
> return first + second;
> }
>
> int hash2(long val) {
> int first = key2row(val) & 0x0000FFFF;
> int second = key2col(val) << 16;
> return first + second;
> }
>
> long key(int row, int column) {
> return (((long) row) << 32) + (long) column;
> }
>
> int key2col(long key) {
> return (int) (key & 0x00000000FFFFFFFFL);
> }
>
> int key2row(long key) {
> return (int) (key >>> 32);
> }
>
>
> Bill Barker wrote:
>> I have a slight preference for leaving the markers empty until 3.0, but I
>> can remove them as well. But I can wait to see what the community
>> consensus
>> is before making changes.
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
It's a shame you can't use the GNU Trove project, it would be a perfect
robust backend for primitive mapping.
Bill Barker wrote:
>
> I've just been playing Code-Monkey, so Phil and Luc can do more
> interesting
> stuff, and we can get the 2.0 version released ;). OpenIntToDoubleHashMap
> was already there when I got involved with [math], so I just used it. Luc
> was the one that initially committed OpenIntToDoulbleHashMap, but I think
> that it came from a Jira patch (and, yes, I'm too lazy to look up which
> one).
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23666388.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Bill Barker <bi...@verizon.net>.
Time zones mean that I tend to come in late here :(. More answers inline.
----- Original Message -----
From: "Sam Halliday" <sa...@gmail.com>
To: <de...@commons.apache.org>
Sent: Thursday, May 21, 2009 3:13 AM
Subject: Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
>
> Bill, I've had a look at some of the recent changes... some comments,
> including some new thoughts:-
>
> - changing the return type to be actual classes was only supposed to be
> for
> copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
> is no guarantee that is the best thing to do and it is likely this will
> change in the future. This stands for all implementations. Great thought
> needs to go into each method, not just by reading the current
> implementation.
>
I sort of like being able to use the optimized methods for inlining, but
don't feel strongly on this. I'll go for whatever the community consensus
is (and I don't see one at the moment). It was a search/replace before and
I can do it again ;).
> - I *strongly* urge you to remove Serializable from everything! Please, we
> did this in MTJ and it turned out to be a major pain. A more appropriate
> approach is to define a class for reading/writing Matrix Market files.
> This
> can be a new feature in 2.1. If you're going to leave it, at least
> document
> that the Serializable form is not guaranteed to remain compatible across
> versions.
>
Like Luc, I'm generallly in favor of Serializable. Since some of the posts
on this thread have suggested problems with the current implementation, I'll
try and run some tests over the (what is here, long) weekend. Again, no
consensus so not doing anything immediately.
> - I discourage the use of the classes named *Impl. They will get very
> confusing when other implementations are added later! Instead, I recommend
> the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
> backed implementation using row ordering). This allows a column-based or
> 1D
> implementation in the future without names getting very confusing. These
> implementations are hidden from users who just use the MatrixUtils help
>
Have no problem doing a deprecate-and-copy. Will do once there is a
consensus on the name.
> - -1 for inclusion of methods in the sparse interfaces and shape type
> enum.
> This needs more thought and discussion before inclusion. I am happy enough
> for marker interfaces to be included (as long as it is documented that
> these
> interfaces are subject to update in the next release), but strongly
> against
> including methods in them.
>
In the current SVN, these are empty marker interfaces, since we seem to have
a consensus on that.
> - could you please describe the hashing algorithm that OpenMap is using? I
> have previously written a GNU Trove based sparse matrix implementation
> which
> was a map from the primitive "long" to "double". It looks analagous. I
> broke
> the row/column index into a long for the key using binary operations, but
> the hashcode was tricky as it had to be an integer (not long). A naive
> implementation can lead to collisions for typical or symmetric matrices.
> It
> looks like the current implementation is using an int -> double backing
> map!
> Wouldn't it make more sense to use a long?
>
I've just been playing Code-Monkey, so Phil and Luc can do more interesting
stuff, and we can get the 2.0 version released ;). OpenIntToDoubleHashMap
was already there when I got involved with [math], so I just used it. Luc
was the one that initially committed OpenIntToDoulbleHashMap, but I think
that it came from a Jira patch (and, yes, I'm too lazy to look up which
one).
> The following code might be useful to you (don't forget the L marker and
> casts or the compiler will silently cast to an int), there are two options
> for a hashcode calculator... I've not stress tested the second one but it
> might be more appropriate as it gives more weight to lower bits. Remember
> that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16
> hex
> characters in Java bitwise.
>
> int hash1(long val) {
> int first = Integer.reverse(key2row(val));
> int second = key2col(val);
> return first + second;
> }
>
> int hash2(long val) {
> int first = key2row(val) & 0x0000FFFF;
> int second = key2col(val) << 16;
> return first + second;
> }
>
> long key(int row, int column) {
> return (((long) row) << 32) + (long) column;
> }
>
> int key2col(long key) {
> return (int) (key & 0x00000000FFFFFFFFL);
> }
>
> int key2row(long key) {
> return (int) (key >>> 32);
> }
>
>
> Bill Barker wrote:
>>
>> I have a slight preference for leaving the markers empty until 3.0, but I
>> can remove them as well. But I can wait to see what the community
>> consensus
>> is before making changes.
>>
>
> --
> View this message in context:
> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23650705.html
> Sent from the Commons - Dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Bill, I've had a look at some of the recent changes... some comments,
including some new thoughts:-
- changing the return type to be actual classes was only supposed to be for
copy() for 2.0. Doing this on multiply, add, etc is a big mistake... there
is no guarantee that is the best thing to do and it is likely this will
change in the future. This stands for all implementations. Great thought
needs to go into each method, not just by reading the current
implementation.
- I *strongly* urge you to remove Serializable from everything! Please, we
did this in MTJ and it turned out to be a major pain. A more appropriate
approach is to define a class for reading/writing Matrix Market files. This
can be a new feature in 2.1. If you're going to leave it, at least document
that the Serializable form is not guaranteed to remain compatible across
versions.
- I discourage the use of the classes named *Impl. They will get very
confusing when other implementations are added later! Instead, I recommend
the names ArrayRealVector, Array2DRowRealMatrix (to indicate a 2D array
backed implementation using row ordering). This allows a column-based or 1D
implementation in the future without names getting very confusing. These
implementations are hidden from users who just use the MatrixUtils help
- -1 for inclusion of methods in the sparse interfaces and shape type enum.
This needs more thought and discussion before inclusion. I am happy enough
for marker interfaces to be included (as long as it is documented that these
interfaces are subject to update in the next release), but strongly against
including methods in them.
- could you please describe the hashing algorithm that OpenMap is using? I
have previously written a GNU Trove based sparse matrix implementation which
was a map from the primitive "long" to "double". It looks analagous. I broke
the row/column index into a long for the key using binary operations, but
the hashcode was tricky as it had to be an integer (not long). A naive
implementation can lead to collisions for typical or symmetric matrices. It
looks like the current implementation is using an int -> double backing map!
Wouldn't it make more sense to use a long?
The following code might be useful to you (don't forget the L marker and
casts or the compiler will silently cast to an int), there are two options
for a hashcode calculator... I've not stress tested the second one but it
might be more appropriate as it gives more weight to lower bits. Remember
that FF is 256, or a full byte... so a full 64 bit long is 8 bytes or 16 hex
characters in Java bitwise.
int hash1(long val) {
int first = Integer.reverse(key2row(val));
int second = key2col(val);
return first + second;
}
int hash2(long val) {
int first = key2row(val) & 0x0000FFFF;
int second = key2col(val) << 16;
return first + second;
}
long key(int row, int column) {
return (((long) row) << 32) + (long) column;
}
int key2col(long key) {
return (int) (key & 0x00000000FFFFFFFFL);
}
int key2row(long key) {
return (int) (key >>> 32);
}
Bill Barker wrote:
>
> I have a slight preference for leaving the markers empty until 3.0, but I
> can remove them as well. But I can wait to see what the community
> consensus
> is before making changes.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23650705.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Phil Steitz" <ph...@gmail.com>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Wednesday, May 20, 2009 11:05 AM
Subject: Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
> Sam Halliday wrote:
>> Bill, I strongly discourage adding these methods at this time. We will
>> regret
>> it.
>>
>> If you don't want to change (i.e. add new methods) to an interface, then
>> the
>> sensible thing is to omit these interfaces for 2.0 and introduce them
>> with
>> 2.1.
>>
> +1. Unless we are either a) agreed on contents of non-marker interfaces
> or b) willing to leave (until 3.0) markers empty, we should omit them. We
> cannot introduce incompatible changes in point releases.
>
I have a slight preference for leaving the markers empty until 3.0, but I
can remove them as well. But I can wait to see what the community consensus
is before making changes.
> Phil
>>
>> Bill Barker wrote:
>>
>>> What I actually went for is to add getSparcity to the SparseRealVector
>>> interface (should be easy to calculate in one dimension, and mostly
>>> useful for DEBUG level logging, so stubbing it shouldn't be a problem if
>>> not),
>>> and getShape for the SparseRealMatrix interface that returns an enum
>>> that currently only has the value of 'Any'.
>>>
>>> I really don't like the idea of changing the API between minor releases,
>>> so put in place-holders for what seems to be the consensus for going
>>> forward.
>>>
>>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Sam Halliday wrote:
> Bill, I strongly discourage adding these methods at this time. We will regret
> it.
>
> If you don't want to change (i.e. add new methods) to an interface, then the
> sensible thing is to omit these interfaces for 2.0 and introduce them with
> 2.1.
>
+1. Unless we are either a) agreed on contents of non-marker interfaces
or b) willing to leave (until 3.0) markers empty, we should omit them.
We cannot introduce incompatible changes in point releases.
Phil
>
> Bill Barker wrote:
>
>> What I actually went for is to add getSparcity to the SparseRealVector
>> interface (should be easy to calculate in one dimension, and mostly useful
>> for DEBUG level logging, so stubbing it shouldn't be a problem if not),
>> and
>> getShape for the SparseRealMatrix interface that returns an enum that
>> currently only has the value of 'Any'.
>>
>> I really don't like the idea of changing the API between minor releases,
>> so
>> put in place-holders for what seems to be the consensus for going forward.
>>
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Bill, I strongly discourage adding these methods at this time. We will regret
it.
If you don't want to change (i.e. add new methods) to an interface, then the
sensible thing is to omit these interfaces for 2.0 and introduce them with
2.1.
Bill Barker wrote:
>
> What I actually went for is to add getSparcity to the SparseRealVector
> interface (should be easy to calculate in one dimension, and mostly useful
> for DEBUG level logging, so stubbing it shouldn't be a problem if not),
> and
> getShape for the SparseRealMatrix interface that returns an enum that
> currently only has the value of 'Any'.
>
> I really don't like the idea of changing the API between minor releases,
> so
> put in place-holders for what seems to be the consensus for going forward.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23611886.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Sam Halliday" <sa...@gmail.com>
To: <de...@commons.apache.org>
Sent: Monday, May 18, 2009 4:29 AM
Subject: Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
>
> Excellent :-)
>
> I think the consensus is to keep the sparse interfaces empty for now...
> don't want to lock ourselves into anything without detailed discussion,
> which there is no time for before 2.0.
>
> Just drop a note into the Javadocs to the effect that methods may be added
> in the next release. I can't see users extending these interfaces in 2.0,
> because they are not even going to be used internally.
>
What I actually went for is to add getSparcity to the SparseRealVector
interface (should be easy to calculate in one dimension, and mostly useful
for DEBUG level logging, so stubbing it shouldn't be a problem if not), and
getShape for the SparseRealMatrix interface that returns an enum that
currently only has the value of 'Any'.
I really don't like the idea of changing the API between minor releases, so
put in place-holders for what seems to be the consensus for going forward.
>
> Bill Barker wrote:
>>
>> I'm +1 on these grounds (and, yes, that means I'm volunteering to do the
>> work). I could add the getSparcity method as well (the only one that is
>> useful for 2.0). Could do the getShape which would presumably return an
>> enum (which would only have one element in 2.0) to ease the pain of
>> upgrading to 2.1 when there are more.
>>
>
> --
> View this message in context:
> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23595640.html
> Sent from the Commons - Dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Excellent :-)
I think the consensus is to keep the sparse interfaces empty for now...
don't want to lock ourselves into anything without detailed discussion,
which there is no time for before 2.0.
Just drop a note into the Javadocs to the effect that methods may be added
in the next release. I can't see users extending these interfaces in 2.0,
because they are not even going to be used internally.
Bill Barker wrote:
>
> I'm +1 on these grounds (and, yes, that means I'm volunteering to do the
> work). I could add the getSparcity method as well (the only one that is
> useful for 2.0). Could do the getShape which would presumably return an
> enum (which would only have one element in 2.0) to ease the pain of
> upgrading to 2.1 when there are more.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23595640.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Ted Dunning" <te...@gmail.com>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Saturday, May 16, 2009 1:01 PM
Subject: Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
> +1 on this change. Grand names like SparseRealMatrix should be reserved
> for
> the interfaces.
>
I'm +1 on these grounds (and, yes, that means I'm volunteering to do the
work). I could add the getSparcity method as well (the only one that is
useful for 2.0). Could do the getShape which would presumably return an
enum (which would only have one element in 2.0) to ease the pain of
upgrading to 2.1 when there are more.
> On Sat, May 16, 2009 at 10:26 AM, Luc Maisonobe
> <Lu...@free.fr>wrote:
>
>>
>> > - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix,
>> > Vector}.
>> > Implement above interfaces.
>>
>> I have no opinion on that. What do others think ?
>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
+1 on this change. Grand names like SparseRealMatrix should be reserved for
the interfaces.
On Sat, May 16, 2009 at 10:26 AM, Luc Maisonobe <Lu...@free.fr>wrote:
>
> > - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix, Vector}.
> > Implement above interfaces.
>
> I have no opinion on that. What do others think ?
--
Ted Dunning, CTO
DeepDyve
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Phil Steitz a écrit :
>
>> Luc Maisonobe wrote:
>>
>>> Sam Halliday a écrit :
>>>
>>>
>>>> I've had a quick look at the 2.0 API and the only changes I'd like to
>>>> request
>>>> are:-
>>>>
>>>> - create interfaces RealSparse{Matrix, Vector} to indicate a sparse
>>>> storage
>>>> implementation. Can be empty (for now).
>>>>
>>>>
>>> I suppose these interfaces would extend Real{Matrix, Vector} ?
>>>
>>>
>>>
>>>> - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix,
>>>> Vector}.
>>>> Implement above interfaces.
>>>>
>>>>
>>> I have no opinion on that. What do others think ?
>>>
>>>
>> +1 for "dethroning" ;)
>>
>> With the new interfaces, we could also drop the "Sparse" from the name -
>> i.e, "OpenMapRealMatrix"
>>
>> I am wondering now whether similar marking and "dethroning" should
>> happen on the dense side - i.e DenseReal{Matrix, Vector},
>> BlockRealMatrix (currently DenseRealMatrix), ArrayRealMatrix (currently
>> RealMatrixImpl)?
>>
>
> +1. This is simple for DenseRealMatrix since it has been created after
> 1.2, but RealMatrixImpl may induce more problems to users. Should we
> copy an deprecate ?
>
Yes, for RealMatrixImpl, we should copy and deprecate the original.
Phil
>
>>>
>>>
>>>> - implementations should subclass the return type of some methods in the
>>>> Real{Matrix, Vector} API. e.g. RealVectorImpl.copy() returns a
>>>> RealVector,
>>>> but could declare that it returns a RealVectorImpl. This will avoid
>>>> needless
>>>> user casts. In future APIs, this could be a powerful way to define the
>>>> return type of matrix-matrix computation when the storage classes are
>>>> known... e.g. declaring that you return a DiagonalMatrix.
>>>>
>>> I didn't know changing the return type to a subtype was allowed when
>>> implementing an interface! This is for sure a good thing to do and could
>>> probably be used in many other places too.
>>>
>>>
>> +1 in general, but need to realize that when implemented, this changes
>> locks us in. In the specific cases mentioned above, this is a
>> no-brainer, but there are others (mostly in the decomp classes) where it
>> is not so obvious.
>>
>>
>>>
>>>
>>>> - is it too late to define a Norm enum and take that as a parameter,
>>>> rather
>>>> than explicit methods for each Norm type?
>>>>
>>>>
>>> There are many places where we use the same pattern outside of linear
>>> algebra. I'm reluctant to change this because if we extend the enum
>>> later, we may forget to add a new case in all implementations (somewhere
>>> in a switch statement), so we should add an exception for defensive
>>> programming. A method name enforced in an interface prevent such errors,
>>> you cannot compile your implementation if you forget a method.
>>>
>>>
>> Agree with Luc on this, but more from a Java style than maintenance
>> standpoint. We have tried in general to stay away from "behavior flags"
>> in method signatures up to now.
>>
>> Phil
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Phil Steitz a écrit :
> Luc Maisonobe wrote:
>> Sam Halliday a écrit :
>>
>>> I've had a quick look at the 2.0 API and the only changes I'd like to
>>> request
>>> are:-
>>>
>>> - create interfaces RealSparse{Matrix, Vector} to indicate a sparse
>>> storage
>>> implementation. Can be empty (for now).
>>>
>>
>> I suppose these interfaces would extend Real{Matrix, Vector} ?
>>
>>
>>> - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix,
>>> Vector}.
>>> Implement above interfaces.
>>>
>>
>> I have no opinion on that. What do others think ?
>>
> +1 for "dethroning" ;)
>
> With the new interfaces, we could also drop the "Sparse" from the name -
> i.e, "OpenMapRealMatrix"
>
> I am wondering now whether similar marking and "dethroning" should
> happen on the dense side - i.e DenseReal{Matrix, Vector},
> BlockRealMatrix (currently DenseRealMatrix), ArrayRealMatrix (currently
> RealMatrixImpl)?
+1. This is simple for DenseRealMatrix since it has been created after
1.2, but RealMatrixImpl may induce more problems to users. Should we
copy an deprecate ?
>>
>>> - implementations should subclass the return type of some methods in the
>>> Real{Matrix, Vector} API. e.g. RealVectorImpl.copy() returns a
>>> RealVector,
>>> but could declare that it returns a RealVectorImpl. This will avoid
>>> needless
>>> user casts. In future APIs, this could be a powerful way to define the
>>> return type of matrix-matrix computation when the storage classes are
>>> known... e.g. declaring that you return a DiagonalMatrix.
>>
>> I didn't know changing the return type to a subtype was allowed when
>> implementing an interface! This is for sure a good thing to do and could
>> probably be used in many other places too.
>>
> +1 in general, but need to realize that when implemented, this changes
> locks us in. In the specific cases mentioned above, this is a
> no-brainer, but there are others (mostly in the decomp classes) where it
> is not so obvious.
>
>>
>>> - is it too late to define a Norm enum and take that as a parameter,
>>> rather
>>> than explicit methods for each Norm type?
>>>
>>
>> There are many places where we use the same pattern outside of linear
>> algebra. I'm reluctant to change this because if we extend the enum
>> later, we may forget to add a new case in all implementations (somewhere
>> in a switch statement), so we should add an exception for defensive
>> programming. A method name enforced in an interface prevent such errors,
>> you cannot compile your implementation if you forget a method.
>>
> Agree with Luc on this, but more from a Java style than maintenance
> standpoint. We have tried in general to stay away from "behavior flags"
> in method signatures up to now.
>
> Phil
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Sam Halliday a écrit :
>
>> I've had a quick look at the 2.0 API and the only changes I'd like to request
>> are:-
>>
>> - create interfaces RealSparse{Matrix, Vector} to indicate a sparse storage
>> implementation. Can be empty (for now).
>>
>
> I suppose these interfaces would extend Real{Matrix, Vector} ?
>
>
>> - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix, Vector}.
>> Implement above interfaces.
>>
>
> I have no opinion on that. What do others think ?
>
+1 for "dethroning" ;)
With the new interfaces, we could also drop the "Sparse" from the name -
i.e, "OpenMapRealMatrix"
I am wondering now whether similar marking and "dethroning" should
happen on the dense side - i.e DenseReal{Matrix, Vector},
BlockRealMatrix (currently DenseRealMatrix), ArrayRealMatrix (currently
RealMatrixImpl)?
>
>> - implementations should subclass the return type of some methods in the
>> Real{Matrix, Vector} API. e.g. RealVectorImpl.copy() returns a RealVector,
>> but could declare that it returns a RealVectorImpl. This will avoid needless
>> user casts. In future APIs, this could be a powerful way to define the
>> return type of matrix-matrix computation when the storage classes are
>> known... e.g. declaring that you return a DiagonalMatrix.
>>
>
> I didn't know changing the return type to a subtype was allowed when
> implementing an interface! This is for sure a good thing to do and could
> probably be used in many other places too.
>
+1 in general, but need to realize that when implemented, this changes
locks us in. In the specific cases mentioned above, this is a
no-brainer, but there are others (mostly in the decomp classes) where it
is not so obvious.
>
>> - is it too late to define a Norm enum and take that as a parameter, rather
>> than explicit methods for each Norm type?
>>
>
> There are many places where we use the same pattern outside of linear
> algebra. I'm reluctant to change this because if we extend the enum
> later, we may forget to add a new case in all implementations (somewhere
> in a switch statement), so we should add an exception for defensive
> programming. A method name enforced in an interface prevent such errors,
> you cannot compile your implementation if you forget a method.
>
Agree with Luc on this, but more from a Java style than maintenance
standpoint. We have tried in general to stay away from "behavior flags"
in method signatures up to now.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
getScarcity is probably still useful.
getSparseShapeOrClassOrWhateverItShouldBeCalled would be very, very helpful.
On Sat, May 16, 2009 at 11:12 AM, Sam Halliday <sa...@gmail.com>wrote:
> Luc Maisonobe wrote:
> >
> > Perhaps having a single additional method getSparcity() or
> > getLoadRatio() returning the ratio of elements set to the total number
> > as a double would be useful ?
> >
>
> Perhaps... but the shape is probably more important than the sparsity score
> (which doesn't necessarily mean anything, and might actually be difficult
> to
> calculate for some implementations). I recommend leaving it empty for now
> and we can discuss it in much more detail for 2.1 :-)
--
Ted Dunning, CTO
DeepDyve
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Sam Halliday wrote:
> Luc Maisonobe wrote:
>
>> Perhaps having a single additional method getSparcity() or
>> getLoadRatio() returning the ratio of elements set to the total number
>> as a double would be useful ?
>>
>>
>
> Perhaps... but the shape is probably more important than the sparsity score
> (which doesn't necessarily mean anything, and might actually be difficult to
> calculate for some implementations). I recommend leaving it empty for now
> and we can discuss it in much more detail for 2.1 :-)
>
+1 for leaving leaving RealSparse{Matrix, Vector} as markers (empty).
Defining a suitable shape property and exposing that is a good idea, but
will take some time and discussion. Also, if getSparcity() is of
limited value and may be hard to compute for some impls, we need think
carefully about putting it into the base interface. So leaving these as
markers seems reasonable.
Phil
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Luc Maisonobe wrote:
>
> Perhaps having a single additional method getSparcity() or
> getLoadRatio() returning the ratio of elements set to the total number
> as a double would be useful ?
>
Perhaps... but the shape is probably more important than the sparsity score
(which doesn't necessarily mean anything, and might actually be difficult to
calculate for some implementations). I recommend leaving it empty for now
and we can discuss it in much more detail for 2.1 :-)
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23576401.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
>
> Luc Maisonobe wrote:
>>> - create interfaces RealSparse{Matrix, Vector} to indicate a sparse
>>> storage
>>> implementation. Can be empty (for now).
>> I suppose these interfaces would extend Real{Matrix, Vector} ?
>>
>
> Exactly. In future, it might be wise to use the "subclass return trick" here
> too, but leaving it blank for now gives the best options for future
> releases.
Perhaps having a single additional method getSparcity() or
getLoadRatio() returning the ratio of elements set to the total number
as a double would be useful ?
>
>
> Luc Maisonobe wrote:
>> I didn't know changing the return type to a subtype was allowed when
>> implementing an interface! This is for sure a good thing to do and could
>> probably be used in many other places too.
>>
>
> Cool, eh? :-) In Joshua Bloch's book, he recommends being as general as
> possible in the parameters and as specific as possible in the return type.
> The only problem with being specific in the return type is that you are
> stuck with it. For that reason, it's probably best not to do this everywhere
> just yet. It's one of those little API details that can be open for change
> in the future whilst also being fully backwards compatible ;-)
>
>
> Luc Maisonobe wrote:
>>> - is it too late to define a Norm enum and take that as a parameter,
>>> rather
>>> than explicit methods for each Norm type?
>> There are many places where we use the same pattern outside of linear
>> algebra. I'm reluctant to change this because if we extend the enum
>> later...
>>
>
> OK, that makes sense.
>
> One of the first tasks for MTJ integration can be the inclusion of the many
> sparse storage types (and associated numerical optimisations). That's why
> I'm keen to have the current sparse implementation be specific about its
> "shape" in its name. Otherwise it'll be the odd one out ;-)
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Luc Maisonobe wrote:
>
>> - create interfaces RealSparse{Matrix, Vector} to indicate a sparse
>> storage
>> implementation. Can be empty (for now).
>
> I suppose these interfaces would extend Real{Matrix, Vector} ?
>
Exactly. In future, it might be wise to use the "subclass return trick" here
too, but leaving it blank for now gives the best options for future
releases.
Luc Maisonobe wrote:
>
> I didn't know changing the return type to a subtype was allowed when
> implementing an interface! This is for sure a good thing to do and could
> probably be used in many other places too.
>
Cool, eh? :-) In Joshua Bloch's book, he recommends being as general as
possible in the parameters and as specific as possible in the return type.
The only problem with being specific in the return type is that you are
stuck with it. For that reason, it's probably best not to do this everywhere
just yet. It's one of those little API details that can be open for change
in the future whilst also being fully backwards compatible ;-)
Luc Maisonobe wrote:
>
>> - is it too late to define a Norm enum and take that as a parameter,
>> rather
>> than explicit methods for each Norm type?
>
> There are many places where we use the same pattern outside of linear
> algebra. I'm reluctant to change this because if we extend the enum
> later...
>
OK, that makes sense.
One of the first tasks for MTJ integration can be the inclusion of the many
sparse storage types (and associated numerical optimisations). That's why
I'm keen to have the current sparse implementation be specific about its
"shape" in its name. Otherwise it'll be the odd one out ;-)
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23576105.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> I've had a quick look at the 2.0 API and the only changes I'd like to request
> are:-
>
> - create interfaces RealSparse{Matrix, Vector} to indicate a sparse storage
> implementation. Can be empty (for now).
I suppose these interfaces would extend Real{Matrix, Vector} ?
> - rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix, Vector}.
> Implement above interfaces.
I have no opinion on that. What do others think ?
> - implementations should subclass the return type of some methods in the
> Real{Matrix, Vector} API. e.g. RealVectorImpl.copy() returns a RealVector,
> but could declare that it returns a RealVectorImpl. This will avoid needless
> user casts. In future APIs, this could be a powerful way to define the
> return type of matrix-matrix computation when the storage classes are
> known... e.g. declaring that you return a DiagonalMatrix.
I didn't know changing the return type to a subtype was allowed when
implementing an interface! This is for sure a good thing to do and could
probably be used in many other places too.
> - is it too late to define a Norm enum and take that as a parameter, rather
> than explicit methods for each Norm type?
There are many places where we use the same pattern outside of linear
algebra. I'm reluctant to change this because if we extend the enum
later, we may forget to add a new case in all implementations (somewhere
in a switch statement), so we should add an exception for defensive
programming. A method name enforced in an interface prevent such errors,
you cannot compile your implementation if you forget a method.
>
> Other than that, looks good and I can't see any obvious conflicts that would
> void an MTJ merge! Most of the changes will be internal anyway, depending
> primarily on the path forward for BLAS/LAPACK use.
Fine.
Luc
>
>
> Sam Halliday wrote:
>> can I be involved in an API review to minimise any future problems? This
>> might involve some API changes from today's snapshot (although, no extra
>> features or anything heavy like that).
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I've had a quick look at the 2.0 API and the only changes I'd like to request
are:-
- create interfaces RealSparse{Matrix, Vector} to indicate a sparse storage
implementation. Can be empty (for now).
- rename SparseReal{Matrix, Vector} to OpenMapRealSparse{Matrix, Vector}.
Implement above interfaces.
- implementations should subclass the return type of some methods in the
Real{Matrix, Vector} API. e.g. RealVectorImpl.copy() returns a RealVector,
but could declare that it returns a RealVectorImpl. This will avoid needless
user casts. In future APIs, this could be a powerful way to define the
return type of matrix-matrix computation when the storage classes are
known... e.g. declaring that you return a DiagonalMatrix.
- is it too late to define a Norm enum and take that as a parameter, rather
than explicit methods for each Norm type?
Other than that, looks good and I can't see any obvious conflicts that would
void an MTJ merge! Most of the changes will be internal anyway, depending
primarily on the path forward for BLAS/LAPACK use.
Sam Halliday wrote:
>
> can I be involved in an API review to minimise any future problems? This
> might involve some API changes from today's snapshot (although, no extra
> features or anything heavy like that).
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23575831.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Sam Halliday a écrit :
>
>> I don't want to hold up a 2.0 release... however it would be a shame to
>> release an API that would restrict further inclusion of MTJ/BLAS, especially
>> in the sparse API. Would it be possible to hold back the linear algebra
>> updates for now?
>>
>> If you'd really like to get 2.0 out as it stands, then can I be involved in
>> an API review to minimise any future problems? This might involve some API
>> changes from today's snapshot (although, no extra features or anything heavy
>> like that).
>>
>
> Of course you can look at the API and comment on it or suggest changes.
> If they are wise and can be implemented quickly, we'll probably do it.
> If you want some changes to be included, your best bet is to explain
> them on the list and to provide patches for that using JIRA.
>
++1 - by all means, now is the time to comment on / suggest changes to
the 2.0 API.
Phil
>
>> I agree... this is far too big a task for 2.0!
>>
>>
>> Luc Maisonobe wrote:
>>
>>> However, as both Phil and I have said, we still want to have 2.0 out
>>> very soon. Adding a complete BLAS API is a huge task so I really prefer
>>> to wait after 2.0. We can discuss about it here, of course. You could
>>> also open a JIRA issue for requesting the enhancement, targeting 2.1.
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> I don't want to hold up a 2.0 release... however it would be a shame to
> release an API that would restrict further inclusion of MTJ/BLAS, especially
> in the sparse API. Would it be possible to hold back the linear algebra
> updates for now?
>
> If you'd really like to get 2.0 out as it stands, then can I be involved in
> an API review to minimise any future problems? This might involve some API
> changes from today's snapshot (although, no extra features or anything heavy
> like that).
Of course you can look at the API and comment on it or suggest changes.
If they are wise and can be implemented quickly, we'll probably do it.
If you want some changes to be included, your best bet is to explain
them on the list and to provide patches for that using JIRA.
>
> I agree... this is far too big a task for 2.0!
>
>
> Luc Maisonobe wrote:
>> However, as both Phil and I have said, we still want to have 2.0 out
>> very soon. Adding a complete BLAS API is a huge task so I really prefer
>> to wait after 2.0. We can discuss about it here, of course. You could
>> also open a JIRA issue for requesting the enhancement, targeting 2.1.
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I don't want to hold up a 2.0 release... however it would be a shame to
release an API that would restrict further inclusion of MTJ/BLAS, especially
in the sparse API. Would it be possible to hold back the linear algebra
updates for now?
If you'd really like to get 2.0 out as it stands, then can I be involved in
an API review to minimise any future problems? This might involve some API
changes from today's snapshot (although, no extra features or anything heavy
like that).
I agree... this is far too big a task for 2.0!
Luc Maisonobe wrote:
>
> However, as both Phil and I have said, we still want to have 2.0 out
> very soon. Adding a complete BLAS API is a huge task so I really prefer
> to wait after 2.0. We can discuss about it here, of course. You could
> also open a JIRA issue for requesting the enhancement, targeting 2.1.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23575674.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> Luc, if the Apache team are happy to include source generated by f2j (which
> is therefore BSD license) then there is no reason at all to have a
> dependency!
Fine.
>
> The generator code from netlib-java need not be distributed as part of the
> final commons-math binary, it is only needed to generate the .c files which
> allow for a native library at runtime. I would foresee the .c files being
> distributed as part of the commons-math binary download, with instructions
> on how to build the optional native library. The entire mechanism for doing
> this is entirely up for debate and review. The important thing is that there
> be a standardised BLAS/LAPACK API available.
Yes.
However, as both Phil and I have said, we still want to have 2.0 out
very soon. Adding a complete BLAS API is a huge task so I really prefer
to wait after 2.0. We can discuss about it here, of course. You could
also open a JIRA issue for requesting the enhancement, targeting 2.1.
Thanks
Luc
>
>
> Luc Maisonobe wrote:
>> Sam Halliday a écrit :
>>> I've somehow missed much of this discussion, which has got a little
>>> confused.
>>> I'll repeat some key facts here:-
>>>
>>> - MTJ depends on netlib-java
>>> - I'm the maintainer of netlib-java
>>> - netlib-java depends on PURE JAVA code, generated by F2J from netlib.org
>>> BLAS/LAPACK (and ARPACK). Keith Seymour (author of f2j) deserves all the
>>> praise for that magnificent task! The necessary jar is distributed with
>>> netlib-java.
>>> - BLAS/LAPACK are industry standard APIs.
>>> - netlib-java is technically a "translation" of netlib.org's
>>> BLAS/LAPACK/ARPACK API, so is therefore BSD licensed
>>> - netlib-java can be *optionally* configured at runtime to use a native
>>> library instead of the Java implementation.
>>> - the java implementation is pretty damn fast and will be more than
>>> adequate
>>> for most users. However, it will *never* be as fast as native code
>>> running
>>> on specialist hardware (no matter how much the JVM improves).
>>>
>>> Being the maintainer of netlib-java, I'd be more than happy to re-license
>>> all the bits that aren't technically "translations" of netlib.org, for
>>> inclusion in commons-math (in fact, it makes sense to do so). But you'd
>>> still need to depend on the f2j translated implementation. They are BSD
>>> license.
>> This is becoming more and more interesting. However, do yo think it
>> would be possible to "include" the source (either manually written or
>> automatically translated) into [math] ? This would allow a
>> self-contained package.
>>
>> We already provide some code which technically comes from translated
>> netlib routines, for example part of the Levenberg-Marquardt or almost
>> everything in the singular value decomposition. The Netlib license
>> allows that and we have set up the appropriate notices (see the javadoc
>> and the NOTICE.txt file).
>>
>>> Hell, it makes a *lot* of sense for commons-math to provide the
>>> BLAS/LAPACK
>>> API... they are industry standards after all, and all reference
>>> implementations for linear algebra algorithms make use of them.
>> I strongly approve that for BLAS. I dream of the BLAS API being
>> mandatory in JVM implementations, but this will probably never happen.
>> Considering LAPACK, I am less convinced because the API is strongly
>> fortran-oriented, not using some of the object-oriented features that
>> are well suited for mathematical concepts. The algorithms and their
>> implementations are very good, and we already use them inside, but with
>> a different API.
>>
>> Luc
>>
>>>
>>> Luc Maisonobe wrote:
>>>> I have an additional reason for avoiding native libraries. Pure Java can
>>>> be processed by external tools for either inspection (think findbugs,
>>>> cobertura, traceability, auditing) or modification (think Nabla!). The
>>>> Nabla case is especially important to me, but I am aware this is a
>>>> corner-case.
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Luc, if the Apache team are happy to include source generated by f2j (which
is therefore BSD license) then there is no reason at all to have a
dependency!
The generator code from netlib-java need not be distributed as part of the
final commons-math binary, it is only needed to generate the .c files which
allow for a native library at runtime. I would foresee the .c files being
distributed as part of the commons-math binary download, with instructions
on how to build the optional native library. The entire mechanism for doing
this is entirely up for debate and review. The important thing is that there
be a standardised BLAS/LAPACK API available.
Luc Maisonobe wrote:
>
> Sam Halliday a écrit :
>> I've somehow missed much of this discussion, which has got a little
>> confused.
>> I'll repeat some key facts here:-
>>
>> - MTJ depends on netlib-java
>> - I'm the maintainer of netlib-java
>> - netlib-java depends on PURE JAVA code, generated by F2J from netlib.org
>> BLAS/LAPACK (and ARPACK). Keith Seymour (author of f2j) deserves all the
>> praise for that magnificent task! The necessary jar is distributed with
>> netlib-java.
>> - BLAS/LAPACK are industry standard APIs.
>> - netlib-java is technically a "translation" of netlib.org's
>> BLAS/LAPACK/ARPACK API, so is therefore BSD licensed
>> - netlib-java can be *optionally* configured at runtime to use a native
>> library instead of the Java implementation.
>> - the java implementation is pretty damn fast and will be more than
>> adequate
>> for most users. However, it will *never* be as fast as native code
>> running
>> on specialist hardware (no matter how much the JVM improves).
>>
>> Being the maintainer of netlib-java, I'd be more than happy to re-license
>> all the bits that aren't technically "translations" of netlib.org, for
>> inclusion in commons-math (in fact, it makes sense to do so). But you'd
>> still need to depend on the f2j translated implementation. They are BSD
>> license.
>
> This is becoming more and more interesting. However, do yo think it
> would be possible to "include" the source (either manually written or
> automatically translated) into [math] ? This would allow a
> self-contained package.
>
> We already provide some code which technically comes from translated
> netlib routines, for example part of the Levenberg-Marquardt or almost
> everything in the singular value decomposition. The Netlib license
> allows that and we have set up the appropriate notices (see the javadoc
> and the NOTICE.txt file).
>
>>
>> Hell, it makes a *lot* of sense for commons-math to provide the
>> BLAS/LAPACK
>> API... they are industry standards after all, and all reference
>> implementations for linear algebra algorithms make use of them.
>
> I strongly approve that for BLAS. I dream of the BLAS API being
> mandatory in JVM implementations, but this will probably never happen.
> Considering LAPACK, I am less convinced because the API is strongly
> fortran-oriented, not using some of the object-oriented features that
> are well suited for mathematical concepts. The algorithms and their
> implementations are very good, and we already use them inside, but with
> a different API.
>
> Luc
>
>>
>>
>> Luc Maisonobe wrote:
>>> I have an additional reason for avoiding native libraries. Pure Java can
>>> be processed by external tools for either inspection (think findbugs,
>>> cobertura, traceability, auditing) or modification (think Nabla!). The
>>> Nabla case is especially important to me, but I am aware this is a
>>> corner-case.
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23575419.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I think netlib-java might actually be using the CLAPACK version of LAPACK...
the biggest problem with C/Fortran is the array indexing is different for
double[][]. CLAPACK addresses this.
LAPACK is still heavily used in reference implementations of standard
algorithms, although admittedly not as *core* as BLAS. The ARPACK API is
also worthwhile considering for inclusion (it's part of netlib-java and
f2j's translations).
Luc Maisonobe wrote:
>
> I strongly approve that for BLAS. I dream of the BLAS API being
> mandatory in JVM implementations, but this will probably never happen.
> Considering LAPACK, I am less convinced because the API is strongly
> fortran-oriented, not using some of the object-oriented features that
> are well suited for mathematical concepts. The algorithms and their
> implementations are very good, and we already use them inside, but with
> a different API.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23575455.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> I've somehow missed much of this discussion, which has got a little confused.
> I'll repeat some key facts here:-
>
> - MTJ depends on netlib-java
> - I'm the maintainer of netlib-java
> - netlib-java depends on PURE JAVA code, generated by F2J from netlib.org
> BLAS/LAPACK (and ARPACK). Keith Seymour (author of f2j) deserves all the
> praise for that magnificent task! The necessary jar is distributed with
> netlib-java.
> - BLAS/LAPACK are industry standard APIs.
> - netlib-java is technically a "translation" of netlib.org's
> BLAS/LAPACK/ARPACK API, so is therefore BSD licensed
> - netlib-java can be *optionally* configured at runtime to use a native
> library instead of the Java implementation.
> - the java implementation is pretty damn fast and will be more than adequate
> for most users. However, it will *never* be as fast as native code running
> on specialist hardware (no matter how much the JVM improves).
>
> Being the maintainer of netlib-java, I'd be more than happy to re-license
> all the bits that aren't technically "translations" of netlib.org, for
> inclusion in commons-math (in fact, it makes sense to do so). But you'd
> still need to depend on the f2j translated implementation. They are BSD
> license.
This is becoming more and more interesting. However, do yo think it
would be possible to "include" the source (either manually written or
automatically translated) into [math] ? This would allow a
self-contained package.
We already provide some code which technically comes from translated
netlib routines, for example part of the Levenberg-Marquardt or almost
everything in the singular value decomposition. The Netlib license
allows that and we have set up the appropriate notices (see the javadoc
and the NOTICE.txt file).
>
> Hell, it makes a *lot* of sense for commons-math to provide the BLAS/LAPACK
> API... they are industry standards after all, and all reference
> implementations for linear algebra algorithms make use of them.
I strongly approve that for BLAS. I dream of the BLAS API being
mandatory in JVM implementations, but this will probably never happen.
Considering LAPACK, I am less convinced because the API is strongly
fortran-oriented, not using some of the object-oriented features that
are well suited for mathematical concepts. The algorithms and their
implementations are very good, and we already use them inside, but with
a different API.
Luc
>
>
> Luc Maisonobe wrote:
>> I have an additional reason for avoiding native libraries. Pure Java can
>> be processed by external tools for either inspection (think findbugs,
>> cobertura, traceability, auditing) or modification (think Nabla!). The
>> Nabla case is especially important to me, but I am aware this is a
>> corner-case.
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I've somehow missed much of this discussion, which has got a little confused.
I'll repeat some key facts here:-
- MTJ depends on netlib-java
- I'm the maintainer of netlib-java
- netlib-java depends on PURE JAVA code, generated by F2J from netlib.org
BLAS/LAPACK (and ARPACK). Keith Seymour (author of f2j) deserves all the
praise for that magnificent task! The necessary jar is distributed with
netlib-java.
- BLAS/LAPACK are industry standard APIs.
- netlib-java is technically a "translation" of netlib.org's
BLAS/LAPACK/ARPACK API, so is therefore BSD licensed
- netlib-java can be *optionally* configured at runtime to use a native
library instead of the Java implementation.
- the java implementation is pretty damn fast and will be more than adequate
for most users. However, it will *never* be as fast as native code running
on specialist hardware (no matter how much the JVM improves).
Being the maintainer of netlib-java, I'd be more than happy to re-license
all the bits that aren't technically "translations" of netlib.org, for
inclusion in commons-math (in fact, it makes sense to do so). But you'd
still need to depend on the f2j translated implementation. They are BSD
license.
Hell, it makes a *lot* of sense for commons-math to provide the BLAS/LAPACK
API... they are industry standards after all, and all reference
implementations for linear algebra algorithms make use of them.
Luc Maisonobe wrote:
>
> I have an additional reason for avoiding native libraries. Pure Java can
> be processed by external tools for either inspection (think findbugs,
> cobertura, traceability, auditing) or modification (think Nabla!). The
> Nabla case is especially important to me, but I am aware this is a
> corner-case.
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23574535.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Phil Steitz a écrit :
> Ted Dunning wrote:
>> Phil, I think we have much of the same desires and motivations, but we
>> seem
>> to come to somewhat, but not entirely different conclusions.
>>
>> Assuming that (1) can be dealt with and assuming that (3) is already
>> dealt
>> with, do you still mind the inclusion of *optional*, automatically
>> generated
>> native code?
>>
> Part of 3) is having full code available with the package for
> inspection. That is part of the reason that we have avoided external
> dependencies. I would be open to making our fully self-contained, fully
> documented, fully open source library extensible to use other libraries,
> including native libraries, but I would not want to distribute anything
> associated with external libraries. The reason for this is the
> commitment made early on that all numerics and algorithms would be
> immediately visible to the user - no chasing down external, possibly
> incomplete or ambiguous docs to figure out what our code is doing.
I have an additional reason for avoiding native libraries. Pure Java can
be processed by external tools for either inspection (think findbugs,
cobertura, traceability, auditing) or modification (think Nabla!). The
Nabla case is especially important to me, but I am aware this is a
corner-case.
>> This page has some useful speed comparisons. For matrix-matrix
>> multiply up
>> to size 50, java is competitive. If you get up to roughly n = 500 or
>> 1000,
>> then heavily optimized native code can be up to 3x faster. Note the
>> line in
>> the third graph for colt (a reasonably well written pure java
>> implementation) and MTJ (which is running in pure java mode here). In my
>> case, I will generally opt for portability, but I like to have a portable
>> option for speed. It is also important to remember that numerical codes
>> more often need blinding speed than most other applications.
>>
> As Luc said, commons-math aims to be a general-purpose applied math
> package implementing good, well-documented, unencumbered numerical
> algorithms. I think this can be done in Java and we are doing it. We
> are never going to compete with optimized native code in speed, but
> strong numerics, JRE improvements and Moore's law are rapidly shrinking
> the class of real-world applications where the 3x difference above is
> material.
Perhaps we should have some benchmarks including our new linear package.
Something more serious than my little experiement with QR decomposition.
Unfortunately, I clearly have no time for it now. My current priotity is
to publish 2.0 as soon as possible and I am already late on my own schedule.
Luc
>
> Phil
>
>
>> http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
>>
>> Is that optional dependency really all that bad?
>>
>> On Fri, May 15, 2009 at 6:23 PM, Phil Steitz <ph...@gmail.com>
>> wrote:
>>
>>
>>> [math] also has currently zero dependencies. We had two dependencies on
>>>
>>>> other commons components up to version 1.2 and removed them when we
>>>> started work on version 2.0. Adding new dependencies, and especially
>>>> dependencies that involve native libraries is a difficult decision that
>>>> needs lots of discussion. We are currently trying to have 2.0 published
>>>> very soon now, such a decision would delay the publication several
>>>> months.
>>>>
>>>>
>>>>
>>> -1 for adding dependencies, especially on native code. Commons math
>>> needs
>>> to remain
>>>
>>> 1) ASL licensed
>>> 2) self-contained
>>> 3) fully documented, full open source
>>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Ted Dunning wrote:
> Phil, I think we have much of the same desires and motivations, but we seem
> to come to somewhat, but not entirely different conclusions.
>
> Assuming that (1) can be dealt with and assuming that (3) is already dealt
> with, do you still mind the inclusion of *optional*, automatically generated
> native code?
>
Part of 3) is having full code available with the package for
inspection. That is part of the reason that we have avoided external
dependencies. I would be open to making our fully self-contained, fully
documented, fully open source library extensible to use other libraries,
including native libraries, but I would not want to distribute anything
associated with external libraries. The reason for this is the
commitment made early on that all numerics and algorithms would be
immediately visible to the user - no chasing down external, possibly
incomplete or ambiguous docs to figure out what our code is doing.
> This page has some useful speed comparisons. For matrix-matrix multiply up
> to size 50, java is competitive. If you get up to roughly n = 500 or 1000,
> then heavily optimized native code can be up to 3x faster. Note the line in
> the third graph for colt (a reasonably well written pure java
> implementation) and MTJ (which is running in pure java mode here). In my
> case, I will generally opt for portability, but I like to have a portable
> option for speed. It is also important to remember that numerical codes
> more often need blinding speed than most other applications.
>
As Luc said, commons-math aims to be a general-purpose applied math
package implementing good, well-documented, unencumbered numerical
algorithms. I think this can be done in Java and we are doing it. We
are never going to compete with optimized native code in speed, but
strong numerics, JRE improvements and Moore's law are rapidly shrinking
the class of real-world applications where the 3x difference above is
material.
Phil
> http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
>
> Is that optional dependency really all that bad?
>
> On Fri, May 15, 2009 at 6:23 PM, Phil Steitz <ph...@gmail.com> wrote:
>
>
>> [math] also has currently zero dependencies. We had two dependencies on
>>
>>> other commons components up to version 1.2 and removed them when we
>>> started work on version 2.0. Adding new dependencies, and especially
>>> dependencies that involve native libraries is a difficult decision that
>>> needs lots of discussion. We are currently trying to have 2.0 published
>>> very soon now, such a decision would delay the publication several months.
>>>
>>>
>>>
>> -1 for adding dependencies, especially on native code. Commons math needs
>> to remain
>>
>> 1) ASL licensed
>> 2) self-contained
>> 3) fully documented, full open source
>>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
Phil, I think we have much of the same desires and motivations, but we seem
to come to somewhat, but not entirely different conclusions.
Assuming that (1) can be dealt with and assuming that (3) is already dealt
with, do you still mind the inclusion of *optional*, automatically generated
native code?
This page has some useful speed comparisons. For matrix-matrix multiply up
to size 50, java is competitive. If you get up to roughly n = 500 or 1000,
then heavily optimized native code can be up to 3x faster. Note the line in
the third graph for colt (a reasonably well written pure java
implementation) and MTJ (which is running in pure java mode here). In my
case, I will generally opt for portability, but I like to have a portable
option for speed. It is also important to remember that numerical codes
more often need blinding speed than most other applications.
http://blog.mikiobraun.de/2009/04/some-benchmark-numbers-for-jblas.html
Is that optional dependency really all that bad?
On Fri, May 15, 2009 at 6:23 PM, Phil Steitz <ph...@gmail.com> wrote:
> [math] also has currently zero dependencies. We had two dependencies on
>> other commons components up to version 1.2 and removed them when we
>> started work on version 2.0. Adding new dependencies, and especially
>> dependencies that involve native libraries is a difficult decision that
>> needs lots of discussion. We are currently trying to have 2.0 published
>> very soon now, such a decision would delay the publication several months.
>>
>>
> -1 for adding dependencies, especially on native code. Commons math needs
> to remain
>
> 1) ASL licensed
> 2) self-contained
> 3) fully documented, full open source
--
Ted Dunning, CTO
DeepDyve
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Ted Dunning a écrit :
>
>> On Thu, May 14, 2009 at 3:18 AM, Sam Halliday <sa...@gmail.com>wrote:
>>
>>
>>> I am a maintainer of the matrix-toolkits-java
>>>
>> Which is an impressive piece of work, especially the transparent but
>> non-binding interface to the Atlas and Blas native packages. My compliments
>> to Bjørn-Ove and all who have followed up on his original work.
>>
>> This note is in regard to the commons-math library. It is clear that our
>>
>>> projects dovetail, especially when I look at "linear" in version 2.0 of the
>>> API. It would be good if we could either complement or consolidate efforts,
>>> rather than reproduce.
>>>
>> That sounds good to me.
>>
>
> You are right.
>
>
>> As a start, I'd like to discourage the use of a solid implementation for
>>
>>> SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
>>> implementations based on the Templates project:-
>>>
>
> This is exactly the purpose of the RealMatrix/RealVector and FieldMatrix
> /FieldVector interfaces on one side and of the CholeskiDecomposition,
> EigenDecomposition, LUDecomposition, QRDecomposition,
> SingularValueDecomposition and Decompositionsolver interfaces on the
> other side.
>
> RealMatrix (resp. FieldMatrix) is the top level interfaces that does not
> mandate any specific storage. It has several implementations
> (RealMatrixImpl with a simple double[][] array, DenseRealMatrix with a
> block implementation, SparseRealMatrix). We also have in mind (not
> implemented yet) things like DiagonalRealMatrix or BandRealMatrix or
> Lower/UpperTriangularRealMatrix. All implementations can be mixed
> together and specific cases are automatically detected and handled to
> avoid using the general embedded loops and use smart algorithms when
> possible. There was also an attempt using recursive layouts (with
> Gray-Morton space filling curve).
>
> Maybe SparseRealMatrix was a bad name and should have been
> SimpleSparseRealMatrix to avoid confusion with other sparse storage and
> dedicated algorithms.
>
>
>> Can you say more about what aspects of the Templates project you feel are
>> important? You mention one case of storage layout.
>>
>>
>>
>>> I believe commons-math should move to a netlib-java backend (allowing the
>>> use of machine optimised BLAS/LAPACK).
>>>
>> This is an interesting suggestion. Obviously adopting MTJ wholesale would
>> accomplish that.
>>
>> Can you say something about the licensing issues if we were to explore, for
>> discussion sake, MTJ being folded into commons-math? MTJ is LGPL while
>> commons has to stay Apache licensed. This licensing issue has been the
>> biggest sticking point in the past.
>>
>
> This is really an issue. Apache projects cannot use LGPL (or GPL) code.
> See http://www.apache.org/legal/resolved.html for the policy.
>
> [math] also has currently zero dependencies. We had two dependencies on
> other commons components up to version 1.2 and removed them when we
> started work on version 2.0. Adding new dependencies, and especially
> dependencies that involve native libraries is a difficult decision that
> needs lots of discussion. We are currently trying to have 2.0 published
> very soon now, such a decision would delay the publication several months.
>
-1 for adding dependencies, especially on native code. Commons math
needs to remain
1) ASL licensed
2) self-contained
3) fully documented, full open source
Phil
> Some benchmarks I did a few weeks ago showed the new [math] linear
> package implementation was quite fast and compared very well with native
> fortran libraries for QR decomposition with similar non-blocked
> algorithms. In fact, it was 7 times faster than unoptimized numerical
> recipes, about 2 or 3 times faster than optimized numerical recipes, and
> very slightly (a few percents) faster than optimized lapack with Atlas
> as the BLAS implementation. Faster QR decomposition required changing
> the algorithm so the blocked lapack implementation was the only native
> implementation faster than [math]. Of course, I now do want to also
> implement a blocked QR decomposition in [math] ...
>
> I am aware that we still lack lots of very efficient linear algebra
> algorithms. Joining efforts with you would be a real gain if we can
> solve the licensing issues and avoid new dependencies if possible.
> [math] has already adopted a lot of external code, even complete
> libraries. I came in by donating the whole mantissa library, merging it
> in and now contributing to the maintainance of the component with the
> other developers.
>
> Luc
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Ted Dunning a écrit :
> Dang.
>
> That is fast.
>
> What size matrices was this for?
>
> On Thu, May 14, 2009 at 12:09 PM, Luc Maisonobe <Lu...@free.fr>wrote:
>
>> Some benchmarks I did a few weeks ago showed the new [math] linear
>> package implementation was quite fast and compared very well with native
>> fortran libraries for QR decomposition with similar non-blocked
>> algorithms. In fact, it was 7 times faster than unoptimized numerical
>> recipes, about 2 or 3 times faster than optimized numerical recipes, and
>> very slightly (a few percents) faster than optimized lapack with Atlas
>> as the BLAS implementation. Faster QR decomposition required changing
>> the algorithm so the blocked lapack implementation was the only native
>> implementation faster than [math].
>>
>
Medium size, up to 600 if I remember correctly (I don't have the curve
available here). The pattern with O(n^3) was clearly visible so the
result can be be extrapolated for medium size I think. I did not check
larger matrices like 3000 or more, results may be different.
Beware, that was one algorithm only: QR decomposition and on one host
only (AMD64 phenom quad core, Linux, sun Java 6 on one side, Gnu fortran
on the other side). GNU fortran is not a very fast compiler so the
result would obviously be very different with a better compiler. The
purpose of this test was not to say that Java is faster, it was to show
that the performance difference between Java and fortran was smaller
than the difference you get by changing algorithms (in this case blocked
vs. non-blocked). I was surprised by the results.
For other kind of algorithms, mixing linear algebra, trigonometric
computation, ODE integration, root searching ... I often see performance
differences of about a factor 2, which in my opinion is a factor similar
to other changes one can do (algorithms, CPU, BLAS, compiler,
parallelism, memory ...). Of course, when comparing an highly optimized
algorithm with the perfect cache size, and the best compiler to a
standard setting we can get very different factors. So do not take these
numbers for granted.
Trying to get the fastest computation is not the purpose of [math]. It
should be reasonably efficient with respect to other libraries,
including native ones, but is not dedicated to speed. It should remain a
general-purpose library.
Luc
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
Dang.
That is fast.
What size matrices was this for?
On Thu, May 14, 2009 at 12:09 PM, Luc Maisonobe <Lu...@free.fr>wrote:
> Some benchmarks I did a few weeks ago showed the new [math] linear
> package implementation was quite fast and compared very well with native
> fortran libraries for QR decomposition with similar non-blocked
> algorithms. In fact, it was 7 times faster than unoptimized numerical
> recipes, about 2 or 3 times faster than optimized numerical recipes, and
> very slightly (a few percents) faster than optimized lapack with Atlas
> as the BLAS implementation. Faster QR decomposition required changing
> the algorithm so the blocked lapack implementation was the only native
> implementation faster than [math].
>
Re: [math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Ted Dunning a écrit :
> On Thu, May 14, 2009 at 3:18 AM, Sam Halliday <sa...@gmail.com>wrote:
>
>> I am a maintainer of the matrix-toolkits-java
>
>
> Which is an impressive piece of work, especially the transparent but
> non-binding interface to the Atlas and Blas native packages. My compliments
> to Bjørn-Ove and all who have followed up on his original work.
>
> This note is in regard to the commons-math library. It is clear that our
>> projects dovetail, especially when I look at "linear" in version 2.0 of the
>> API. It would be good if we could either complement or consolidate efforts,
>> rather than reproduce.
>
>
> That sounds good to me.
You are right.
>
> As a start, I'd like to discourage the use of a solid implementation for
>> SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
>> implementations based on the Templates project:-
This is exactly the purpose of the RealMatrix/RealVector and FieldMatrix
/FieldVector interfaces on one side and of the CholeskiDecomposition,
EigenDecomposition, LUDecomposition, QRDecomposition,
SingularValueDecomposition and Decompositionsolver interfaces on the
other side.
RealMatrix (resp. FieldMatrix) is the top level interfaces that does not
mandate any specific storage. It has several implementations
(RealMatrixImpl with a simple double[][] array, DenseRealMatrix with a
block implementation, SparseRealMatrix). We also have in mind (not
implemented yet) things like DiagonalRealMatrix or BandRealMatrix or
Lower/UpperTriangularRealMatrix. All implementations can be mixed
together and specific cases are automatically detected and handled to
avoid using the general embedded loops and use smart algorithms when
possible. There was also an attempt using recursive layouts (with
Gray-Morton space filling curve).
Maybe SparseRealMatrix was a bad name and should have been
SimpleSparseRealMatrix to avoid confusion with other sparse storage and
dedicated algorithms.
>
>
> Can you say more about what aspects of the Templates project you feel are
> important? You mention one case of storage layout.
>
>
>> I believe commons-math should move to a netlib-java backend (allowing the
>> use of machine optimised BLAS/LAPACK).
>
>
> This is an interesting suggestion. Obviously adopting MTJ wholesale would
> accomplish that.
>
> Can you say something about the licensing issues if we were to explore, for
> discussion sake, MTJ being folded into commons-math? MTJ is LGPL while
> commons has to stay Apache licensed. This licensing issue has been the
> biggest sticking point in the past.
This is really an issue. Apache projects cannot use LGPL (or GPL) code.
See http://www.apache.org/legal/resolved.html for the policy.
[math] also has currently zero dependencies. We had two dependencies on
other commons components up to version 1.2 and removed them when we
started work on version 2.0. Adding new dependencies, and especially
dependencies that involve native libraries is a difficult decision that
needs lots of discussion. We are currently trying to have 2.0 published
very soon now, such a decision would delay the publication several months.
Some benchmarks I did a few weeks ago showed the new [math] linear
package implementation was quite fast and compared very well with native
fortran libraries for QR decomposition with similar non-blocked
algorithms. In fact, it was 7 times faster than unoptimized numerical
recipes, about 2 or 3 times faster than optimized numerical recipes, and
very slightly (a few percents) faster than optimized lapack with Atlas
as the BLAS implementation. Faster QR decomposition required changing
the algorithm so the blocked lapack implementation was the only native
implementation faster than [math]. Of course, I now do want to also
implement a blocked QR decomposition in [math] ...
I am aware that we still lack lots of very efficient linear algebra
algorithms. Joining efforts with you would be a real gain if we can
solve the licensing issues and avoid new dependencies if possible.
[math] has already adopted a lot of external code, even complete
libraries. I came in by donating the whole mantissa library, merging it
in and now contributing to the maintainance of the component with the
other developers.
Luc
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
[math] Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
On Thu, May 14, 2009 at 3:18 AM, Sam Halliday <sa...@gmail.com>wrote:
>
> I am a maintainer of the matrix-toolkits-java
Which is an impressive piece of work, especially the transparent but
non-binding interface to the Atlas and Blas native packages. My compliments
to Bjørn-Ove and all who have followed up on his original work.
This note is in regard to the commons-math library. It is clear that our
> projects dovetail, especially when I look at "linear" in version 2.0 of the
> API. It would be good if we could either complement or consolidate efforts,
> rather than reproduce.
That sounds good to me.
As a start, I'd like to discourage the use of a solid implementation for
> SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
> implementations based on the Templates project:-
Can you say more about what aspects of the Templates project you feel are
important? You mention one case of storage layout.
> I believe commons-math should move to a netlib-java backend (allowing the
> use of machine optimised BLAS/LAPACK).
This is an interesting suggestion. Obviously adopting MTJ wholesale would
accomplish that.
Can you say something about the licensing issues if we were to explore, for
discussion sake, MTJ being folded into commons-math? MTJ is LGPL while
commons has to stay Apache licensed. This licensing issue has been the
biggest sticking point in the past.
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
I discovered/remembered something that might make the inclusion of Java
source translated BLAS/LAPACK code troublesome. The F2J translator does a
two step process... it generates Java from the Fortran, but then has to do
some jiggery-pokery to the class file to handle GOTO statements. That means
any source code would not compile correctly without an additional
post-compilation stage.
The obvious answer is to just depend on the translated binaries... which
could of course be distributed with commons-math. The F2J translator allows
the namespace to be specified, so there is no reason the translated binaries
couldn't be "org.apache.commons.math.{blas, lapack}" etc.
I'm currently asking the author if he can create a Maven bundle of the
resulting binaries. That leaves external dependency an option. I'd greatly
appreciate anybody contacting me off-list to help with this if they have any
Maven experience!
Sam Halliday wrote:
>
> Dear all,
>
> I am a maintainer of the matrix-toolkits-java
>
> http://code.google.com/p/matrix-toolkits-java/
>
> which is a comprehensive collection of matrix data structures, linear
> solvers, least squares methods, eigenvalue and singular value
> decompositions.
>
> This note is in regard to the commons-math library. It is clear that our
> projects dovetail, especially when I look at "linear" in version 2.0 of
> the API. It would be good if we could either complement or consolidate
> efforts, rather than reproduce.
>
> It would be excellent if all the functionality of matrix-toolkits-java
> were available as part of commons-math. There is already too much
> diversity and un-maintained maths code out there for Java!
>
> As a start, I'd like to discourage the use of a solid implementation for
> SparseReal{Vector, Matrix}... please prefer an interface approach,
> allowing implementations based on the Templates project:-
>
> http://www.netlib.org/templates
>
> The reason is that the storage implementation should be related to the
> type of data being stored. For example, there are many well-known kinds of
> sparse matrix that are well suited to particular kinds of calculations...
> consider multiplying sparse matrices that you know to be diagonal!
>
> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
> thinking about linear algebra and have set up unrivalled standard APIs
> which have been implemented right down to the architecture level. It would
> be a major mistake if commons-math didn't build on their good work.
>
> I believe commons-math should move to a netlib-java backend (allowing the
> use of machine optimised BLAS/LAPACK).
>
> http://code.google.com/p/netlib-java/
>
> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
> former should be possible with some work within the current API, but I
> fear major API changes would be needed for the latter. I do not want the
> commons-math API to walk into this trap without having first considered
> future architectures! MTJ has a distributed package, but I am not sure if
> this is something that is completely future proof either.
>
> What say ye'?
>
> --
> Sam
>
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23766139.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by "Edward J. Yoon" <ed...@apache.org>.
Oh,.. That's really cool.
On Sat, May 23, 2009 at 4:02 PM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> On Fri, May 22, 2009 at 6:17 PM, Edward J. Yoon <ed...@apache.org> wrote:
>> What do you think about below?
>
> i been wonder for a while whether it's time for a Apache Math TLP...
>
> - robert
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Fri, May 22, 2009 at 6:17 PM, Edward J. Yoon <ed...@apache.org> wrote:
> What do you think about below?
i been wonder for a while whether it's time for a Apache Math TLP...
- robert
Fwd: commons-math, matrix-toolkits-java and consolidation
Posted by "Edward J. Yoon" <ed...@apache.org>.
What do you think about below?
---------- Forwarded message ----------
From: Sam Halliday <sa...@gmail.com>
Date: Fri, May 22, 2009 at 7:37 PM
Subject: Re: commons-math, matrix-toolkits-java and consolidation
To: dev@commons.apache.org
Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
with commons-math. However, there might be parts of it that are relevant to
Hama.
We should absolutely ensure that the "ducks are aligned" between
commons-math and Hama. It would be win-win if Hama were able to use
commons-math's linear interfaces. commons-math will hopefully be moving to
use a netlib-java style backend (not user facing), we should ensure that
ScaLAPACK is handled in the same way.
I have some ideas of people who might be interested... I've encountered them
as maintainer of MTJ, but never worked with them. I'll dig through my e-mail
archive and make the intros.
Edward J. Yoon-2 wrote:
>
> That's really cool.
>
> BTW, Can I ask about the plan of data distribution strategies of your
> 'distributed' package in the future? IMO, it seems, it doesn't sit
> well with 'common-math' project.
>
> If there is a developer who wants to implement 'distributed', pls
> let us know, too. I'm working for the Hama
> (http://incubator.apache.org/hama) with ScaLAPACK members.
>
> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com>
> wrote:
>>
>> Dear all,
>>
>> I am a maintainer of the matrix-toolkits-java
>>
>> http://code.google.com/p/matrix-toolkits-java/
>>
>> which is a comprehensive collection of matrix data structures, linear
>> solvers, least squares methods, eigenvalue and singular value
>> decompositions.
>>
>> This note is in regard to the commons-math library. It is clear that our
>> projects dovetail, especially when I look at "linear" in version 2.0 of
>> the
>> API. It would be good if we could either complement or consolidate
>> efforts,
>> rather than reproduce.
>>
>> It would be excellent if all the functionality of matrix-toolkits-java
>> were
>> available as part of commons-math. There is already too much diversity
>> and
>> un-maintained maths code out there for Java!
>>
>> As a start, I'd like to discourage the use of a solid implementation for
>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>> allowing
>> implementations based on the Templates project:-
>>
>> http://www.netlib.org/templates
>>
>> The reason is that the storage implementation should be related to the
>> type
>> of data being stored. For example, there are many well-known kinds of
>> sparse
>> matrix that are well suited to particular kinds of calculations...
>> consider
>> multiplying sparse matrices that you know to be diagonal!
>>
>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>> thinking about linear algebra and have set up unrivalled standard APIs
>> which
>> have been implemented right down to the architecture level. It would be a
>> major mistake if commons-math didn't build on their good work.
>>
>> I believe commons-math should move to a netlib-java backend (allowing the
>> use of machine optimised BLAS/LAPACK).
>>
>> http://code.google.com/p/netlib-java/
>>
>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>> former should be possible with some work within the current API, but I
>> fear
>> major API changes would be needed for the latter. I do not want the
>> commons-math API to walk into this trap without having first considered
>> future architectures! MTJ has a distributed package, but I am not sure if
>> this is something that is completely future proof either.
>>
>> What say ye'?
>>
>> --
>> Sam
>>
>> --
>> View this message in context:
>> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
>> Sent from the Commons - Dev mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
>
--
View this message in context:
http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23668079.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
-1 to a TLP at the current time.
I really agree that the current momentum may be a flash.
On Sat, May 23, 2009 at 6:28 AM, Jin Mingjian <ji...@gmail.com> wrote:
> I like the idea about the "top level math project":) Is it possible or
> interesting to host a sub-project for DSL to commons-math?(such as an
> enhanced Object-supported matlab-compatible script)
>
>
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Jin Mingjian <ji...@gmail.com>.
I like the idea about the "top level math project":) Is it possible or
interesting to host a sub-project for DSL to commons-math?(such as an
enhanced Object-supported matlab-compatible script)
2009/5/23 Edward J. Yoon <ed...@apache.org>
> Hmm. It's a really great idea.
>
> I think It could be a top level math project of apache.
>
> On Fri, May 22, 2009 at 7:37 PM, Sam Halliday <sa...@gmail.com>
> wrote:
> >
> > Edward, Hama looks fantastic! I fully agree, 'distributed' isn't
> agreeable
> > with commons-math. However, there might be parts of it that are relevant
> to
> > Hama.
> >
> > We should absolutely ensure that the "ducks are aligned" between
> > commons-math and Hama. It would be win-win if Hama were able to use
> > commons-math's linear interfaces. commons-math will hopefully be moving
> to
> > use a netlib-java style backend (not user facing), we should ensure that
> > ScaLAPACK is handled in the same way.
> >
> > I have some ideas of people who might be interested... I've encountered
> them
> > as maintainer of MTJ, but never worked with them. I'll dig through my
> e-mail
> > archive and make the intros.
> >
> >
> > Edward J. Yoon-2 wrote:
> >>
> >> That's really cool.
> >>
> >> BTW, Can I ask about the plan of data distribution strategies of your
> >> 'distributed' package in the future? IMO, it seems, it doesn't sit
> >> well with 'common-math' project.
> >>
> >> If if there is a developer who wants to implement 'distributed', pls
> >> let us know, too. I'm working for the Hama
> >> (http://incubator.apache.org/hama) with ScaLAPACK members.
> >>
> >> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com>
> >> wrote:
> >>>
> >>> Dear all,
> >>>
> >>> I am a maintainer of the matrix-toolkits-java
> >>>
> >>> http://code.google.com/p/matrix-toolkits-java/
> >>>
> >>> which is a comprehensive collection of matrix data structures, linear
> >>> solvers, least squares methods, eigenvalue and singular value
> >>> decompositions.
> >>>
> >>> This note is in regard to the commons-math library. It is clear that
> our
> >>> projects dovetail, especially when I look at "linear" in version 2.0 of
> >>> the
> >>> API. It would be good if we could either complement or consolidate
> >>> efforts,
> >>> rather than reproduce.
> >>>
> >>> It would be excellent if all the functionality of matrix-toolkits-java
> >>> were
> >>> available as part of commons-math. There is already too much diversity
> >>> and
> >>> un-maintained maths code out there for Java!
> >>>
> >>> As a start, I'd like to discourage the use of a solid implementation
> for
> >>> SparseReal{Vector, Matrix}... please prefer an interface approach,
> >>> allowing
> >>> implementations based on the Templates project:-
> >>>
> >>> http://www.netlib.org/templates
> >>>
> >>> The reason is that the storage implementation should be related to the
> >>> type
> >>> of data being stored. For example, there are many well-known kinds of
> >>> sparse
> >>> matrix that are well suited to particular kinds of calculations...
> >>> consider
> >>> multiplying sparse matrices that you know to be diagonal!
> >>>
> >>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of
> time
> >>> thinking about linear algebra and have set up unrivalled standard APIs
> >>> which
> >>> have been implemented right down to the architecture level. It would be
> a
> >>> major mistake if commons-math didn't build on their good work.
> >>>
> >>> I believe commons-math should move to a netlib-java backend (allowing
> the
> >>> use of machine optimised BLAS/LAPACK).
> >>>
> >>> http://code.google.com/p/netlib-java/
> >>>
> >>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
> >>> scalability to parallel architectures which use Parallel BLAS/LAPACK.
> The
> >>> former should be possible with some work within the current API, but I
> >>> fear
> >>> major API changes would be needed for the latter. I do not want the
> >>> commons-math API to walk into this trap without having first considered
> >>> future architectures! MTJ has a distributed package, but I am not sure
> if
> >>> this is something that is completely future proof either.
> >>>
> >>> What say ye'?
> >>>
> >>> --
> >>> Sam
> >>>
> >>> --
> >>> View this message in context:
> >>>
> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
> >>> Sent from the Commons - Dev mailing list archive at Nabble.com.
> >>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>
> >>>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon @ NHN, corp.
> >> edwardyoon@apache.org
> >> http://blog.udanax.org
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >>
> >
> > --
> > View this message in context:
> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23668079.html
> > Sent from the Commons - Dev mailing list archive at Nabble.com.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: commons-math, matrix-toolkits-java and consolidation
Posted by "Edward J. Yoon" <ed...@apache.org>.
Hmm. It's a really great idea.
I think It could be a top level math project of apache.
On Fri, May 22, 2009 at 7:37 PM, Sam Halliday <sa...@gmail.com> wrote:
>
> Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
> with commons-math. However, there might be parts of it that are relevant to
> Hama.
>
> We should absolutely ensure that the "ducks are aligned" between
> commons-math and Hama. It would be win-win if Hama were able to use
> commons-math's linear interfaces. commons-math will hopefully be moving to
> use a netlib-java style backend (not user facing), we should ensure that
> ScaLAPACK is handled in the same way.
>
> I have some ideas of people who might be interested... I've encountered them
> as maintainer of MTJ, but never worked with them. I'll dig through my e-mail
> archive and make the intros.
>
>
> Edward J. Yoon-2 wrote:
>>
>> That's really cool.
>>
>> BTW, Can I ask about the plan of data distribution strategies of your
>> 'distributed' package in the future? IMO, it seems, it doesn't sit
>> well with 'common-math' project.
>>
>> If if there is a developer who wants to implement 'distributed', pls
>> let us know, too. I'm working for the Hama
>> (http://incubator.apache.org/hama) with ScaLAPACK members.
>>
>> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com>
>> wrote:
>>>
>>> Dear all,
>>>
>>> I am a maintainer of the matrix-toolkits-java
>>>
>>> http://code.google.com/p/matrix-toolkits-java/
>>>
>>> which is a comprehensive collection of matrix data structures, linear
>>> solvers, least squares methods, eigenvalue and singular value
>>> decompositions.
>>>
>>> This note is in regard to the commons-math library. It is clear that our
>>> projects dovetail, especially when I look at "linear" in version 2.0 of
>>> the
>>> API. It would be good if we could either complement or consolidate
>>> efforts,
>>> rather than reproduce.
>>>
>>> It would be excellent if all the functionality of matrix-toolkits-java
>>> were
>>> available as part of commons-math. There is already too much diversity
>>> and
>>> un-maintained maths code out there for Java!
>>>
>>> As a start, I'd like to discourage the use of a solid implementation for
>>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>>> allowing
>>> implementations based on the Templates project:-
>>>
>>> http://www.netlib.org/templates
>>>
>>> The reason is that the storage implementation should be related to the
>>> type
>>> of data being stored. For example, there are many well-known kinds of
>>> sparse
>>> matrix that are well suited to particular kinds of calculations...
>>> consider
>>> multiplying sparse matrices that you know to be diagonal!
>>>
>>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>>> thinking about linear algebra and have set up unrivalled standard APIs
>>> which
>>> have been implemented right down to the architecture level. It would be a
>>> major mistake if commons-math didn't build on their good work.
>>>
>>> I believe commons-math should move to a netlib-java backend (allowing the
>>> use of machine optimised BLAS/LAPACK).
>>>
>>> http://code.google.com/p/netlib-java/
>>>
>>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>>> former should be possible with some work within the current API, but I
>>> fear
>>> major API changes would be needed for the latter. I do not want the
>>> commons-math API to walk into this trap without having first considered
>>> future architectures! MTJ has a distributed package, but I am not sure if
>>> this is something that is completely future proof either.
>>>
>>> What say ye'?
>>>
>>> --
>>> Sam
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
>>> Sent from the Commons - Dev mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23668079.html
> Sent from the Commons - Dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Ted Dunning <te...@gmail.com>.
Hama is a proposed distributed matrix implementation system that is based on
using map-reduce to implement basic matrix operations and uses hbase to
store matrices. Getting useful performance out of this substrate for dense
matrix operations is likely to be fairly challenging due to I/O costs. For
sparse operations that exceed memory size, it may be more attractive.
Hama has been around for nearly two years. So far, it appears that there is
an implementation of matrix multiply and add. Performance numbers are
underwhelming for dense matrices. On sample problem of multiplying 5000 x
5000 random matrices, hama achieves a speed on 8 workstations that is about
1/3 of the speed of R running on a laptop.
Performance on sparse matrices may be better.
On Fri, May 22, 2009 at 10:13 AM, Edward J. Yoon <ed...@apache.org>wrote:
> > Is Hama related to Hadoop ?
>
> Yes, it is.
--
Ted Dunning, CTO
DeepDyve
Re: commons-math, matrix-toolkits-java and consolidation
Posted by "Edward J. Yoon" <ed...@apache.org>.
> Is Hama related to Hadoop ?
Yes, it is.
On Fri, May 22, 2009 at 8:05 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> Sam Halliday a écrit :
>> Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
>> with commons-math. However, there might be parts of it that are relevant to
>> Hama.
>
> Is Hama related to Hadoop ?
>
>>
>> We should absolutely ensure that the "ducks are aligned" between
>> commons-math and Hama. It would be win-win if Hama were able to use
>> commons-math's linear interfaces. commons-math will hopefully be moving to
>> use a netlib-java style backend (not user facing), we should ensure that
>> ScaLAPACK is handled in the same way.
>>
>> I have some ideas of people who might be interested... I've encountered them
>> as maintainer of MTJ, but never worked with them. I'll dig through my e-mail
>> archive and make the intros.
>>
>>
>> Edward J. Yoon-2 wrote:
>>> That's really cool.
>>>
>>> BTW, Can I ask about the plan of data distribution strategies of your
>>> 'distributed' package in the future? IMO, it seems, it doesn't sit
>>> well with 'common-math' project.
>>>
>>> If if there is a developer who wants to implement 'distributed', pls
>>> let us know, too. I'm working for the Hama
>>> (http://incubator.apache.org/hama) with ScaLAPACK members.
>>>
>>> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com>
>>> wrote:
>>>> Dear all,
>>>>
>>>> I am a maintainer of the matrix-toolkits-java
>>>>
>>>> http://code.google.com/p/matrix-toolkits-java/
>>>>
>>>> which is a comprehensive collection of matrix data structures, linear
>>>> solvers, least squares methods, eigenvalue and singular value
>>>> decompositions.
>>>>
>>>> This note is in regard to the commons-math library. It is clear that our
>>>> projects dovetail, especially when I look at "linear" in version 2.0 of
>>>> the
>>>> API. It would be good if we could either complement or consolidate
>>>> efforts,
>>>> rather than reproduce.
>>>>
>>>> It would be excellent if all the functionality of matrix-toolkits-java
>>>> were
>>>> available as part of commons-math. There is already too much diversity
>>>> and
>>>> un-maintained maths code out there for Java!
>>>>
>>>> As a start, I'd like to discourage the use of a solid implementation for
>>>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>>>> allowing
>>>> implementations based on the Templates project:-
>>>>
>>>> http://www.netlib.org/templates
>>>>
>>>> The reason is that the storage implementation should be related to the
>>>> type
>>>> of data being stored. For example, there are many well-known kinds of
>>>> sparse
>>>> matrix that are well suited to particular kinds of calculations...
>>>> consider
>>>> multiplying sparse matrices that you know to be diagonal!
>>>>
>>>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>>>> thinking about linear algebra and have set up unrivalled standard APIs
>>>> which
>>>> have been implemented right down to the architecture level. It would be a
>>>> major mistake if commons-math didn't build on their good work.
>>>>
>>>> I believe commons-math should move to a netlib-java backend (allowing the
>>>> use of machine optimised BLAS/LAPACK).
>>>>
>>>> http://code.google.com/p/netlib-java/
>>>>
>>>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>>>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>>>> former should be possible with some work within the current API, but I
>>>> fear
>>>> major API changes would be needed for the latter. I do not want the
>>>> commons-math API to walk into this trap without having first considered
>>>> future architectures! MTJ has a distributed package, but I am not sure if
>>>> this is something that is completely future proof either.
>>>>
>>>> What say ye'?
>>>>
>>>> --
>>>> Sam
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
>>>> Sent from the Commons - Dev mailing list archive at Nabble.com.
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon @ NHN, corp.
>>> edwardyoon@apache.org
>>> http://blog.udanax.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Luc Maisonobe <Lu...@free.fr>.
Sam Halliday a écrit :
> Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
> with commons-math. However, there might be parts of it that are relevant to
> Hama.
Is Hama related to Hadoop ?
>
> We should absolutely ensure that the "ducks are aligned" between
> commons-math and Hama. It would be win-win if Hama were able to use
> commons-math's linear interfaces. commons-math will hopefully be moving to
> use a netlib-java style backend (not user facing), we should ensure that
> ScaLAPACK is handled in the same way.
>
> I have some ideas of people who might be interested... I've encountered them
> as maintainer of MTJ, but never worked with them. I'll dig through my e-mail
> archive and make the intros.
>
>
> Edward J. Yoon-2 wrote:
>> That's really cool.
>>
>> BTW, Can I ask about the plan of data distribution strategies of your
>> 'distributed' package in the future? IMO, it seems, it doesn't sit
>> well with 'common-math' project.
>>
>> If if there is a developer who wants to implement 'distributed', pls
>> let us know, too. I'm working for the Hama
>> (http://incubator.apache.org/hama) with ScaLAPACK members.
>>
>> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com>
>> wrote:
>>> Dear all,
>>>
>>> I am a maintainer of the matrix-toolkits-java
>>>
>>> http://code.google.com/p/matrix-toolkits-java/
>>>
>>> which is a comprehensive collection of matrix data structures, linear
>>> solvers, least squares methods, eigenvalue and singular value
>>> decompositions.
>>>
>>> This note is in regard to the commons-math library. It is clear that our
>>> projects dovetail, especially when I look at "linear" in version 2.0 of
>>> the
>>> API. It would be good if we could either complement or consolidate
>>> efforts,
>>> rather than reproduce.
>>>
>>> It would be excellent if all the functionality of matrix-toolkits-java
>>> were
>>> available as part of commons-math. There is already too much diversity
>>> and
>>> un-maintained maths code out there for Java!
>>>
>>> As a start, I'd like to discourage the use of a solid implementation for
>>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>>> allowing
>>> implementations based on the Templates project:-
>>>
>>> http://www.netlib.org/templates
>>>
>>> The reason is that the storage implementation should be related to the
>>> type
>>> of data being stored. For example, there are many well-known kinds of
>>> sparse
>>> matrix that are well suited to particular kinds of calculations...
>>> consider
>>> multiplying sparse matrices that you know to be diagonal!
>>>
>>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>>> thinking about linear algebra and have set up unrivalled standard APIs
>>> which
>>> have been implemented right down to the architecture level. It would be a
>>> major mistake if commons-math didn't build on their good work.
>>>
>>> I believe commons-math should move to a netlib-java backend (allowing the
>>> use of machine optimised BLAS/LAPACK).
>>>
>>> http://code.google.com/p/netlib-java/
>>>
>>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>>> former should be possible with some work within the current API, but I
>>> fear
>>> major API changes would be needed for the latter. I do not want the
>>> commons-math API to walk into this trap without having first considered
>>> future architectures! MTJ has a distributed package, but I am not sure if
>>> this is something that is completely future proof either.
>>>
>>> What say ye'?
>>>
>>> --
>>> Sam
>>>
>>> --
>>> View this message in context:
>>> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
>>> Sent from the Commons - Dev mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon @ NHN, corp.
>> edwardyoon@apache.org
>> http://blog.udanax.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
Edward, Hama looks fantastic! I fully agree, 'distributed' isn't agreeable
with commons-math. However, there might be parts of it that are relevant to
Hama.
We should absolutely ensure that the "ducks are aligned" between
commons-math and Hama. It would be win-win if Hama were able to use
commons-math's linear interfaces. commons-math will hopefully be moving to
use a netlib-java style backend (not user facing), we should ensure that
ScaLAPACK is handled in the same way.
I have some ideas of people who might be interested... I've encountered them
as maintainer of MTJ, but never worked with them. I'll dig through my e-mail
archive and make the intros.
Edward J. Yoon-2 wrote:
>
> That's really cool.
>
> BTW, Can I ask about the plan of data distribution strategies of your
> 'distributed' package in the future? IMO, it seems, it doesn't sit
> well with 'common-math' project.
>
> If if there is a developer who wants to implement 'distributed', pls
> let us know, too. I'm working for the Hama
> (http://incubator.apache.org/hama) with ScaLAPACK members.
>
> On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com>
> wrote:
>>
>> Dear all,
>>
>> I am a maintainer of the matrix-toolkits-java
>>
>> http://code.google.com/p/matrix-toolkits-java/
>>
>> which is a comprehensive collection of matrix data structures, linear
>> solvers, least squares methods, eigenvalue and singular value
>> decompositions.
>>
>> This note is in regard to the commons-math library. It is clear that our
>> projects dovetail, especially when I look at "linear" in version 2.0 of
>> the
>> API. It would be good if we could either complement or consolidate
>> efforts,
>> rather than reproduce.
>>
>> It would be excellent if all the functionality of matrix-toolkits-java
>> were
>> available as part of commons-math. There is already too much diversity
>> and
>> un-maintained maths code out there for Java!
>>
>> As a start, I'd like to discourage the use of a solid implementation for
>> SparseReal{Vector, Matrix}... please prefer an interface approach,
>> allowing
>> implementations based on the Templates project:-
>>
>> http://www.netlib.org/templates
>>
>> The reason is that the storage implementation should be related to the
>> type
>> of data being stored. For example, there are many well-known kinds of
>> sparse
>> matrix that are well suited to particular kinds of calculations...
>> consider
>> multiplying sparse matrices that you know to be diagonal!
>>
>> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
>> thinking about linear algebra and have set up unrivalled standard APIs
>> which
>> have been implemented right down to the architecture level. It would be a
>> major mistake if commons-math didn't build on their good work.
>>
>> I believe commons-math should move to a netlib-java backend (allowing the
>> use of machine optimised BLAS/LAPACK).
>>
>> http://code.google.com/p/netlib-java/
>>
>> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
>> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
>> former should be possible with some work within the current API, but I
>> fear
>> major API changes would be needed for the latter. I do not want the
>> commons-math API to walk into this trap without having first considered
>> future architectures! MTJ has a distributed package, but I am not sure if
>> this is something that is completely future proof either.
>>
>> What say ye'?
>>
>> --
>> Sam
>>
>> --
>> View this message in context:
>> http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
>> Sent from the Commons - Dev mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> edwardyoon@apache.org
> http://blog.udanax.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23668079.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by "Edward J. Yoon" <ed...@apache.org>.
That's really cool.
BTW, Can I ask about the plan of data distribution strategies of your
'distributed' package in the future? IMO, it seems, it doesn't sit
well with 'common-math' project.
If if there is a developer who wants to implement 'distributed', pls
let us know, too. I'm working for the Hama
(http://incubator.apache.org/hama) with ScaLAPACK members.
On Thu, May 14, 2009 at 7:18 PM, Sam Halliday <sa...@gmail.com> wrote:
>
> Dear all,
>
> I am a maintainer of the matrix-toolkits-java
>
> http://code.google.com/p/matrix-toolkits-java/
>
> which is a comprehensive collection of matrix data structures, linear
> solvers, least squares methods, eigenvalue and singular value
> decompositions.
>
> This note is in regard to the commons-math library. It is clear that our
> projects dovetail, especially when I look at "linear" in version 2.0 of the
> API. It would be good if we could either complement or consolidate efforts,
> rather than reproduce.
>
> It would be excellent if all the functionality of matrix-toolkits-java were
> available as part of commons-math. There is already too much diversity and
> un-maintained maths code out there for Java!
>
> As a start, I'd like to discourage the use of a solid implementation for
> SparseReal{Vector, Matrix}... please prefer an interface approach, allowing
> implementations based on the Templates project:-
>
> http://www.netlib.org/templates
>
> The reason is that the storage implementation should be related to the type
> of data being stored. For example, there are many well-known kinds of sparse
> matrix that are well suited to particular kinds of calculations... consider
> multiplying sparse matrices that you know to be diagonal!
>
> In general, the netlib.org folk (BLAS/LAPACK) have spent a *lot* of time
> thinking about linear algebra and have set up unrivalled standard APIs which
> have been implemented right down to the architecture level. It would be a
> major mistake if commons-math didn't build on their good work.
>
> I believe commons-math should move to a netlib-java backend (allowing the
> use of machine optimised BLAS/LAPACK).
>
> http://code.google.com/p/netlib-java/
>
> The largest problems facing MTJ are support for Sparse BLAS/LAPACK and
> scalability to parallel architectures which use Parallel BLAS/LAPACK. The
> former should be possible with some work within the current API, but I fear
> major API changes would be needed for the latter. I do not want the
> commons-math API to walk into this trap without having first considered
> future architectures! MTJ has a distributed package, but I am not sure if
> this is something that is completely future proof either.
>
> What say ye'?
>
> --
> Sam
>
> --
> View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23537813.html
> Sent from the Commons - Dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
--
Best Regards, Edward J. Yoon @ NHN, corp.
edwardyoon@apache.org
http://blog.udanax.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: commons-math, matrix-toolkits-java and consolidation
Posted by Sam Halliday <sa...@gmail.com>.
In preparation for the post-2.0 discussions, I've created three issues on the
tracker to deal with the consolidation of F2J (aka JLAPACK), netlib-java and
matrix-toolkits-java.
http://issues.apache.org/jira/browse/MATH-269
http://issues.apache.org/jira/browse/MATH-270
http://issues.apache.org/jira/browse/MATH-271
Sam Halliday wrote:
>
> ...
> This note is in regard to the commons-math library. It is clear that our
> projects dovetail, especially when I look at "linear" in version 2.0 of
> the API. It would be good if we could either complement or consolidate
> efforts, rather than reproduce.
> ...
>
--
View this message in context: http://www.nabble.com/commons-math%2C-matrix-toolkits-java-and-consolidation-tp23537813p23767938.html
Sent from the Commons - Dev mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org