You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Benson Margulies <bi...@gmail.com> on 2009/12/08 14:18:28 UTC
Home for the colt fork
Mahout now has a fork of a portion of the 'category A' portion of the
CERN colt library forked. The Mahout fork is, of course, in the Mahout
tree under a Mahout Java package and Maven triple.
I want to use the collections classes from Mahout as the core to a new
set of commons-primitives classes that do the useful things that GNU
Trove does.
The classes I want to start from depend on the classes that are in the
Mahout fork.
As a temporary expedient, I can depend on their there. However, I
submit that it would be more better if the mathematical code were in
commons-math. Was this option explored?
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Phil Steitz <ph...@gmail.com>.
Jake Mannix wrote:
> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
>
>> This is interesting. We have a raft of mathematically qualified
>> committers on Mahout, and this message asking for help on
>> commons-math, and a raft of code marooned at mahout that wants to be
>> in commons math. If I were one of those mathematically competant
>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>
>
> The commons-math linear APIs have been described as effectively locked
> until 3.0, due to back-compat requirements. This means that any code
> contributed
> into c-math would live in a parallel (no pun intended) to the linear
> primitives which
> exist already in there.
>
> Adopting something like MTJ or Colt in Mahout turned out to be easier,
> because
> we are on release 0.2 (heading for 0.3 now), and have less stringent
> back-compat
> requirements, so we are overhauling our linear apis (read: even user-facing
> interface changes) to take advantage of useful parts of Colt, and are
> planning on
> using our Colt fork as the underlying implementation.
>
> Commons-math expressed that changing linear APIs is not something they can
> do,
> given the maturity of their library, so where would Colt *go* in c-math?
> It's own
> submodule, having its own eigendecompositions and svd and so forth, running
> parallel to the current c-math impls? Why?
>
> Who would maintain it and write tests for it, and how do you explain to
> end-users which they should use?
>
>
>
>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>> wrote:
>>> Ted Dunning a écrit :
>>>> Actually, the reason that we have Colt in Mahout is it has proven
>> impossible
>>>> to get changes into commons math. We really, really wanted to use
>> commons
>>>> math rather than have our own linear algebra package, but it just proved
>>>> impossible and we didn't want to wait forever.
>>> If you really, really wants to use commons math and want changes to be
>>> included, contribute them.
>
> I have submitted patches for the following tickets: MATH-312 (and acceptance
> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
> of which have appear to have had much progress on. All of my patches come
> with unit tests for new functionality.
>
> On the other hand, when I opened the discussion about extending the
> functions
> package to enable composable functions (MATH-313), I got an entirely hostile
> response, which only tempered as far as "+0" on adding it after discussion.
>
> In particular, my first step at making commons-math something Mahout could
> standardize on for linear work was MATH-312, which I did submit a patch for,
> and revised it many times after discussion about what is acceptable practice
> in c-math. Not yet applied, months later. It's probably far out of date
> now...
>
> Similarly, when I tried to ask what the status on decisions on whether to
> adopt
> MTJ or Colt, the statement by Phil was basically that commons-math would not
> adopt anything which had any external dependencies or
> not-easily-human-readable java source (which ruled out MTJ because of f2j
> produced code), and which had to be fully tested and maintained prior to
> adoption (which rules out Colt which has no unit tests yet).
We agreed early on that commons-math was going to be self-contained.
It is fine to reopen that discussion. I stated my opinion, which
is to stay away from *required* external dependencies. I have been
wrong before, and will be wrong again. Clear arguments can
enlighten me. I also feel obligated to support the code that we
ship. Others may disagree with this and feel that it is OK to ship
code that we - the committers voting on the release - cannot debug
or understand. It will take some very "enlightening" arguments to
get me to agree to releasing code that I can't understand.
Regarding unit tests, the answer is simple - volunteer to write them
and include them in patches. Code without unit tests is not
complete code. If I commit it, it means that *I* am going to have to
write the unit tests and I am going to do that before committing.
Patches without unit tests will therefore take longer to commit.
Phil
>
> Ted and I weren't making "requests" for other people to do work, we were
> wondering whether even offers to do some of the work would be accepted,
> and for many of the questions/suggestions we had, it seems the desires
> and requirements of the Mahout community were incompatible with those
> of commons-math.
>
> -jake
>
>
>> > I think the only change that was proposed and not done because of lack
>>> of consensus was the inclusion of MTJ (and I don't consider the
>>> discussion closed on that topic either, so it may still happen some
>>> day). All the other changes that are desired are simply lacking someone
>>> to do the work. There were proposals to extend the linear algebra API,
>>> proposals to add more support for sparse matrices, proposals to get
>>> partial decomposition ... But sparse contributions (pun intended).
>>>
>>> I try to do what I can, but as you have probably seen have been rather
>>> silent since 2.0 release. For my part, I really, really need help. I
>>> would like to fix the problems in the eigen decomposition and SVD but
>>> need a good kick to get on it, and having only requests and no help is
>>> not really motivating.
>>>
>>> Luc
>>>
>>>> If that problem were solved, then it would be great to depend on commons
>>>> math. If that problem isn't solved, then there is no way to depend on
>>>> commons math.
>>>>
>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
>>> wrote:
>>>>> We can't possibly have a dependency on Mahout in the long term. Either
>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>> with two forks, which would be sad.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>> james@carmanconsulting.com>
>>>>> wrote:
>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>> bimargulies@gmail.com>
>>>>> wrote:
>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>> Mahout
>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>
>>>>>>> I want to use the collections classes from Mahout as the core to a
>> new
>>>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>>>> Trove does.
>>>>>>>
>>>>>>> The classes I want to start from depend on the classes that are in
>> the
>>>>>>> Mahout fork.
>>>>>>>
>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>> submit that it would be more better if the mathematical code were in
>>>>>>> commons-math. Was this option explored?
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Homefor
the colt fork)
Posted by Bill Barker <bi...@verizon.net>.
----- Original Message -----
From: "Ted Dunning" <te...@gmail.com>
To: "Commons Developers List" <de...@commons.apache.org>
Sent: Saturday, December 12, 2009 11:17 AM
Subject: Re: [math] getting changes included into commons-math (was Re:
Homefor the colt fork)
> MATH-312, 314, 316 and 317 were the first round.
>
> Some of the necessary changes included:
>
> - sparse iterators for sparse vectors
> - view semantics for sub-matrix and sub-vectors
> - additional sparse vector types for specialized applications
> - introduction of an unbounded sparse matrix/vector that simply remembers
> the highest bound set
> - introduction of abstract classes to allow off the cuff inner
> implementations of virtual vectors
> - introduction of additional decomposition classes
> - introduction of row and column label abstraction for matrices and
> vectors
> - higher performance implementations
>
> The 5 issues that Jake filed were mostly address the first point. That
> stalled out when he hit resistance to change.
>
> Since then we have adopted Colt as our matrix/linear algebra/collections
> implementation and we don't really need any changes to MATH.
>
Except for MATH-314 (which doesn't have a patch), these JIRA issues are
largely implemented in [math] now. It will probably have to be improved
before a 2.1 release, but works well enough for now.
I'm sorry that you feel that [math] doesn't meet your needs. However, I'm
grateful to Jake for his contributions that have made [math] a better
project.
> On Sat, Dec 12, 2009 at 8:35 AM, J.Pietschmann <j3...@yahoo.de> wrote:
>
>> On 09.12.2009 20:58, Jake Mannix wrote:
>>
>>> The commons-math linear APIs have been described as effectively locked
>>> until 3.0, due to back-compat requirements.
>>>
>>
>> Can you give a short summary of the API changes which are necessary
>> to incorporate and use the functionality you need? Some functionality
>> I've seen mentioned here could probably be pressed in the existing
>> API, but I may have missed some details.
>>
>> J.Pietschmann
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Ted Dunning <te...@gmail.com>.
MATH-312, 314, 316 and 317 were the first round.
Some of the necessary changes included:
- sparse iterators for sparse vectors
- view semantics for sub-matrix and sub-vectors
- additional sparse vector types for specialized applications
- introduction of an unbounded sparse matrix/vector that simply remembers
the highest bound set
- introduction of abstract classes to allow off the cuff inner
implementations of virtual vectors
- introduction of additional decomposition classes
- introduction of row and column label abstraction for matrices and vectors
- higher performance implementations
The 5 issues that Jake filed were mostly address the first point. That
stalled out when he hit resistance to change.
Since then we have adopted Colt as our matrix/linear algebra/collections
implementation and we don't really need any changes to MATH.
On Sat, Dec 12, 2009 at 8:35 AM, J.Pietschmann <j3...@yahoo.de> wrote:
> On 09.12.2009 20:58, Jake Mannix wrote:
>
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements.
>>
>
> Can you give a short summary of the API changes which are necessary
> to incorporate and use the functionality you need? Some functionality
> I've seen mentioned here could probably be pressed in the existing
> API, but I may have missed some details.
>
> J.Pietschmann
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by "J.Pietschmann" <j3...@yahoo.de>.
On 09.12.2009 20:58, Jake Mannix wrote:
> The commons-math linear APIs have been described as effectively locked
> until 3.0, due to back-compat requirements.
Can you give a short summary of the API changes which are necessary
to incorporate and use the functionality you need? Some functionality
I've seen mentioned here could probably be pressed in the existing
API, but I may have missed some details.
J.Pietschmann
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Ted Dunning <te...@gmail.com>.
I don't want to denigrate the large contributions of Phil and Luc
(especially Luc), but I have drawn a stronger conclusion that it isn't worth
my time to contribute to commons-math because interesting changes likely
won't get accepted.
On Wed, Dec 9, 2009 at 11:58 AM, Jake Mannix <ja...@gmail.com> wrote:
> Ted and I weren't making "requests" for other people to do work, we were
> wondering whether even offers to do some of the work would be accepted,
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Phil Steitz <ph...@gmail.com>.
Ted Dunning wrote:
> Similarly, https://issues.apache.org/jira/browse/MATH-310 had code from a
> contributor and got totally shot down basically with "We don't care what you
> think, we don't want it". Extensive attempts on my part to find common
> ground just got shot down by Phil with -1 and essentially no more discussion
> than "don't like it".
Sorry you feel that way. My objections were to the design, not in
any way personal or dismissive. I did indicate alternative ways to
get the desired functionality implemented. I also explained, I
think, pretty clearly why "I did not like it" and I stand behind my
comments.
Phil
>
> Mahout has some unusual requirements and definitely can't use Math as is.
> It is also fast moving and needs changes to actually happen. The experience
> has been that only totally trivial non-changes get accepted (
> https://issues.apache.org/jira/browse/MATH-267 and
> https://issues.apache.org/jira/browse/MATH-222 are examples). Anything more
> interesting seems to get bogged down in NIH or endless alternations of
> "not-now we are about to have a major release and can't be distracted" and
> "not-now because we aren't doing a major release and have stay compatible".
>
> On Wed, Dec 9, 2009 at 11:58 AM, Jake Mannix <ja...@gmail.com> wrote:
>
>> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bimargulies@gmail.com
>>> wrote:
>>> This is interesting. We have a raft of mathematically qualified
>>> committers on Mahout, and this message asking for help on
>>> commons-math, and a raft of code marooned at mahout that wants to be
>>> in commons math. If I were one of those mathematically competant
>>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>>
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements. This means that any code
>> contributed
>> into c-math would live in a parallel (no pun intended) to the linear
>> primitives which
>> exist already in there.
>>
>> Adopting something like MTJ or Colt in Mahout turned out to be easier,
>> because
>> we are on release 0.2 (heading for 0.3 now), and have less stringent
>> back-compat
>> requirements, so we are overhauling our linear apis (read: even user-facing
>> interface changes) to take advantage of useful parts of Colt, and are
>> planning on
>> using our Colt fork as the underlying implementation.
>>
>> Commons-math expressed that changing linear APIs is not something they can
>> do,
>> given the maturity of their library, so where would Colt *go* in c-math?
>> It's own
>> submodule, having its own eigendecompositions and svd and so forth, running
>> parallel to the current c-math impls? Why?
>>
>> Who would maintain it and write tests for it, and how do you explain to
>> end-users which they should use?
>>
>>
>>
>>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>>> wrote:
>>>> Ted Dunning a écrit :
>>>>> Actually, the reason that we have Colt in Mahout is it has proven
>>> impossible
>>>>> to get changes into commons math. We really, really wanted to use
>>> commons
>>>>> math rather than have our own linear algebra package, but it just
>> proved
>>>>> impossible and we didn't want to wait forever.
>>>> If you really, really wants to use commons math and want changes to be
>>>> included, contribute them.
>> I have submitted patches for the following tickets: MATH-312 (and
>> acceptance
>> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
>> of which have appear to have had much progress on. All of my patches come
>> with unit tests for new functionality.
>>
>> On the other hand, when I opened the discussion about extending the
>> functions
>> package to enable composable functions (MATH-313), I got an entirely
>> hostile
>> response, which only tempered as far as "+0" on adding it after discussion.
>>
>> In particular, my first step at making commons-math something Mahout could
>> standardize on for linear work was MATH-312, which I did submit a patch
>> for,
>> and revised it many times after discussion about what is acceptable
>> practice
>> in c-math. Not yet applied, months later. It's probably far out of date
>> now...
>>
>> Similarly, when I tried to ask what the status on decisions on whether to
>> adopt
>> MTJ or Colt, the statement by Phil was basically that commons-math would
>> not
>> adopt anything which had any external dependencies or
>> not-easily-human-readable java source (which ruled out MTJ because of f2j
>> produced code), and which had to be fully tested and maintained prior to
>> adoption (which rules out Colt which has no unit tests yet).
>>
>> Ted and I weren't making "requests" for other people to do work, we were
>> wondering whether even offers to do some of the work would be accepted,
>> and for many of the questions/suggestions we had, it seems the desires
>> and requirements of the Mahout community were incompatible with those
>> of commons-math.
>>
>> -jake
>>
>>
>>> > I think the only change that was proposed and not done because of lack
>>>> of consensus was the inclusion of MTJ (and I don't consider the
>>>> discussion closed on that topic either, so it may still happen some
>>>> day). All the other changes that are desired are simply lacking someone
>>>> to do the work. There were proposals to extend the linear algebra API,
>>>> proposals to add more support for sparse matrices, proposals to get
>>>> partial decomposition ... But sparse contributions (pun intended).
>>>>
>>>> I try to do what I can, but as you have probably seen have been rather
>>>> silent since 2.0 release. For my part, I really, really need help. I
>>>> would like to fix the problems in the eigen decomposition and SVD but
>>>> need a good kick to get on it, and having only requests and no help is
>>>> not really motivating.
>>>>
>>>> Luc
>>>>
>>>>> If that problem were solved, then it would be great to depend on
>> commons
>>>>> math. If that problem isn't solved, then there is no way to depend on
>>>>> commons math.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <
>> bimargulies@gmail.com
>>>> wrote:
>>>>>> We can't possibly have a dependency on Mahout in the long term.
>> Either
>>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>>> with two forks, which would be sad.
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>>> james@carmanconsulting.com>
>>>>>> wrote:
>>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>>
>>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>>> bimargulies@gmail.com>
>>>>>> wrote:
>>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of
>> the
>>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>>> Mahout
>>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>>
>>>>>>>> I want to use the collections classes from Mahout as the core to a
>>> new
>>>>>>>> set of commons-primitives classes that do the useful things that
>> GNU
>>>>>>>> Trove does.
>>>>>>>>
>>>>>>>> The classes I want to start from depend on the classes that are in
>>> the
>>>>>>>> Mahout fork.
>>>>>>>>
>>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>>> submit that it would be more better if the mathematical code were
>> in
>>>>>>>> commons-math. Was this option explored?
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>>>
>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Ted Dunning <te...@gmail.com>.
Similarly, https://issues.apache.org/jira/browse/MATH-310 had code from a
contributor and got totally shot down basically with "We don't care what you
think, we don't want it". Extensive attempts on my part to find common
ground just got shot down by Phil with -1 and essentially no more discussion
than "don't like it".
Mahout has some unusual requirements and definitely can't use Math as is.
It is also fast moving and needs changes to actually happen. The experience
has been that only totally trivial non-changes get accepted (
https://issues.apache.org/jira/browse/MATH-267 and
https://issues.apache.org/jira/browse/MATH-222 are examples). Anything more
interesting seems to get bogged down in NIH or endless alternations of
"not-now we are about to have a major release and can't be distracted" and
"not-now because we aren't doing a major release and have stay compatible".
On Wed, Dec 9, 2009 at 11:58 AM, Jake Mannix <ja...@gmail.com> wrote:
> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bimargulies@gmail.com
> >wrote:
>
> > This is interesting. We have a raft of mathematically qualified
> > committers on Mahout, and this message asking for help on
> > commons-math, and a raft of code marooned at mahout that wants to be
> > in commons math. If I were one of those mathematically competant
> > individuals, I'd be off attaching a patch or three to a JIRA or two
> >
>
> The commons-math linear APIs have been described as effectively locked
> until 3.0, due to back-compat requirements. This means that any code
> contributed
> into c-math would live in a parallel (no pun intended) to the linear
> primitives which
> exist already in there.
>
> Adopting something like MTJ or Colt in Mahout turned out to be easier,
> because
> we are on release 0.2 (heading for 0.3 now), and have less stringent
> back-compat
> requirements, so we are overhauling our linear apis (read: even user-facing
> interface changes) to take advantage of useful parts of Colt, and are
> planning on
> using our Colt fork as the underlying implementation.
>
> Commons-math expressed that changing linear APIs is not something they can
> do,
> given the maturity of their library, so where would Colt *go* in c-math?
> It's own
> submodule, having its own eigendecompositions and svd and so forth, running
> parallel to the current c-math impls? Why?
>
> Who would maintain it and write tests for it, and how do you explain to
> end-users which they should use?
>
>
>
> > On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
> > wrote:
> > > Ted Dunning a écrit :
> > >> Actually, the reason that we have Colt in Mahout is it has proven
> > impossible
> > >> to get changes into commons math. We really, really wanted to use
> > commons
> > >> math rather than have our own linear algebra package, but it just
> proved
> > >> impossible and we didn't want to wait forever.
> > >
> > > If you really, really wants to use commons math and want changes to be
> > > included, contribute them.
> >
>
> I have submitted patches for the following tickets: MATH-312 (and
> acceptance
> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
> of which have appear to have had much progress on. All of my patches come
> with unit tests for new functionality.
>
> On the other hand, when I opened the discussion about extending the
> functions
> package to enable composable functions (MATH-313), I got an entirely
> hostile
> response, which only tempered as far as "+0" on adding it after discussion.
>
> In particular, my first step at making commons-math something Mahout could
> standardize on for linear work was MATH-312, which I did submit a patch
> for,
> and revised it many times after discussion about what is acceptable
> practice
> in c-math. Not yet applied, months later. It's probably far out of date
> now...
>
> Similarly, when I tried to ask what the status on decisions on whether to
> adopt
> MTJ or Colt, the statement by Phil was basically that commons-math would
> not
> adopt anything which had any external dependencies or
> not-easily-human-readable java source (which ruled out MTJ because of f2j
> produced code), and which had to be fully tested and maintained prior to
> adoption (which rules out Colt which has no unit tests yet).
>
> Ted and I weren't making "requests" for other people to do work, we were
> wondering whether even offers to do some of the work would be accepted,
> and for many of the questions/suggestions we had, it seems the desires
> and requirements of the Mahout community were incompatible with those
> of commons-math.
>
> -jake
>
>
> > > I think the only change that was proposed and not done because of lack
> > > of consensus was the inclusion of MTJ (and I don't consider the
> > > discussion closed on that topic either, so it may still happen some
> > > day). All the other changes that are desired are simply lacking someone
> > > to do the work. There were proposals to extend the linear algebra API,
> > > proposals to add more support for sparse matrices, proposals to get
> > > partial decomposition ... But sparse contributions (pun intended).
> > >
> > > I try to do what I can, but as you have probably seen have been rather
> > > silent since 2.0 release. For my part, I really, really need help. I
> > > would like to fix the problems in the eigen decomposition and SVD but
> > > need a good kick to get on it, and having only requests and no help is
> > > not really motivating.
> > >
> > > Luc
> > >
> > >>
> > >> If that problem were solved, then it would be great to depend on
> commons
> > >> math. If that problem isn't solved, then there is no way to depend on
> > >> commons math.
> > >>
> > >> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <
> bimargulies@gmail.com
> > >wrote:
> > >>
> > >>> We can't possibly have a dependency on Mahout in the long term.
> Either
> > >>> we all go shares on code in some other piece of commons, or we end up
> > >>> with two forks, which would be sad.
> > >>>
> > >>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
> > james@carmanconsulting.com>
> > >>> wrote:
> > >>>> I wouldn't like to see a dependency on mahout code in a "commons"
> > >>>> library. That seems kind of backwards. If Mahout wants to offload
> > >>>> this stuff, we can move it into a library in commons (which is
> > >>>> typically how stuff used to happen in Jakarta).
> > >>>>
> > >>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
> > bimargulies@gmail.com>
> > >>> wrote:
> > >>>>> Mahout now has a fork of a portion of the 'category A' portion of
> the
> > >>>>> CERN colt library forked. The Mahout fork is, of course, in the
> > Mahout
> > >>>>> tree under a Mahout Java package and Maven triple.
> > >>>>>
> > >>>>> I want to use the collections classes from Mahout as the core to a
> > new
> > >>>>> set of commons-primitives classes that do the useful things that
> GNU
> > >>>>> Trove does.
> > >>>>>
> > >>>>> The classes I want to start from depend on the classes that are in
> > the
> > >>>>> Mahout fork.
> > >>>>>
> > >>>>> As a temporary expedient, I can depend on their there. However, I
> > >>>>> submit that it would be more better if the mathematical code were
> in
> > >>>>> commons-math. Was this option explored?
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>>>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>>>
> > >>>>>
> > >>>>
> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>>
> > >>>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>
> > >>>
> > >>
> > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Jake Mannix <ja...@gmail.com>.
On Wed, Dec 9, 2009 at 1:17 PM, Benson Margulies <bi...@gmail.com>wrote:
>
> CERN Colt is a library with a mixture of 'category A' material and
> 'category B-or-worse' material. In other words, it is not an
> attractive dependency for ASF code as a lump.
>
Yep, that's why when I made my original patch of Colt for Mahout, I
stripped out all dependency on the 'category B-or-worse' material, and
tried to keep everything else other than the physics specific stuff that
nobody was going to need.
> Part of the 'category A' code is a set of very pretty associative
> containers for primitive types. My goal is to take this code and
> modernize it to use Generics as appropriate -- and otherwise make it a
> replacement for the LGPL Trove library.
>
I believe Mahout kept all this: look in o.a.m.matrix.list and
o.a.m.matrix.map
The package name they're both in (o.a.m.matrix) is bad - it used to be
"colt",
but that name is trademarked by CERN, so even if the code is freely
licensable, the name isn't. It might be better called "collections" or
something.
The associative code uses some math code. I'm pretty sure that all of
> the math code that it uses is also category A. In fact, I believe that
> all of it is in the portion that Mahout forked. This includes some
> things that aren't in commons-math at all.
>
Oh, this is what you're saying here, ok.
> So, we have some adoptable code in Colt that overlaps functionality in
> -math, some that doesn't, and some that I want to use as the basis for
> work in -primitives.
>
The stuff you want in primitives is also stuff we use/want to use for
our linear work in Mahout, by the way (OpenDoubleIntHashMap,
DoubleArrayList, etc...).
> If the messages in this thread mean that the code already in -math is
> in fact moving in a direction to support mahout, then it might be
> acceptable for the additional, non-overlapping, Colt math code to end
> up in -math, and then everyone ends up happy?
>
None of the patches mentioned above contain Colt code, so no, c-math
is not moving in a direction to support what mahout needs, not really,
because of the backwards compatibility constraints and the fact that
it's hard to tell what is the "non-overlapping" part of Colt math: for
example: vectors in Colt provide complex aggregation methods, but
c-math vectors do not (as an interface). Is this non-overlapping
code? If it is, then how would you suppose this would be incorporated
in c-math? Add a new interface, the "ColtVector", which is basically
the same as c-math's "RealVector", but with some methods removed
and some others added?
This doesn't really make sense. Implementations are one thing, but
interfaces which consumers see is another.
Another question: OK, -math is a stable, high-compatibility library.
> Mahout, on the other hand, wants to be a fast-moving, somewhat fluid,
> build of code (naturally including some math code) adapted for
> map-reduce. So, how about a branch of math that could release
> frequently with a new major version number?
Well, actually, a more feasible version of this, instead of branches, would
be to use the current "experimental" source submodule in commons-math.
If that could be a faster moving fluid build which could include colt math
etc, that would be great, and Mahout could possibly use this.
This brings up another point though: if commons-primitives is what you're
creating, and want to base it on colt, well, we in Mahout could probably
handle
depending on you, but colt math linear depends on these, so can
commons-math depend on commons-primitives, or does that violate
c-math's "no external dependencies" rule? This would keep c-math from
adopting colt math, because those would need the colt-primitives stuff.
-jake
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Jake Mannix <ja...@gmail.com>.
On Wed, Dec 9, 2009 at 1:17 PM, Benson Margulies <bi...@gmail.com>wrote:
>
> CERN Colt is a library with a mixture of 'category A' material and
> 'category B-or-worse' material. In other words, it is not an
> attractive dependency for ASF code as a lump.
>
Yep, that's why when I made my original patch of Colt for Mahout, I
stripped out all dependency on the 'category B-or-worse' material, and
tried to keep everything else other than the physics specific stuff that
nobody was going to need.
> Part of the 'category A' code is a set of very pretty associative
> containers for primitive types. My goal is to take this code and
> modernize it to use Generics as appropriate -- and otherwise make it a
> replacement for the LGPL Trove library.
>
I believe Mahout kept all this: look in o.a.m.matrix.list and
o.a.m.matrix.map
The package name they're both in (o.a.m.matrix) is bad - it used to be
"colt",
but that name is trademarked by CERN, so even if the code is freely
licensable, the name isn't. It might be better called "collections" or
something.
The associative code uses some math code. I'm pretty sure that all of
> the math code that it uses is also category A. In fact, I believe that
> all of it is in the portion that Mahout forked. This includes some
> things that aren't in commons-math at all.
>
Oh, this is what you're saying here, ok.
> So, we have some adoptable code in Colt that overlaps functionality in
> -math, some that doesn't, and some that I want to use as the basis for
> work in -primitives.
>
The stuff you want in primitives is also stuff we use/want to use for
our linear work in Mahout, by the way (OpenDoubleIntHashMap,
DoubleArrayList, etc...).
> If the messages in this thread mean that the code already in -math is
> in fact moving in a direction to support mahout, then it might be
> acceptable for the additional, non-overlapping, Colt math code to end
> up in -math, and then everyone ends up happy?
>
None of the patches mentioned above contain Colt code, so no, c-math
is not moving in a direction to support what mahout needs, not really,
because of the backwards compatibility constraints and the fact that
it's hard to tell what is the "non-overlapping" part of Colt math: for
example: vectors in Colt provide complex aggregation methods, but
c-math vectors do not (as an interface). Is this non-overlapping
code? If it is, then how would you suppose this would be incorporated
in c-math? Add a new interface, the "ColtVector", which is basically
the same as c-math's "RealVector", but with some methods removed
and some others added?
This doesn't really make sense. Implementations are one thing, but
interfaces which consumers see is another.
Another question: OK, -math is a stable, high-compatibility library.
> Mahout, on the other hand, wants to be a fast-moving, somewhat fluid,
> build of code (naturally including some math code) adapted for
> map-reduce. So, how about a branch of math that could release
> frequently with a new major version number?
Well, actually, a more feasible version of this, instead of branches, would
be to use the current "experimental" source submodule in commons-math.
If that could be a faster moving fluid build which could include colt math
etc, that would be great, and Mahout could possibly use this.
This brings up another point though: if commons-primitives is what you're
creating, and want to base it on colt, well, we in Mahout could probably
handle
depending on you, but colt math linear depends on these, so can
commons-math depend on commons-primitives, or does that violate
c-math's "no external dependencies" rule? This would keep c-math from
adopting colt math, because those would need the colt-primitives stuff.
-jake
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Benson Margulies <bi...@gmail.com>.
I can see how everyone ends up with a headache here.
As the person who threw the most recent rock into the lake, let me
re-present the situation as I got into it.
CERN Colt is a library with a mixture of 'category A' material and
'category B-or-worse' material. In other words, it is not an
attractive dependency for ASF code as a lump.
Part of the 'category A' code is a set of very pretty associative
containers for primitive types. My goal is to take this code and
modernize it to use Generics as appropriate -- and otherwise make it a
replacement for the LGPL Trove library.
The associative code uses some math code. I'm pretty sure that all of
the math code that it uses is also category A. In fact, I believe that
all of it is in the portion that Mahout forked. This includes some
things that aren't in commons-math at all.
So, we have some adoptable code in Colt that overlaps functionality in
-math, some that doesn't, and some that I want to use as the basis for
work in -primitives.
If the messages in this thread mean that the code already in -math is
in fact moving in a direction to support mahout, then it might be
acceptable for the additional, non-overlapping, Colt math code to end
up in -math, and then everyone ends up happy?
Another question: OK, -math is a stable, high-compatibility library.
Mahout, on the other hand, wants to be a fast-moving, somewhat fluid,
build of code (naturally including some math code) adapted for
map-reduce. So, how about a branch of math that could release
frequently with a new major version number? Obviously, from a work
standpoint, this would depend on some of the Mahouts achieving
committer status in commons.
On Wed, Dec 9, 2009 at 3:46 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> Jake Mannix a écrit :
>> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
>>
>>> This is interesting. We have a raft of mathematically qualified
>>> committers on Mahout, and this message asking for help on
>>> commons-math, and a raft of code marooned at mahout that wants to be
>>> in commons math. If I were one of those mathematically competant
>>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>>
>>
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements. This means that any code
>> contributed
>> into c-math would live in a parallel (no pun intended) to the linear
>> primitives which
>> exist already in there.
>>
>> Adopting something like MTJ or Colt in Mahout turned out to be easier,
>> because
>> we are on release 0.2 (heading for 0.3 now), and have less stringent
>> back-compat
>> requirements, so we are overhauling our linear apis (read: even user-facing
>> interface changes) to take advantage of useful parts of Colt, and are
>> planning on
>> using our Colt fork as the underlying implementation.
>>
>> Commons-math expressed that changing linear APIs is not something they can
>> do,
>> given the maturity of their library, so where would Colt *go* in c-math?
>> It's own
>> submodule, having its own eigendecompositions and svd and so forth, running
>> parallel to the current c-math impls? Why?
>>
>> Who would maintain it and write tests for it, and how do you explain to
>> end-users which they should use?
>>
>>
>>
>>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>>> wrote:
>>>> Ted Dunning a écrit :
>>>>> Actually, the reason that we have Colt in Mahout is it has proven
>>> impossible
>>>>> to get changes into commons math. We really, really wanted to use
>>> commons
>>>>> math rather than have our own linear algebra package, but it just proved
>>>>> impossible and we didn't want to wait forever.
>>>> If you really, really wants to use commons math and want changes to be
>>>> included, contribute them.
>>
>> I have submitted patches for the following tickets: MATH-312 (and acceptance
>> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
>> of which have appear to have had much progress on. All of my patches come
>> with unit tests for new functionality.
>
> I had these patches in my backlog and considered them accepted. I should
> have commited them before, sorry for that. I'll take care of them right now.
>
>>
>> On the other hand, when I opened the discussion about extending the
>> functions
>> package to enable composable functions (MATH-313), I got an entirely hostile
>> response, which only tempered as far as "+0" on adding it after discussion.
>
> The discussion was not entirely hostile as we get some intermediate
> consensus at some points. I understand your feelings after several
> patches that did not get committed fast enough.
>
> Please accept my apologizes for this.
>
> Luc
>
>>
>> In particular, my first step at making commons-math something Mahout could
>> standardize on for linear work was MATH-312, which I did submit a patch for,
>> and revised it many times after discussion about what is acceptable practice
>> in c-math. Not yet applied, months later. It's probably far out of date
>> now...
>>
>> Similarly, when I tried to ask what the status on decisions on whether to
>> adopt
>> MTJ or Colt, the statement by Phil was basically that commons-math would not
>> adopt anything which had any external dependencies or
>> not-easily-human-readable java source (which ruled out MTJ because of f2j
>> produced code), and which had to be fully tested and maintained prior to
>> adoption (which rules out Colt which has no unit tests yet).
>>
>> Ted and I weren't making "requests" for other people to do work, we were
>> wondering whether even offers to do some of the work would be accepted,
>> and for many of the questions/suggestions we had, it seems the desires
>> and requirements of the Mahout community were incompatible with those
>> of commons-math.
>>
>> -jake
>>
>>
>>> > I think the only change that was proposed and not done because of lack
>>>> of consensus was the inclusion of MTJ (and I don't consider the
>>>> discussion closed on that topic either, so it may still happen some
>>>> day). All the other changes that are desired are simply lacking someone
>>>> to do the work. There were proposals to extend the linear algebra API,
>>>> proposals to add more support for sparse matrices, proposals to get
>>>> partial decomposition ... But sparse contributions (pun intended).
>>>>
>>>> I try to do what I can, but as you have probably seen have been rather
>>>> silent since 2.0 release. For my part, I really, really need help. I
>>>> would like to fix the problems in the eigen decomposition and SVD but
>>>> need a good kick to get on it, and having only requests and no help is
>>>> not really motivating.
>>>>
>>>> Luc
>>>>
>>>>> If that problem were solved, then it would be great to depend on commons
>>>>> math. If that problem isn't solved, then there is no way to depend on
>>>>> commons math.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
>>>> wrote:
>>>>>> We can't possibly have a dependency on Mahout in the long term. Either
>>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>>> with two forks, which would be sad.
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>>> james@carmanconsulting.com>
>>>>>> wrote:
>>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>>
>>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>>> bimargulies@gmail.com>
>>>>>> wrote:
>>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>>> Mahout
>>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>>
>>>>>>>> I want to use the collections classes from Mahout as the core to a
>>> new
>>>>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>>>>> Trove does.
>>>>>>>>
>>>>>>>> The classes I want to start from depend on the classes that are in
>>> the
>>>>>>>> Mahout fork.
>>>>>>>>
>>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>>> submit that it would be more better if the mathematical code were in
>>>>>>>> commons-math. Was this option explored?
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Henri Yandell <fl...@gmail.com>.
Btw, useful link that shows you the patches (or at least attachments)
you currently have on open tickets:
https://issues.apache.org/jira/secure/ConfigureReport.jspa?versionId=-1&issueStatus=open&selectedProjectId=12310485&reportKey=com.sourcelabs.jira.plugin.report.contributions%3Acontributionreport&Next=Next
On Wed, Dec 9, 2009 at 12:46 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> Jake Mannix a écrit :
>> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
>>
>>> This is interesting. We have a raft of mathematically qualified
>>> committers on Mahout, and this message asking for help on
>>> commons-math, and a raft of code marooned at mahout that wants to be
>>> in commons math. If I were one of those mathematically competant
>>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>>
>>
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements. This means that any code
>> contributed
>> into c-math would live in a parallel (no pun intended) to the linear
>> primitives which
>> exist already in there.
>>
>> Adopting something like MTJ or Colt in Mahout turned out to be easier,
>> because
>> we are on release 0.2 (heading for 0.3 now), and have less stringent
>> back-compat
>> requirements, so we are overhauling our linear apis (read: even user-facing
>> interface changes) to take advantage of useful parts of Colt, and are
>> planning on
>> using our Colt fork as the underlying implementation.
>>
>> Commons-math expressed that changing linear APIs is not something they can
>> do,
>> given the maturity of their library, so where would Colt *go* in c-math?
>> It's own
>> submodule, having its own eigendecompositions and svd and so forth, running
>> parallel to the current c-math impls? Why?
>>
>> Who would maintain it and write tests for it, and how do you explain to
>> end-users which they should use?
>>
>>
>>
>>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>>> wrote:
>>>> Ted Dunning a écrit :
>>>>> Actually, the reason that we have Colt in Mahout is it has proven
>>> impossible
>>>>> to get changes into commons math. We really, really wanted to use
>>> commons
>>>>> math rather than have our own linear algebra package, but it just proved
>>>>> impossible and we didn't want to wait forever.
>>>> If you really, really wants to use commons math and want changes to be
>>>> included, contribute them.
>>
>> I have submitted patches for the following tickets: MATH-312 (and acceptance
>> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
>> of which have appear to have had much progress on. All of my patches come
>> with unit tests for new functionality.
>
> I had these patches in my backlog and considered them accepted. I should
> have commited them before, sorry for that. I'll take care of them right now.
>
>>
>> On the other hand, when I opened the discussion about extending the
>> functions
>> package to enable composable functions (MATH-313), I got an entirely hostile
>> response, which only tempered as far as "+0" on adding it after discussion.
>
> The discussion was not entirely hostile as we get some intermediate
> consensus at some points. I understand your feelings after several
> patches that did not get committed fast enough.
>
> Please accept my apologizes for this.
>
> Luc
>
>>
>> In particular, my first step at making commons-math something Mahout could
>> standardize on for linear work was MATH-312, which I did submit a patch for,
>> and revised it many times after discussion about what is acceptable practice
>> in c-math. Not yet applied, months later. It's probably far out of date
>> now...
>>
>> Similarly, when I tried to ask what the status on decisions on whether to
>> adopt
>> MTJ or Colt, the statement by Phil was basically that commons-math would not
>> adopt anything which had any external dependencies or
>> not-easily-human-readable java source (which ruled out MTJ because of f2j
>> produced code), and which had to be fully tested and maintained prior to
>> adoption (which rules out Colt which has no unit tests yet).
>>
>> Ted and I weren't making "requests" for other people to do work, we were
>> wondering whether even offers to do some of the work would be accepted,
>> and for many of the questions/suggestions we had, it seems the desires
>> and requirements of the Mahout community were incompatible with those
>> of commons-math.
>>
>> -jake
>>
>>
>>> > I think the only change that was proposed and not done because of lack
>>>> of consensus was the inclusion of MTJ (and I don't consider the
>>>> discussion closed on that topic either, so it may still happen some
>>>> day). All the other changes that are desired are simply lacking someone
>>>> to do the work. There were proposals to extend the linear algebra API,
>>>> proposals to add more support for sparse matrices, proposals to get
>>>> partial decomposition ... But sparse contributions (pun intended).
>>>>
>>>> I try to do what I can, but as you have probably seen have been rather
>>>> silent since 2.0 release. For my part, I really, really need help. I
>>>> would like to fix the problems in the eigen decomposition and SVD but
>>>> need a good kick to get on it, and having only requests and no help is
>>>> not really motivating.
>>>>
>>>> Luc
>>>>
>>>>> If that problem were solved, then it would be great to depend on commons
>>>>> math. If that problem isn't solved, then there is no way to depend on
>>>>> commons math.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
>>>> wrote:
>>>>>> We can't possibly have a dependency on Mahout in the long term. Either
>>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>>> with two forks, which would be sad.
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>>> james@carmanconsulting.com>
>>>>>> wrote:
>>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>>
>>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>>> bimargulies@gmail.com>
>>>>>> wrote:
>>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>>> Mahout
>>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>>
>>>>>>>> I want to use the collections classes from Mahout as the core to a
>>> new
>>>>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>>>>> Trove does.
>>>>>>>>
>>>>>>>> The classes I want to start from depend on the classes that are in
>>> the
>>>>>>>> Mahout fork.
>>>>>>>>
>>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>>> submit that it would be more better if the mathematical code were in
>>>>>>>> commons-math. Was this option explored?
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Benson Margulies <bi...@gmail.com>.
I can see how everyone ends up with a headache here.
As the person who threw the most recent rock into the lake, let me
re-present the situation as I got into it.
CERN Colt is a library with a mixture of 'category A' material and
'category B-or-worse' material. In other words, it is not an
attractive dependency for ASF code as a lump.
Part of the 'category A' code is a set of very pretty associative
containers for primitive types. My goal is to take this code and
modernize it to use Generics as appropriate -- and otherwise make it a
replacement for the LGPL Trove library.
The associative code uses some math code. I'm pretty sure that all of
the math code that it uses is also category A. In fact, I believe that
all of it is in the portion that Mahout forked. This includes some
things that aren't in commons-math at all.
So, we have some adoptable code in Colt that overlaps functionality in
-math, some that doesn't, and some that I want to use as the basis for
work in -primitives.
If the messages in this thread mean that the code already in -math is
in fact moving in a direction to support mahout, then it might be
acceptable for the additional, non-overlapping, Colt math code to end
up in -math, and then everyone ends up happy?
Another question: OK, -math is a stable, high-compatibility library.
Mahout, on the other hand, wants to be a fast-moving, somewhat fluid,
build of code (naturally including some math code) adapted for
map-reduce. So, how about a branch of math that could release
frequently with a new major version number? Obviously, from a work
standpoint, this would depend on some of the Mahouts achieving
committer status in commons.
On Wed, Dec 9, 2009 at 3:46 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> Jake Mannix a écrit :
>> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
>>
>>> This is interesting. We have a raft of mathematically qualified
>>> committers on Mahout, and this message asking for help on
>>> commons-math, and a raft of code marooned at mahout that wants to be
>>> in commons math. If I were one of those mathematically competant
>>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>>
>>
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements. This means that any code
>> contributed
>> into c-math would live in a parallel (no pun intended) to the linear
>> primitives which
>> exist already in there.
>>
>> Adopting something like MTJ or Colt in Mahout turned out to be easier,
>> because
>> we are on release 0.2 (heading for 0.3 now), and have less stringent
>> back-compat
>> requirements, so we are overhauling our linear apis (read: even user-facing
>> interface changes) to take advantage of useful parts of Colt, and are
>> planning on
>> using our Colt fork as the underlying implementation.
>>
>> Commons-math expressed that changing linear APIs is not something they can
>> do,
>> given the maturity of their library, so where would Colt *go* in c-math?
>> It's own
>> submodule, having its own eigendecompositions and svd and so forth, running
>> parallel to the current c-math impls? Why?
>>
>> Who would maintain it and write tests for it, and how do you explain to
>> end-users which they should use?
>>
>>
>>
>>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>>> wrote:
>>>> Ted Dunning a écrit :
>>>>> Actually, the reason that we have Colt in Mahout is it has proven
>>> impossible
>>>>> to get changes into commons math. We really, really wanted to use
>>> commons
>>>>> math rather than have our own linear algebra package, but it just proved
>>>>> impossible and we didn't want to wait forever.
>>>> If you really, really wants to use commons math and want changes to be
>>>> included, contribute them.
>>
>> I have submitted patches for the following tickets: MATH-312 (and acceptance
>> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
>> of which have appear to have had much progress on. All of my patches come
>> with unit tests for new functionality.
>
> I had these patches in my backlog and considered them accepted. I should
> have commited them before, sorry for that. I'll take care of them right now.
>
>>
>> On the other hand, when I opened the discussion about extending the
>> functions
>> package to enable composable functions (MATH-313), I got an entirely hostile
>> response, which only tempered as far as "+0" on adding it after discussion.
>
> The discussion was not entirely hostile as we get some intermediate
> consensus at some points. I understand your feelings after several
> patches that did not get committed fast enough.
>
> Please accept my apologizes for this.
>
> Luc
>
>>
>> In particular, my first step at making commons-math something Mahout could
>> standardize on for linear work was MATH-312, which I did submit a patch for,
>> and revised it many times after discussion about what is acceptable practice
>> in c-math. Not yet applied, months later. It's probably far out of date
>> now...
>>
>> Similarly, when I tried to ask what the status on decisions on whether to
>> adopt
>> MTJ or Colt, the statement by Phil was basically that commons-math would not
>> adopt anything which had any external dependencies or
>> not-easily-human-readable java source (which ruled out MTJ because of f2j
>> produced code), and which had to be fully tested and maintained prior to
>> adoption (which rules out Colt which has no unit tests yet).
>>
>> Ted and I weren't making "requests" for other people to do work, we were
>> wondering whether even offers to do some of the work would be accepted,
>> and for many of the questions/suggestions we had, it seems the desires
>> and requirements of the Mahout community were incompatible with those
>> of commons-math.
>>
>> -jake
>>
>>
>>> > I think the only change that was proposed and not done because of lack
>>>> of consensus was the inclusion of MTJ (and I don't consider the
>>>> discussion closed on that topic either, so it may still happen some
>>>> day). All the other changes that are desired are simply lacking someone
>>>> to do the work. There were proposals to extend the linear algebra API,
>>>> proposals to add more support for sparse matrices, proposals to get
>>>> partial decomposition ... But sparse contributions (pun intended).
>>>>
>>>> I try to do what I can, but as you have probably seen have been rather
>>>> silent since 2.0 release. For my part, I really, really need help. I
>>>> would like to fix the problems in the eigen decomposition and SVD but
>>>> need a good kick to get on it, and having only requests and no help is
>>>> not really motivating.
>>>>
>>>> Luc
>>>>
>>>>> If that problem were solved, then it would be great to depend on commons
>>>>> math. If that problem isn't solved, then there is no way to depend on
>>>>> commons math.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
>>>> wrote:
>>>>>> We can't possibly have a dependency on Mahout in the long term. Either
>>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>>> with two forks, which would be sad.
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>>> james@carmanconsulting.com>
>>>>>> wrote:
>>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>>
>>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>>> bimargulies@gmail.com>
>>>>>> wrote:
>>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>>> Mahout
>>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>>
>>>>>>>> I want to use the collections classes from Mahout as the core to a
>>> new
>>>>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>>>>> Trove does.
>>>>>>>>
>>>>>>>> The classes I want to start from depend on the classes that are in
>>> the
>>>>>>>> Mahout fork.
>>>>>>>>
>>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>>> submit that it would be more better if the mathematical code were in
>>>>>>>> commons-math. Was this option explored?
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Phil Steitz <ph...@gmail.com>.
Luc Maisonobe wrote:
> Jake Mannix a écrit :
>> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
>>
>>> This is interesting. We have a raft of mathematically qualified
>>> committers on Mahout, and this message asking for help on
>>> commons-math, and a raft of code marooned at mahout that wants to be
>>> in commons math. If I were one of those mathematically competant
>>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>>
>> The commons-math linear APIs have been described as effectively locked
>> until 3.0, due to back-compat requirements. This means that any code
>> contributed
>> into c-math would live in a parallel (no pun intended) to the linear
>> primitives which
>> exist already in there.
>>
>> Adopting something like MTJ or Colt in Mahout turned out to be easier,
>> because
>> we are on release 0.2 (heading for 0.3 now), and have less stringent
>> back-compat
>> requirements, so we are overhauling our linear apis (read: even user-facing
>> interface changes) to take advantage of useful parts of Colt, and are
>> planning on
>> using our Colt fork as the underlying implementation.
>>
>> Commons-math expressed that changing linear APIs is not something they can
>> do,
>> given the maturity of their library, so where would Colt *go* in c-math?
>> It's own
>> submodule, having its own eigendecompositions and svd and so forth, running
>> parallel to the current c-math impls? Why?
>>
>> Who would maintain it and write tests for it, and how do you explain to
>> end-users which they should use?
>>
>>
>>
>>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>>> wrote:
>>>> Ted Dunning a écrit :
>>>>> Actually, the reason that we have Colt in Mahout is it has proven
>>> impossible
>>>>> to get changes into commons math. We really, really wanted to use
>>> commons
>>>>> math rather than have our own linear algebra package, but it just proved
>>>>> impossible and we didn't want to wait forever.
>>>> If you really, really wants to use commons math and want changes to be
>>>> included, contribute them.
>> I have submitted patches for the following tickets: MATH-312 (and acceptance
>> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
>> of which have appear to have had much progress on. All of my patches come
>> with unit tests for new functionality.
>
> I had these patches in my backlog and considered them accepted. I should
> have commited them before, sorry for that. I'll take care of them right now.
>
>> On the other hand, when I opened the discussion about extending the
>> functions
>> package to enable composable functions (MATH-313), I got an entirely hostile
>> response, which only tempered as far as "+0" on adding it after discussion.
>
> The discussion was not entirely hostile as we get some intermediate
> consensus at some points. I understand your feelings after several
> patches that did not get committed fast enough.
>
> Please accept my apologizes for this.
I have to apologize as well here for lack of cycles to help get
[math] patches in recently. That should improve shortly.
Phil
>
> Luc
>
>> In particular, my first step at making commons-math something Mahout could
>> standardize on for linear work was MATH-312, which I did submit a patch for,
>> and revised it many times after discussion about what is acceptable practice
>> in c-math. Not yet applied, months later. It's probably far out of date
>> now...
>>
>> Similarly, when I tried to ask what the status on decisions on whether to
>> adopt
>> MTJ or Colt, the statement by Phil was basically that commons-math would not
>> adopt anything which had any external dependencies or
>> not-easily-human-readable java source (which ruled out MTJ because of f2j
>> produced code), and which had to be fully tested and maintained prior to
>> adoption (which rules out Colt which has no unit tests yet).
>>
>> Ted and I weren't making "requests" for other people to do work, we were
>> wondering whether even offers to do some of the work would be accepted,
>> and for many of the questions/suggestions we had, it seems the desires
>> and requirements of the Mahout community were incompatible with those
>> of commons-math.
>>
>> -jake
>>
>>
>>> > I think the only change that was proposed and not done because of lack
>>>> of consensus was the inclusion of MTJ (and I don't consider the
>>>> discussion closed on that topic either, so it may still happen some
>>>> day). All the other changes that are desired are simply lacking someone
>>>> to do the work. There were proposals to extend the linear algebra API,
>>>> proposals to add more support for sparse matrices, proposals to get
>>>> partial decomposition ... But sparse contributions (pun intended).
>>>>
>>>> I try to do what I can, but as you have probably seen have been rather
>>>> silent since 2.0 release. For my part, I really, really need help. I
>>>> would like to fix the problems in the eigen decomposition and SVD but
>>>> need a good kick to get on it, and having only requests and no help is
>>>> not really motivating.
>>>>
>>>> Luc
>>>>
>>>>> If that problem were solved, then it would be great to depend on commons
>>>>> math. If that problem isn't solved, then there is no way to depend on
>>>>> commons math.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
>>>> wrote:
>>>>>> We can't possibly have a dependency on Mahout in the long term. Either
>>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>>> with two forks, which would be sad.
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>>> james@carmanconsulting.com>
>>>>>> wrote:
>>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>>
>>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>>> bimargulies@gmail.com>
>>>>>> wrote:
>>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>>> Mahout
>>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>>
>>>>>>>> I want to use the collections classes from Mahout as the core to a
>>> new
>>>>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>>>>> Trove does.
>>>>>>>>
>>>>>>>> The classes I want to start from depend on the classes that are in
>>> the
>>>>>>>> Mahout fork.
>>>>>>>>
>>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>>> submit that it would be more better if the mathematical code were in
>>>>>>>> commons-math. Was this option explored?
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Jake Mannix <ja...@gmail.com>.
On Wed, Dec 9, 2009 at 12:46 PM, Luc Maisonobe <Lu...@free.fr>wrote:
> >
> > I have submitted patches for the following tickets: MATH-312 (and
> acceptance
> > of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
> > of which have appear to have had much progress on. All of my patches
> come
> > with unit tests for new functionality.
>
> I had these patches in my backlog and considered them accepted. I should
> have commited them before, sorry for that. I'll take care of them right
> now.
>
> >
> > On the other hand, when I opened the discussion about extending the
> > functions
> > package to enable composable functions (MATH-313), I got an entirely
> hostile
> > response, which only tempered as far as "+0" on adding it after
> discussion.
>
> The discussion was not entirely hostile as we get some intermediate
> consensus at some points. I understand your feelings after several
> patches that did not get committed fast enough.
>
> Please accept my apologizes for this.
>
Luc, looking back over that thread, I should say that no, it was not
entirely
hostile - and we did come to more of a consensus later on (although even
that consensus was given a very lukewarm +0 from Phil). I apologize for
any implication that I was being treated poorly or unfairly.
It just seems that as Benson says later, that the rate of change of c-math
and
mahout are rather different, and this discussion around those JIRA tickets
simply highlighted what this understanding was.
-jake
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Luc Maisonobe <Lu...@free.fr>.
Jake Mannix a écrit :
> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
>
>> This is interesting. We have a raft of mathematically qualified
>> committers on Mahout, and this message asking for help on
>> commons-math, and a raft of code marooned at mahout that wants to be
>> in commons math. If I were one of those mathematically competant
>> individuals, I'd be off attaching a patch or three to a JIRA or two
>>
>
> The commons-math linear APIs have been described as effectively locked
> until 3.0, due to back-compat requirements. This means that any code
> contributed
> into c-math would live in a parallel (no pun intended) to the linear
> primitives which
> exist already in there.
>
> Adopting something like MTJ or Colt in Mahout turned out to be easier,
> because
> we are on release 0.2 (heading for 0.3 now), and have less stringent
> back-compat
> requirements, so we are overhauling our linear apis (read: even user-facing
> interface changes) to take advantage of useful parts of Colt, and are
> planning on
> using our Colt fork as the underlying implementation.
>
> Commons-math expressed that changing linear APIs is not something they can
> do,
> given the maturity of their library, so where would Colt *go* in c-math?
> It's own
> submodule, having its own eigendecompositions and svd and so forth, running
> parallel to the current c-math impls? Why?
>
> Who would maintain it and write tests for it, and how do you explain to
> end-users which they should use?
>
>
>
>> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
>> wrote:
>>> Ted Dunning a écrit :
>>>> Actually, the reason that we have Colt in Mahout is it has proven
>> impossible
>>>> to get changes into commons math. We really, really wanted to use
>> commons
>>>> math rather than have our own linear algebra package, but it just proved
>>>> impossible and we didn't want to wait forever.
>>> If you really, really wants to use commons math and want changes to be
>>> included, contribute them.
>
> I have submitted patches for the following tickets: MATH-312 (and acceptance
> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
> of which have appear to have had much progress on. All of my patches come
> with unit tests for new functionality.
I had these patches in my backlog and considered them accepted. I should
have commited them before, sorry for that. I'll take care of them right now.
>
> On the other hand, when I opened the discussion about extending the
> functions
> package to enable composable functions (MATH-313), I got an entirely hostile
> response, which only tempered as far as "+0" on adding it after discussion.
The discussion was not entirely hostile as we get some intermediate
consensus at some points. I understand your feelings after several
patches that did not get committed fast enough.
Please accept my apologizes for this.
Luc
>
> In particular, my first step at making commons-math something Mahout could
> standardize on for linear work was MATH-312, which I did submit a patch for,
> and revised it many times after discussion about what is acceptable practice
> in c-math. Not yet applied, months later. It's probably far out of date
> now...
>
> Similarly, when I tried to ask what the status on decisions on whether to
> adopt
> MTJ or Colt, the statement by Phil was basically that commons-math would not
> adopt anything which had any external dependencies or
> not-easily-human-readable java source (which ruled out MTJ because of f2j
> produced code), and which had to be fully tested and maintained prior to
> adoption (which rules out Colt which has no unit tests yet).
>
> Ted and I weren't making "requests" for other people to do work, we were
> wondering whether even offers to do some of the work would be accepted,
> and for many of the questions/suggestions we had, it seems the desires
> and requirements of the Mahout community were incompatible with those
> of commons-math.
>
> -jake
>
>
>> > I think the only change that was proposed and not done because of lack
>>> of consensus was the inclusion of MTJ (and I don't consider the
>>> discussion closed on that topic either, so it may still happen some
>>> day). All the other changes that are desired are simply lacking someone
>>> to do the work. There were proposals to extend the linear algebra API,
>>> proposals to add more support for sparse matrices, proposals to get
>>> partial decomposition ... But sparse contributions (pun intended).
>>>
>>> I try to do what I can, but as you have probably seen have been rather
>>> silent since 2.0 release. For my part, I really, really need help. I
>>> would like to fix the problems in the eigen decomposition and SVD but
>>> need a good kick to get on it, and having only requests and no help is
>>> not really motivating.
>>>
>>> Luc
>>>
>>>> If that problem were solved, then it would be great to depend on commons
>>>> math. If that problem isn't solved, then there is no way to depend on
>>>> commons math.
>>>>
>>>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
>>> wrote:
>>>>> We can't possibly have a dependency on Mahout in the long term. Either
>>>>> we all go shares on code in some other piece of commons, or we end up
>>>>> with two forks, which would be sad.
>>>>>
>>>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
>> james@carmanconsulting.com>
>>>>> wrote:
>>>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>>>> this stuff, we can move it into a library in commons (which is
>>>>>> typically how stuff used to happen in Jakarta).
>>>>>>
>>>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
>> bimargulies@gmail.com>
>>>>> wrote:
>>>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>>>> CERN colt library forked. The Mahout fork is, of course, in the
>> Mahout
>>>>>>> tree under a Mahout Java package and Maven triple.
>>>>>>>
>>>>>>> I want to use the collections classes from Mahout as the core to a
>> new
>>>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>>>> Trove does.
>>>>>>>
>>>>>>> The classes I want to start from depend on the classes that are in
>> the
>>>>>>> Mahout fork.
>>>>>>>
>>>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>>>> submit that it would be more better if the mathematical code were in
>>>>>>> commons-math. Was this option explored?
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>>
>>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>>
>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Ted Dunning <te...@gmail.com>.
I don't want to denigrate the large contributions of Phil and Luc
(especially Luc), but I have drawn a stronger conclusion that it isn't worth
my time to contribute to commons-math because interesting changes likely
won't get accepted.
On Wed, Dec 9, 2009 at 11:58 AM, Jake Mannix <ja...@gmail.com> wrote:
> Ted and I weren't making "requests" for other people to do work, we were
> wondering whether even offers to do some of the work would be accepted,
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Ted Dunning <te...@gmail.com>.
Similarly, https://issues.apache.org/jira/browse/MATH-310 had code from a
contributor and got totally shot down basically with "We don't care what you
think, we don't want it". Extensive attempts on my part to find common
ground just got shot down by Phil with -1 and essentially no more discussion
than "don't like it".
Mahout has some unusual requirements and definitely can't use Math as is.
It is also fast moving and needs changes to actually happen. The experience
has been that only totally trivial non-changes get accepted (
https://issues.apache.org/jira/browse/MATH-267 and
https://issues.apache.org/jira/browse/MATH-222 are examples). Anything more
interesting seems to get bogged down in NIH or endless alternations of
"not-now we are about to have a major release and can't be distracted" and
"not-now because we aren't doing a major release and have stay compatible".
On Wed, Dec 9, 2009 at 11:58 AM, Jake Mannix <ja...@gmail.com> wrote:
> On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bimargulies@gmail.com
> >wrote:
>
> > This is interesting. We have a raft of mathematically qualified
> > committers on Mahout, and this message asking for help on
> > commons-math, and a raft of code marooned at mahout that wants to be
> > in commons math. If I were one of those mathematically competant
> > individuals, I'd be off attaching a patch or three to a JIRA or two
> >
>
> The commons-math linear APIs have been described as effectively locked
> until 3.0, due to back-compat requirements. This means that any code
> contributed
> into c-math would live in a parallel (no pun intended) to the linear
> primitives which
> exist already in there.
>
> Adopting something like MTJ or Colt in Mahout turned out to be easier,
> because
> we are on release 0.2 (heading for 0.3 now), and have less stringent
> back-compat
> requirements, so we are overhauling our linear apis (read: even user-facing
> interface changes) to take advantage of useful parts of Colt, and are
> planning on
> using our Colt fork as the underlying implementation.
>
> Commons-math expressed that changing linear APIs is not something they can
> do,
> given the maturity of their library, so where would Colt *go* in c-math?
> It's own
> submodule, having its own eigendecompositions and svd and so forth, running
> parallel to the current c-math impls? Why?
>
> Who would maintain it and write tests for it, and how do you explain to
> end-users which they should use?
>
>
>
> > On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
> > wrote:
> > > Ted Dunning a écrit :
> > >> Actually, the reason that we have Colt in Mahout is it has proven
> > impossible
> > >> to get changes into commons math. We really, really wanted to use
> > commons
> > >> math rather than have our own linear algebra package, but it just
> proved
> > >> impossible and we didn't want to wait forever.
> > >
> > > If you really, really wants to use commons math and want changes to be
> > > included, contribute them.
> >
>
> I have submitted patches for the following tickets: MATH-312 (and
> acceptance
> of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
> of which have appear to have had much progress on. All of my patches come
> with unit tests for new functionality.
>
> On the other hand, when I opened the discussion about extending the
> functions
> package to enable composable functions (MATH-313), I got an entirely
> hostile
> response, which only tempered as far as "+0" on adding it after discussion.
>
> In particular, my first step at making commons-math something Mahout could
> standardize on for linear work was MATH-312, which I did submit a patch
> for,
> and revised it many times after discussion about what is acceptable
> practice
> in c-math. Not yet applied, months later. It's probably far out of date
> now...
>
> Similarly, when I tried to ask what the status on decisions on whether to
> adopt
> MTJ or Colt, the statement by Phil was basically that commons-math would
> not
> adopt anything which had any external dependencies or
> not-easily-human-readable java source (which ruled out MTJ because of f2j
> produced code), and which had to be fully tested and maintained prior to
> adoption (which rules out Colt which has no unit tests yet).
>
> Ted and I weren't making "requests" for other people to do work, we were
> wondering whether even offers to do some of the work would be accepted,
> and for many of the questions/suggestions we had, it seems the desires
> and requirements of the Mahout community were incompatible with those
> of commons-math.
>
> -jake
>
>
> > > I think the only change that was proposed and not done because of lack
> > > of consensus was the inclusion of MTJ (and I don't consider the
> > > discussion closed on that topic either, so it may still happen some
> > > day). All the other changes that are desired are simply lacking someone
> > > to do the work. There were proposals to extend the linear algebra API,
> > > proposals to add more support for sparse matrices, proposals to get
> > > partial decomposition ... But sparse contributions (pun intended).
> > >
> > > I try to do what I can, but as you have probably seen have been rather
> > > silent since 2.0 release. For my part, I really, really need help. I
> > > would like to fix the problems in the eigen decomposition and SVD but
> > > need a good kick to get on it, and having only requests and no help is
> > > not really motivating.
> > >
> > > Luc
> > >
> > >>
> > >> If that problem were solved, then it would be great to depend on
> commons
> > >> math. If that problem isn't solved, then there is no way to depend on
> > >> commons math.
> > >>
> > >> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <
> bimargulies@gmail.com
> > >wrote:
> > >>
> > >>> We can't possibly have a dependency on Mahout in the long term.
> Either
> > >>> we all go shares on code in some other piece of commons, or we end up
> > >>> with two forks, which would be sad.
> > >>>
> > >>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
> > james@carmanconsulting.com>
> > >>> wrote:
> > >>>> I wouldn't like to see a dependency on mahout code in a "commons"
> > >>>> library. That seems kind of backwards. If Mahout wants to offload
> > >>>> this stuff, we can move it into a library in commons (which is
> > >>>> typically how stuff used to happen in Jakarta).
> > >>>>
> > >>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
> > bimargulies@gmail.com>
> > >>> wrote:
> > >>>>> Mahout now has a fork of a portion of the 'category A' portion of
> the
> > >>>>> CERN colt library forked. The Mahout fork is, of course, in the
> > Mahout
> > >>>>> tree under a Mahout Java package and Maven triple.
> > >>>>>
> > >>>>> I want to use the collections classes from Mahout as the core to a
> > new
> > >>>>> set of commons-primitives classes that do the useful things that
> GNU
> > >>>>> Trove does.
> > >>>>>
> > >>>>> The classes I want to start from depend on the classes that are in
> > the
> > >>>>> Mahout fork.
> > >>>>>
> > >>>>> As a temporary expedient, I can depend on their there. However, I
> > >>>>> submit that it would be more better if the mathematical code were
> in
> > >>>>> commons-math. Was this option explored?
> > >>>>>
> > >>>>>
> ---------------------------------------------------------------------
> > >>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>>>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>>>
> > >>>>>
> > >>>>
> ---------------------------------------------------------------------
> > >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>>
> > >>>>
> > >>> ---------------------------------------------------------------------
> > >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > >>> For additional commands, e-mail: dev-help@commons.apache.org
> > >>>
> > >>>
> > >>
> > >>
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
--
Ted Dunning, CTO
DeepDyve
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Jake Mannix <ja...@gmail.com>.
On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
> This is interesting. We have a raft of mathematically qualified
> committers on Mahout, and this message asking for help on
> commons-math, and a raft of code marooned at mahout that wants to be
> in commons math. If I were one of those mathematically competant
> individuals, I'd be off attaching a patch or three to a JIRA or two
>
The commons-math linear APIs have been described as effectively locked
until 3.0, due to back-compat requirements. This means that any code
contributed
into c-math would live in a parallel (no pun intended) to the linear
primitives which
exist already in there.
Adopting something like MTJ or Colt in Mahout turned out to be easier,
because
we are on release 0.2 (heading for 0.3 now), and have less stringent
back-compat
requirements, so we are overhauling our linear apis (read: even user-facing
interface changes) to take advantage of useful parts of Colt, and are
planning on
using our Colt fork as the underlying implementation.
Commons-math expressed that changing linear APIs is not something they can
do,
given the maturity of their library, so where would Colt *go* in c-math?
It's own
submodule, having its own eigendecompositions and svd and so forth, running
parallel to the current c-math impls? Why?
Who would maintain it and write tests for it, and how do you explain to
end-users which they should use?
> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
> wrote:
> > Ted Dunning a écrit :
> >> Actually, the reason that we have Colt in Mahout is it has proven
> impossible
> >> to get changes into commons math. We really, really wanted to use
> commons
> >> math rather than have our own linear algebra package, but it just proved
> >> impossible and we didn't want to wait forever.
> >
> > If you really, really wants to use commons math and want changes to be
> > included, contribute them.
>
I have submitted patches for the following tickets: MATH-312 (and acceptance
of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
of which have appear to have had much progress on. All of my patches come
with unit tests for new functionality.
On the other hand, when I opened the discussion about extending the
functions
package to enable composable functions (MATH-313), I got an entirely hostile
response, which only tempered as far as "+0" on adding it after discussion.
In particular, my first step at making commons-math something Mahout could
standardize on for linear work was MATH-312, which I did submit a patch for,
and revised it many times after discussion about what is acceptable practice
in c-math. Not yet applied, months later. It's probably far out of date
now...
Similarly, when I tried to ask what the status on decisions on whether to
adopt
MTJ or Colt, the statement by Phil was basically that commons-math would not
adopt anything which had any external dependencies or
not-easily-human-readable java source (which ruled out MTJ because of f2j
produced code), and which had to be fully tested and maintained prior to
adoption (which rules out Colt which has no unit tests yet).
Ted and I weren't making "requests" for other people to do work, we were
wondering whether even offers to do some of the work would be accepted,
and for many of the questions/suggestions we had, it seems the desires
and requirements of the Mahout community were incompatible with those
of commons-math.
-jake
> > I think the only change that was proposed and not done because of lack
> > of consensus was the inclusion of MTJ (and I don't consider the
> > discussion closed on that topic either, so it may still happen some
> > day). All the other changes that are desired are simply lacking someone
> > to do the work. There were proposals to extend the linear algebra API,
> > proposals to add more support for sparse matrices, proposals to get
> > partial decomposition ... But sparse contributions (pun intended).
> >
> > I try to do what I can, but as you have probably seen have been rather
> > silent since 2.0 release. For my part, I really, really need help. I
> > would like to fix the problems in the eigen decomposition and SVD but
> > need a good kick to get on it, and having only requests and no help is
> > not really motivating.
> >
> > Luc
> >
> >>
> >> If that problem were solved, then it would be great to depend on commons
> >> math. If that problem isn't solved, then there is no way to depend on
> >> commons math.
> >>
> >> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
> >wrote:
> >>
> >>> We can't possibly have a dependency on Mahout in the long term. Either
> >>> we all go shares on code in some other piece of commons, or we end up
> >>> with two forks, which would be sad.
> >>>
> >>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
> james@carmanconsulting.com>
> >>> wrote:
> >>>> I wouldn't like to see a dependency on mahout code in a "commons"
> >>>> library. That seems kind of backwards. If Mahout wants to offload
> >>>> this stuff, we can move it into a library in commons (which is
> >>>> typically how stuff used to happen in Jakarta).
> >>>>
> >>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
> bimargulies@gmail.com>
> >>> wrote:
> >>>>> Mahout now has a fork of a portion of the 'category A' portion of the
> >>>>> CERN colt library forked. The Mahout fork is, of course, in the
> Mahout
> >>>>> tree under a Mahout Java package and Maven triple.
> >>>>>
> >>>>> I want to use the collections classes from Mahout as the core to a
> new
> >>>>> set of commons-primitives classes that do the useful things that GNU
> >>>>> Trove does.
> >>>>>
> >>>>> The classes I want to start from depend on the classes that are in
> the
> >>>>> Mahout fork.
> >>>>>
> >>>>> As a temporary expedient, I can depend on their there. However, I
> >>>>> submit that it would be more better if the mathematical code were in
> >>>>> commons-math. Was this option explored?
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>
> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>
> >>>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Jake Mannix <ja...@gmail.com>.
On Wed, Dec 9, 2009 at 11:09 AM, Benson Margulies <bi...@gmail.com>wrote:
> This is interesting. We have a raft of mathematically qualified
> committers on Mahout, and this message asking for help on
> commons-math, and a raft of code marooned at mahout that wants to be
> in commons math. If I were one of those mathematically competant
> individuals, I'd be off attaching a patch or three to a JIRA or two
>
The commons-math linear APIs have been described as effectively locked
until 3.0, due to back-compat requirements. This means that any code
contributed
into c-math would live in a parallel (no pun intended) to the linear
primitives which
exist already in there.
Adopting something like MTJ or Colt in Mahout turned out to be easier,
because
we are on release 0.2 (heading for 0.3 now), and have less stringent
back-compat
requirements, so we are overhauling our linear apis (read: even user-facing
interface changes) to take advantage of useful parts of Colt, and are
planning on
using our Colt fork as the underlying implementation.
Commons-math expressed that changing linear APIs is not something they can
do,
given the maturity of their library, so where would Colt *go* in c-math?
It's own
submodule, having its own eigendecompositions and svd and so forth, running
parallel to the current c-math impls? Why?
Who would maintain it and write tests for it, and how do you explain to
end-users which they should use?
> On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr>
> wrote:
> > Ted Dunning a écrit :
> >> Actually, the reason that we have Colt in Mahout is it has proven
> impossible
> >> to get changes into commons math. We really, really wanted to use
> commons
> >> math rather than have our own linear algebra package, but it just proved
> >> impossible and we didn't want to wait forever.
> >
> > If you really, really wants to use commons math and want changes to be
> > included, contribute them.
>
I have submitted patches for the following tickets: MATH-312 (and acceptance
of that patch blocks my patch for MATH-314), MATH-316 and MATH-317, none
of which have appear to have had much progress on. All of my patches come
with unit tests for new functionality.
On the other hand, when I opened the discussion about extending the
functions
package to enable composable functions (MATH-313), I got an entirely hostile
response, which only tempered as far as "+0" on adding it after discussion.
In particular, my first step at making commons-math something Mahout could
standardize on for linear work was MATH-312, which I did submit a patch for,
and revised it many times after discussion about what is acceptable practice
in c-math. Not yet applied, months later. It's probably far out of date
now...
Similarly, when I tried to ask what the status on decisions on whether to
adopt
MTJ or Colt, the statement by Phil was basically that commons-math would not
adopt anything which had any external dependencies or
not-easily-human-readable java source (which ruled out MTJ because of f2j
produced code), and which had to be fully tested and maintained prior to
adoption (which rules out Colt which has no unit tests yet).
Ted and I weren't making "requests" for other people to do work, we were
wondering whether even offers to do some of the work would be accepted,
and for many of the questions/suggestions we had, it seems the desires
and requirements of the Mahout community were incompatible with those
of commons-math.
-jake
> > I think the only change that was proposed and not done because of lack
> > of consensus was the inclusion of MTJ (and I don't consider the
> > discussion closed on that topic either, so it may still happen some
> > day). All the other changes that are desired are simply lacking someone
> > to do the work. There were proposals to extend the linear algebra API,
> > proposals to add more support for sparse matrices, proposals to get
> > partial decomposition ... But sparse contributions (pun intended).
> >
> > I try to do what I can, but as you have probably seen have been rather
> > silent since 2.0 release. For my part, I really, really need help. I
> > would like to fix the problems in the eigen decomposition and SVD but
> > need a good kick to get on it, and having only requests and no help is
> > not really motivating.
> >
> > Luc
> >
> >>
> >> If that problem were solved, then it would be great to depend on commons
> >> math. If that problem isn't solved, then there is no way to depend on
> >> commons math.
> >>
> >> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bimargulies@gmail.com
> >wrote:
> >>
> >>> We can't possibly have a dependency on Mahout in the long term. Either
> >>> we all go shares on code in some other piece of commons, or we end up
> >>> with two forks, which would be sad.
> >>>
> >>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <
> james@carmanconsulting.com>
> >>> wrote:
> >>>> I wouldn't like to see a dependency on mahout code in a "commons"
> >>>> library. That seems kind of backwards. If Mahout wants to offload
> >>>> this stuff, we can move it into a library in commons (which is
> >>>> typically how stuff used to happen in Jakarta).
> >>>>
> >>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <
> bimargulies@gmail.com>
> >>> wrote:
> >>>>> Mahout now has a fork of a portion of the 'category A' portion of the
> >>>>> CERN colt library forked. The Mahout fork is, of course, in the
> Mahout
> >>>>> tree under a Mahout Java package and Maven triple.
> >>>>>
> >>>>> I want to use the collections classes from Mahout as the core to a
> new
> >>>>> set of commons-primitives classes that do the useful things that GNU
> >>>>> Trove does.
> >>>>>
> >>>>> The classes I want to start from depend on the classes that are in
> the
> >>>>> Mahout fork.
> >>>>>
> >>>>> As a temporary expedient, I can depend on their there. However, I
> >>>>> submit that it would be more better if the mathematical code were in
> >>>>> commons-math. Was this option explored?
> >>>>>
> >>>>> ---------------------------------------------------------------------
> >>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>>
> >>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >>> For additional commands, e-mail: dev-help@commons.apache.org
> >>>
> >>>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Benson Margulies <bi...@gmail.com>.
This is interesting. We have a raft of mathematically qualified
committers on Mahout, and this message asking for help on
commons-math, and a raft of code marooned at mahout that wants to be
in commons math. If I were one of those mathematically competant
individuals, I'd be off attaching a patch or three to a JIRA or two
...
On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> Ted Dunning a écrit :
>> Actually, the reason that we have Colt in Mahout is it has proven impossible
>> to get changes into commons math. We really, really wanted to use commons
>> math rather than have our own linear algebra package, but it just proved
>> impossible and we didn't want to wait forever.
>
> If you really, really wants to use commons math and want changes to be
> included, contribute them.
>
> I think the only change that was proposed and not done because of lack
> of consensus was the inclusion of MTJ (and I don't consider the
> discussion closed on that topic either, so it may still happen some
> day). All the other changes that are desired are simply lacking someone
> to do the work. There were proposals to extend the linear algebra API,
> proposals to add more support for sparse matrices, proposals to get
> partial decomposition ... But sparse contributions (pun intended).
>
> I try to do what I can, but as you have probably seen have been rather
> silent since 2.0 release. For my part, I really, really need help. I
> would like to fix the problems in the eigen decomposition and SVD but
> need a good kick to get on it, and having only requests and no help is
> not really motivating.
>
> Luc
>
>>
>> If that problem were solved, then it would be great to depend on commons
>> math. If that problem isn't solved, then there is no way to depend on
>> commons math.
>>
>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bi...@gmail.com>wrote:
>>
>>> We can't possibly have a dependency on Mahout in the long term. Either
>>> we all go shares on code in some other piece of commons, or we end up
>>> with two forks, which would be sad.
>>>
>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <ja...@carmanconsulting.com>
>>> wrote:
>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>> this stuff, we can move it into a library in commons (which is
>>>> typically how stuff used to happen in Jakarta).
>>>>
>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <bi...@gmail.com>
>>> wrote:
>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>> CERN colt library forked. The Mahout fork is, of course, in the Mahout
>>>>> tree under a Mahout Java package and Maven triple.
>>>>>
>>>>> I want to use the collections classes from Mahout as the core to a new
>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>> Trove does.
>>>>>
>>>>> The classes I want to start from depend on the classes that are in the
>>>>> Mahout fork.
>>>>>
>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>> submit that it would be more better if the mathematical code were in
>>>>> commons-math. Was this option explored?
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
Re: [math] getting changes included into commons-math (was Re: Home
for the colt fork)
Posted by Benson Margulies <bi...@gmail.com>.
This is interesting. We have a raft of mathematically qualified
committers on Mahout, and this message asking for help on
commons-math, and a raft of code marooned at mahout that wants to be
in commons math. If I were one of those mathematically competant
individuals, I'd be off attaching a patch or three to a JIRA or two
...
On Wed, Dec 9, 2009 at 1:48 PM, Luc Maisonobe <Lu...@free.fr> wrote:
> Ted Dunning a écrit :
>> Actually, the reason that we have Colt in Mahout is it has proven impossible
>> to get changes into commons math. We really, really wanted to use commons
>> math rather than have our own linear algebra package, but it just proved
>> impossible and we didn't want to wait forever.
>
> If you really, really wants to use commons math and want changes to be
> included, contribute them.
>
> I think the only change that was proposed and not done because of lack
> of consensus was the inclusion of MTJ (and I don't consider the
> discussion closed on that topic either, so it may still happen some
> day). All the other changes that are desired are simply lacking someone
> to do the work. There were proposals to extend the linear algebra API,
> proposals to add more support for sparse matrices, proposals to get
> partial decomposition ... But sparse contributions (pun intended).
>
> I try to do what I can, but as you have probably seen have been rather
> silent since 2.0 release. For my part, I really, really need help. I
> would like to fix the problems in the eigen decomposition and SVD but
> need a good kick to get on it, and having only requests and no help is
> not really motivating.
>
> Luc
>
>>
>> If that problem were solved, then it would be great to depend on commons
>> math. If that problem isn't solved, then there is no way to depend on
>> commons math.
>>
>> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bi...@gmail.com>wrote:
>>
>>> We can't possibly have a dependency on Mahout in the long term. Either
>>> we all go shares on code in some other piece of commons, or we end up
>>> with two forks, which would be sad.
>>>
>>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <ja...@carmanconsulting.com>
>>> wrote:
>>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>>> library. That seems kind of backwards. If Mahout wants to offload
>>>> this stuff, we can move it into a library in commons (which is
>>>> typically how stuff used to happen in Jakarta).
>>>>
>>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <bi...@gmail.com>
>>> wrote:
>>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>>> CERN colt library forked. The Mahout fork is, of course, in the Mahout
>>>>> tree under a Mahout Java package and Maven triple.
>>>>>
>>>>> I want to use the collections classes from Mahout as the core to a new
>>>>> set of commons-primitives classes that do the useful things that GNU
>>>>> Trove does.
>>>>>
>>>>> The classes I want to start from depend on the classes that are in the
>>>>> Mahout fork.
>>>>>
>>>>> As a temporary expedient, I can depend on their there. However, I
>>>>> submit that it would be more better if the mathematical code were in
>>>>> commons-math. Was this option explored?
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>>
>>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
[math] getting changes included into commons-math (was Re: Home for
the colt fork)
Posted by Luc Maisonobe <Lu...@free.fr>.
Ted Dunning a écrit :
> Actually, the reason that we have Colt in Mahout is it has proven impossible
> to get changes into commons math. We really, really wanted to use commons
> math rather than have our own linear algebra package, but it just proved
> impossible and we didn't want to wait forever.
If you really, really wants to use commons math and want changes to be
included, contribute them.
I think the only change that was proposed and not done because of lack
of consensus was the inclusion of MTJ (and I don't consider the
discussion closed on that topic either, so it may still happen some
day). All the other changes that are desired are simply lacking someone
to do the work. There were proposals to extend the linear algebra API,
proposals to add more support for sparse matrices, proposals to get
partial decomposition ... But sparse contributions (pun intended).
I try to do what I can, but as you have probably seen have been rather
silent since 2.0 release. For my part, I really, really need help. I
would like to fix the problems in the eigen decomposition and SVD but
need a good kick to get on it, and having only requests and no help is
not really motivating.
Luc
>
> If that problem were solved, then it would be great to depend on commons
> math. If that problem isn't solved, then there is no way to depend on
> commons math.
>
> On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bi...@gmail.com>wrote:
>
>> We can't possibly have a dependency on Mahout in the long term. Either
>> we all go shares on code in some other piece of commons, or we end up
>> with two forks, which would be sad.
>>
>> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <ja...@carmanconsulting.com>
>> wrote:
>>> I wouldn't like to see a dependency on mahout code in a "commons"
>>> library. That seems kind of backwards. If Mahout wants to offload
>>> this stuff, we can move it into a library in commons (which is
>>> typically how stuff used to happen in Jakarta).
>>>
>>> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <bi...@gmail.com>
>> wrote:
>>>> Mahout now has a fork of a portion of the 'category A' portion of the
>>>> CERN colt library forked. The Mahout fork is, of course, in the Mahout
>>>> tree under a Mahout Java package and Maven triple.
>>>>
>>>> I want to use the collections classes from Mahout as the core to a new
>>>> set of commons-primitives classes that do the useful things that GNU
>>>> Trove does.
>>>>
>>>> The classes I want to start from depend on the classes that are in the
>>>> Mahout fork.
>>>>
>>>> As a temporary expedient, I can depend on their there. However, I
>>>> submit that it would be more better if the mathematical code were in
>>>> commons-math. Was this option explored?
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>>> For additional commands, e-mail: dev-help@commons.apache.org
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Home for the colt fork
Posted by Ted Dunning <te...@gmail.com>.
Actually, the reason that we have Colt in Mahout is it has proven impossible
to get changes into commons math. We really, really wanted to use commons
math rather than have our own linear algebra package, but it just proved
impossible and we didn't want to wait forever.
If that problem were solved, then it would be great to depend on commons
math. If that problem isn't solved, then there is no way to depend on
commons math.
On Tue, Dec 8, 2009 at 6:19 AM, Benson Margulies <bi...@gmail.com>wrote:
> We can't possibly have a dependency on Mahout in the long term. Either
> we all go shares on code in some other piece of commons, or we end up
> with two forks, which would be sad.
>
> On Tue, Dec 8, 2009 at 8:33 AM, James Carman <ja...@carmanconsulting.com>
> wrote:
> > I wouldn't like to see a dependency on mahout code in a "commons"
> > library. That seems kind of backwards. If Mahout wants to offload
> > this stuff, we can move it into a library in commons (which is
> > typically how stuff used to happen in Jakarta).
> >
> > On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <bi...@gmail.com>
> wrote:
> >> Mahout now has a fork of a portion of the 'category A' portion of the
> >> CERN colt library forked. The Mahout fork is, of course, in the Mahout
> >> tree under a Mahout Java package and Maven triple.
> >>
> >> I want to use the collections classes from Mahout as the core to a new
> >> set of commons-primitives classes that do the useful things that GNU
> >> Trove does.
> >>
> >> The classes I want to start from depend on the classes that are in the
> >> Mahout fork.
> >>
> >> As a temporary expedient, I can depend on their there. However, I
> >> submit that it would be more better if the mathematical code were in
> >> commons-math. Was this option explored?
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> >> For additional commands, e-mail: dev-help@commons.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
--
Ted Dunning, CTO
DeepDyve
Re: Home for the colt fork
Posted by Benson Margulies <bi...@gmail.com>.
We can't possibly have a dependency on Mahout in the long term. Either
we all go shares on code in some other piece of commons, or we end up
with two forks, which would be sad.
On Tue, Dec 8, 2009 at 8:33 AM, James Carman <ja...@carmanconsulting.com> wrote:
> I wouldn't like to see a dependency on mahout code in a "commons"
> library. That seems kind of backwards. If Mahout wants to offload
> this stuff, we can move it into a library in commons (which is
> typically how stuff used to happen in Jakarta).
>
> On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <bi...@gmail.com> wrote:
>> Mahout now has a fork of a portion of the 'category A' portion of the
>> CERN colt library forked. The Mahout fork is, of course, in the Mahout
>> tree under a Mahout Java package and Maven triple.
>>
>> I want to use the collections classes from Mahout as the core to a new
>> set of commons-primitives classes that do the useful things that GNU
>> Trove does.
>>
>> The classes I want to start from depend on the classes that are in the
>> Mahout fork.
>>
>> As a temporary expedient, I can depend on their there. However, I
>> submit that it would be more better if the mathematical code were in
>> commons-math. Was this option explored?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
>> For additional commands, e-mail: dev-help@commons.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: Home for the colt fork
Posted by James Carman <ja...@carmanconsulting.com>.
I wouldn't like to see a dependency on mahout code in a "commons"
library. That seems kind of backwards. If Mahout wants to offload
this stuff, we can move it into a library in commons (which is
typically how stuff used to happen in Jakarta).
On Tue, Dec 8, 2009 at 8:18 AM, Benson Margulies <bi...@gmail.com> wrote:
> Mahout now has a fork of a portion of the 'category A' portion of the
> CERN colt library forked. The Mahout fork is, of course, in the Mahout
> tree under a Mahout Java package and Maven triple.
>
> I want to use the collections classes from Mahout as the core to a new
> set of commons-primitives classes that do the useful things that GNU
> Trove does.
>
> The classes I want to start from depend on the classes that are in the
> Mahout fork.
>
> As a temporary expedient, I can depend on their there. However, I
> submit that it would be more better if the mathematical code were in
> commons-math. Was this option explored?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org