You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Jake Mannix (JIRA)" <ji...@apache.org> on 2009/11/23 19:39:41 UTC
[jira] Created: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Separate and clearly label different SparseVector implementations
-----------------------------------------------------------------
Key: MAHOUT-206
URL: https://issues.apache.org/jira/browse/MAHOUT-206
Project: Mahout
Issue Type: Improvement
Components: Matrix
Affects Versions: 0.2
Environment: all
Reporter: Jake Mannix
Fix For: 0.3
Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label different SparseVector implementations
Posted by Grant Ingersoll <gs...@apache.org>.
Yes, I have lived this pain for a long time with Lucene. Personally, though, a lot of the pain comes from a fairly strict back compatibility policy that to me isn't always well founded given the release cycle Lucene usually operates under. I've always wished there was a @introducing annotation for interfaces, such that you could tell people what is coming down the pike.
I also often feel the right answer is a combination of both. New methods could be added on a new interface that is then applied to an Abstract class, thus it can be inherited by downstream implementors. People who don't inherit from the Abstract can choose to add the new interface if they see fit.
For now, we don't have any back compat commitments. I think once we get to 0.9, we can decide on that.
On Nov 24, 2009, at 3:21 PM, Jake Mannix wrote:
> Oof.
>
> So you're arguing this as a temporary thing, until our interfaces stabilize?
> It makes
> unit testing much harder this way, but I guess I see the rationale.
>
> If we do this, we need to leave a lot out of that base class - there may be
> some really
> big differences in implementation of these classes (for example: distributed
> / hdfs
> backed matrices vs locally memory-resident ones), so very very little should
> be
> assumed in the base impl. I guess more can be done in the vector case,
> however.
>
> -jake
>
> On Tue, Nov 24, 2009 at 12:08 PM, Ted Dunning <te...@gmail.com> wrote:
>
>> Yes. Interfaces are the problem that commons math have boxed themselves in
>> with. The Hadoop crew (especially Doug C) are adamant about using as few
>> interfaces as possible except as mixin signals and only in cases where the
>> interface really is going to be very, very stable.
>>
>> Our vector interfaces are definitely not going to be that stable for quite
>> a
>> while.
>>
>> On Tue, Nov 24, 2009 at 12:03 PM, Jake Mannix <ja...@gmail.com>
>> wrote:
>>
>>> Well we do use AbstractVector. Are you suggesting that we *not* have a
>>> Vector interface
>>> at all, and *only* have an abstract base class? Similarly for Matrix?
>>>
>>> -jake
>>>
>>> On Tue, Nov 24, 2009 at 11:57 AM, Ted Dunning <te...@gmail.com>
>>> wrote:
>>>
>>>> We should use abstract classes almost everywhere instead of interfaces
>> to
>>>> ease backward compatibility issues with user written extensions to
>>> Vectors
>>>> and Matrices.
>>>>
>>>> On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <
>> jira@apache.org
>>>>> wrote:
>>>>
>>>>> It seems like there is still some commonality between the two
>>>>> implementations (size, cardinality, etc.) that I think it would be
>>>>> worthwhile to keep SparseVector as an abstract class which the other
>>> two
>>>>> extend.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ted Dunning, CTO
>>>> DeepDyve
>>>>
>>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene:
http://www.lucidimagination.com/search
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Jake Mannix <ja...@gmail.com>.
Oof.
So you're arguing this as a temporary thing, until our interfaces stabilize?
It makes
unit testing much harder this way, but I guess I see the rationale.
If we do this, we need to leave a lot out of that base class - there may be
some really
big differences in implementation of these classes (for example: distributed
/ hdfs
backed matrices vs locally memory-resident ones), so very very little should
be
assumed in the base impl. I guess more can be done in the vector case,
however.
-jake
On Tue, Nov 24, 2009 at 12:08 PM, Ted Dunning <te...@gmail.com> wrote:
> Yes. Interfaces are the problem that commons math have boxed themselves in
> with. The Hadoop crew (especially Doug C) are adamant about using as few
> interfaces as possible except as mixin signals and only in cases where the
> interface really is going to be very, very stable.
>
> Our vector interfaces are definitely not going to be that stable for quite
> a
> while.
>
> On Tue, Nov 24, 2009 at 12:03 PM, Jake Mannix <ja...@gmail.com>
> wrote:
>
> > Well we do use AbstractVector. Are you suggesting that we *not* have a
> > Vector interface
> > at all, and *only* have an abstract base class? Similarly for Matrix?
> >
> > -jake
> >
> > On Tue, Nov 24, 2009 at 11:57 AM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > > We should use abstract classes almost everywhere instead of interfaces
> to
> > > ease backward compatibility issues with user written extensions to
> > Vectors
> > > and Matrices.
> > >
> > > On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <
> jira@apache.org
> > > >wrote:
> > >
> > > > It seems like there is still some commonality between the two
> > > > implementations (size, cardinality, etc.) that I think it would be
> > > > worthwhile to keep SparseVector as an abstract class which the other
> > two
> > > > extend.
> > > >
> > >
> > >
> > >
> > > --
> > > Ted Dunning, CTO
> > > DeepDyve
> > >
> >
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Ted Dunning <te...@gmail.com>.
On Thu, Nov 26, 2009 at 9:27 AM, Sean Owen <sr...@gmail.com> wrote:
> There are some more issues, I think, with approaching it this way
> (e.g. UnsupportedOperationException defers API contract problems to
> runtime, which could be bad), but I bow to practice, and also don't
> feel strongly about it.
>
>
You defer a few problems to run-time, but not the major ones. The run-time
deferral occurs when there is a new use of an old class implemented by an
outside party that assumes that the old class matches some new contract.
This is less common than the case where you want to extend available
functionality, even in an interim way, and you don't want to completely
break even the old uses of old classes in the process. This is a very
common problem in open source.
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Ted Dunning <te...@gmail.com>.
On Thu, Nov 26, 2009 at 9:27 AM, Sean Owen <sr...@gmail.com> wrote:
> 1. Not everything needs to be extensible. "Recommender" is obviously
> built to be extended. "Integer" is plainly not. So I'd just suggest we
> not try to plan for extension unless there's a plausible case for it.
>
Absolutely.
Commons math tried to plan for extension everywhere and they wound up for
great numbers of Interface/Abstract/Implementation/Factory class quadruples
for things that are akin to Integer. The result is a nearly unusable mess
in certain areas and it certainly is very difficult to understand for a
newcomer.
2. If extension is allowed, it must be on puropse and design for.
>
Yes.
> Making a class robust in the face of subclasses is not trivial and
> doesn't happen automatically. In my head there is more risk of
> breakage for extenders here, than just breaking compile-time
> contracts. The bugs are subtler. And once you figure out someone
> extended something in a slightly unexpected way, and need to support
> it, you get locked in to an unspoken 'API' you didn't intend.
>
Yes, yes, yes.
> This can be accomplished with good design with any reasonable
> approach, and that's all I really care about.
>
This should be the last word, but I am going to add that I think this is a
very important principle that we should all probably repeat at least once a
week. I know that I slack off on design in the heat of implementation. I
know it is rarely a time-saver and never pleasant to fix.
--
Ted Dunning, CTO
DeepDyve
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Sean Owen <sr...@gmail.com>.
There are some more issues, I think, with approaching it this way
(e.g. UnsupportedOperationException defers API contract problems to
runtime, which could be bad), but I bow to practice, and also don't
feel strongly about it.
Instead let me just state the underlying principles that I think we
agree on, and if adhered to, will lead us to design software without
problems:
1. Not everything needs to be extensible. "Recommender" is obviously
built to be extended. "Integer" is plainly not. So I'd just suggest we
not try to plan for extension unless there's a plausible case for it.
2. If extension is allowed, it must be on puropse and design for.
Making a class robust in the face of subclasses is not trivial and
doesn't happen automatically. In my head there is more risk of
breakage for extenders here, than just breaking compile-time
contracts. The bugs are subtler. And once you figure out someone
extended something in a slightly unexpected way, and need to support
it, you get locked in to an unspoken 'API' you didn't intend.
This can be accomplished with good design with any reasonable
approach, and that's all I really care about.
On Thu, Nov 26, 2009 at 2:07 PM, Grant Ingersoll <gs...@apache.org> wrote:
>
> On Nov 25, 2009, at 8:12 PM, Ted Dunning wrote:
>
>> Practically speaking, the huge advantage of the abstract class is that you
>> have lower update requirements and less duplicated code when augmenting the
>> interface. Yes, you can do the dual thing, but the practical experience
>> with Hadoop and Lucene has been that using just the abstract class which is
>> named as if it were an interface works about better in the long run. The
>> update requirements become very onerous when you are dealing with more than
>> one package that have to be updated (and which can't for some reason be
>> updated simultaneously).
>>
>> When adding methods, the standard practice is to add an implementation that
>> throws UnsupportedOperationException or something similar. Yes, you can do
>> this with interace+abstract if *everybody* codes just the right way, but
>> with the abstract only approach, there is one less thing for people to do
>> wrong.
>>
>> I took a long time to come around to this pattern of coding, but I finally
>> agree that publishing abstract classes really is better except where you
>> have to have an interface (for RPC or multiple inheritance). It only takes
>> a little bit of outside coding to run into the problem and the social cost
>> can be enormous.
>>
>
> Well said. It took me a long time too although I do think you can get interfaces right if they are kept small. And I think it really is hammered home by working on open libraries (as opposed to even open applications and especially to closed source) where you never know where the next good idea is coming from.
>
> At any rate, I'm not overly concerned about it just yet. Once the Colt stuff solidifies, we will have a much better idea of what it needs to be.
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label different SparseVector implementations
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 25, 2009, at 8:12 PM, Ted Dunning wrote:
> Practically speaking, the huge advantage of the abstract class is that you
> have lower update requirements and less duplicated code when augmenting the
> interface. Yes, you can do the dual thing, but the practical experience
> with Hadoop and Lucene has been that using just the abstract class which is
> named as if it were an interface works about better in the long run. The
> update requirements become very onerous when you are dealing with more than
> one package that have to be updated (and which can't for some reason be
> updated simultaneously).
>
> When adding methods, the standard practice is to add an implementation that
> throws UnsupportedOperationException or something similar. Yes, you can do
> this with interace+abstract if *everybody* codes just the right way, but
> with the abstract only approach, there is one less thing for people to do
> wrong.
>
> I took a long time to come around to this pattern of coding, but I finally
> agree that publishing abstract classes really is better except where you
> have to have an interface (for RPC or multiple inheritance). It only takes
> a little bit of outside coding to run into the problem and the social cost
> can be enormous.
>
Well said. It took me a long time too although I do think you can get interfaces right if they are kept small. And I think it really is hammered home by working on open libraries (as opposed to even open applications and especially to closed source) where you never know where the next good idea is coming from.
At any rate, I'm not overly concerned about it just yet. Once the Colt stuff solidifies, we will have a much better idea of what it needs to be.
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Ted Dunning <te...@gmail.com>.
Practically speaking, the huge advantage of the abstract class is that you
have lower update requirements and less duplicated code when augmenting the
interface. Yes, you can do the dual thing, but the practical experience
with Hadoop and Lucene has been that using just the abstract class which is
named as if it were an interface works about better in the long run. The
update requirements become very onerous when you are dealing with more than
one package that have to be updated (and which can't for some reason be
updated simultaneously).
When adding methods, the standard practice is to add an implementation that
throws UnsupportedOperationException or something similar. Yes, you can do
this with interace+abstract if *everybody* codes just the right way, but
with the abstract only approach, there is one less thing for people to do
wrong.
I took a long time to come around to this pattern of coding, but I finally
agree that publishing abstract classes really is better except where you
have to have an interface (for RPC or multiple inheritance). It only takes
a little bit of outside coding to run into the problem and the social cost
can be enormous.
On Tue, Nov 24, 2009 at 1:09 PM, Sean Owen <sr...@gmail.com> wrote:
> ...
> Abstract classes afford the possibility of adding methods plus
> implementation, without breaking anybody, so yeah I'm into abstract
> classes. But then that's no argument against an abstract class +
> interface, which would add a small bit of flexibility too.
>
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Sean Owen <sr...@gmail.com>.
True, and that advantage rarely comes up.
However declaring abstract methods in an abstract class has exactly
the same problem that adding interface methods does -- and I take it
this is the heart of the problem -- all implementors get broken
immediately. If you intend to add methods this way, neither one is any
different indeed, both have the same problem.
Abstract classes afford the possibility of adding methods plus
implementation, without breaking anybody, so yeah I'm into abstract
classes. But then that's no argument against an abstract class +
interface, which would add a small bit of flexibility too.
I don't feel strongly about it but do think the interface + abstract
approach is more conventional than declaring APIs through abstract
classes.
On Tue, Nov 24, 2009 at 8:49 PM, Yonik Seeley
<yo...@lucidimagination.com> wrote:
> The only advantage an interface has over an abstract class is multiple
> inheritance.
> You can use abstract classes like interfaces: make it possible to
> override all methods and avoiding state unless absolutely needed for
> back compat changes.
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Tue, Nov 24, 2009 at 3:30 PM, Sean Owen <sr...@gmail.com> wrote:
> I'm willing to be convinced but what is the theoretical argument for this?
Rather the opposite - it's a practical argument gained through experience.
> I am all for interfaces *and* abstract classes. You write the API in
> terms of interfaces for maximum flexibility. You provide abstract
> partial implementations for convenience. Everyone is happy.
The only advantage an interface has over an abstract class is multiple
inheritance.
You can use abstract classes like interfaces: make it possible to
override all methods and avoiding state unless absolutely needed for
back compat changes.
-Yonik
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label different SparseVector implementations
Posted by Grant Ingersoll <gs...@apache.org>.
On Nov 24, 2009, at 3:30 PM, Sean Owen wrote:
> I'm willing to be convinced but what is the theoretical argument for this?
See the Lucene archives: http://search.lucidimagination.com. There has been a lot of discussion on it. And I mean a lot. And then some. :-) Search for anything on interfaces, abstract classes or back compatibility.
>
> I am all for interfaces *and* abstract classes. You write the API in
> terms of interfaces for maximum flexibility. You provide abstract
> partial implementations for convenience. Everyone is happy.
I agree.
>
> The best argument I've seen against it is that it can be overkill. In
> super-performance-critical situations the dynamic dispatch overhead is
> perhaps worth thinking about, but that's rare. What else?
>
> I take the point about interfaces changing, but this is significant
> when you expect a lot of third-party implementers of your interfaces.
> I don't think that is true here.
I disagree here. In open source, you never know where the next good idea is coming from. We should just always assume they will change.
>
> On Tue, Nov 24, 2009 at 8:08 PM, Ted Dunning <te...@gmail.com> wrote:
>> Yes. Interfaces are the problem that commons math have boxed themselves in
>> with. The Hadoop crew (especially Doug C) are adamant about using as few
>> interfaces as possible except as mixin signals and only in cases where the
>> interface really is going to be very, very stable.
>>
>> Our vector interfaces are definitely not going to be that stable for quite a
>> while.
>>
>> On Tue, Nov 24, 2009 at 12:03 PM, Jake Mannix <ja...@gmail.com> wrote:
>>
>>> Well we do use AbstractVector. Are you suggesting that we *not* have a
>>> Vector interface
>>> at all, and *only* have an abstract base class? Similarly for Matrix?
>>>
>>> -jake
>>>
>>> On Tue, Nov 24, 2009 at 11:57 AM, Ted Dunning <te...@gmail.com>
>>> wrote:
>>>
>>>> We should use abstract classes almost everywhere instead of interfaces to
>>>> ease backward compatibility issues with user written extensions to
>>> Vectors
>>>> and Matrices.
>>>>
>>>> On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <jira@apache.org
>>>>> wrote:
>>>>
>>>>> It seems like there is still some commonality between the two
>>>>> implementations (size, cardinality, etc.) that I think it would be
>>>>> worthwhile to keep SparseVector as an abstract class which the other
>>> two
>>>>> extend.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ted Dunning, CTO
>>>> DeepDyve
>>>>
>>>
>>
>>
>>
>> --
>> Ted Dunning, CTO
>> DeepDyve
>>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem (Lucene/Solr/Nutch/Mahout/Tika/Droids) using Solr/Lucene:
http://www.lucidimagination.com/search
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Sean Owen <sr...@gmail.com>.
I'm willing to be convinced but what is the theoretical argument for this?
I am all for interfaces *and* abstract classes. You write the API in
terms of interfaces for maximum flexibility. You provide abstract
partial implementations for convenience. Everyone is happy.
The best argument I've seen against it is that it can be overkill. In
super-performance-critical situations the dynamic dispatch overhead is
perhaps worth thinking about, but that's rare. What else?
I take the point about interfaces changing, but this is significant
when you expect a lot of third-party implementers of your interfaces.
I don't think that is true here.
On Tue, Nov 24, 2009 at 8:08 PM, Ted Dunning <te...@gmail.com> wrote:
> Yes. Interfaces are the problem that commons math have boxed themselves in
> with. The Hadoop crew (especially Doug C) are adamant about using as few
> interfaces as possible except as mixin signals and only in cases where the
> interface really is going to be very, very stable.
>
> Our vector interfaces are definitely not going to be that stable for quite a
> while.
>
> On Tue, Nov 24, 2009 at 12:03 PM, Jake Mannix <ja...@gmail.com> wrote:
>
>> Well we do use AbstractVector. Are you suggesting that we *not* have a
>> Vector interface
>> at all, and *only* have an abstract base class? Similarly for Matrix?
>>
>> -jake
>>
>> On Tue, Nov 24, 2009 at 11:57 AM, Ted Dunning <te...@gmail.com>
>> wrote:
>>
>> > We should use abstract classes almost everywhere instead of interfaces to
>> > ease backward compatibility issues with user written extensions to
>> Vectors
>> > and Matrices.
>> >
>> > On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <jira@apache.org
>> > >wrote:
>> >
>> > > It seems like there is still some commonality between the two
>> > > implementations (size, cardinality, etc.) that I think it would be
>> > > worthwhile to keep SparseVector as an abstract class which the other
>> two
>> > > extend.
>> > >
>> >
>> >
>> >
>> > --
>> > Ted Dunning, CTO
>> > DeepDyve
>> >
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Ted Dunning <te...@gmail.com>.
Yes. Interfaces are the problem that commons math have boxed themselves in
with. The Hadoop crew (especially Doug C) are adamant about using as few
interfaces as possible except as mixin signals and only in cases where the
interface really is going to be very, very stable.
Our vector interfaces are definitely not going to be that stable for quite a
while.
On Tue, Nov 24, 2009 at 12:03 PM, Jake Mannix <ja...@gmail.com> wrote:
> Well we do use AbstractVector. Are you suggesting that we *not* have a
> Vector interface
> at all, and *only* have an abstract base class? Similarly for Matrix?
>
> -jake
>
> On Tue, Nov 24, 2009 at 11:57 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > We should use abstract classes almost everywhere instead of interfaces to
> > ease backward compatibility issues with user written extensions to
> Vectors
> > and Matrices.
> >
> > On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <jira@apache.org
> > >wrote:
> >
> > > It seems like there is still some commonality between the two
> > > implementations (size, cardinality, etc.) that I think it would be
> > > worthwhile to keep SparseVector as an abstract class which the other
> two
> > > extend.
> > >
> >
> >
> >
> > --
> > Ted Dunning, CTO
> > DeepDyve
> >
>
--
Ted Dunning, CTO
DeepDyve
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Jake Mannix <ja...@gmail.com>.
Well we do use AbstractVector. Are you suggesting that we *not* have a
Vector interface
at all, and *only* have an abstract base class? Similarly for Matrix?
-jake
On Tue, Nov 24, 2009 at 11:57 AM, Ted Dunning <te...@gmail.com> wrote:
> We should use abstract classes almost everywhere instead of interfaces to
> ease backward compatibility issues with user written extensions to Vectors
> and Matrices.
>
> On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <jira@apache.org
> >wrote:
>
> > It seems like there is still some commonality between the two
> > implementations (size, cardinality, etc.) that I think it would be
> > worthwhile to keep SparseVector as an abstract class which the other two
> > extend.
> >
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>
Re: [jira] Commented: (MAHOUT-206) Separate and clearly label
different SparseVector implementations
Posted by Ted Dunning <te...@gmail.com>.
We should use abstract classes almost everywhere instead of interfaces to
ease backward compatibility issues with user written extensions to Vectors
and Matrices.
On Tue, Nov 24, 2009 at 9:38 AM, Grant Ingersoll (JIRA) <ji...@apache.org>wrote:
> It seems like there is still some commonality between the two
> implementations (size, cardinality, etc.) that I think it would be
> worthwhile to keep SparseVector as an abstract class which the other two
> extend.
>
--
Ted Dunning, CTO
DeepDyve
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Shashikant Kore (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781793#action_12781793 ]
Shashikant Kore commented on MAHOUT-206:
----------------------------------------
>From my observation, the input vectors are immutable. They will be benefit from the array-based sparse vector implementation. For centroids, the hash-based vector works great.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Fix For: 0.3
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12783788#action_12783788 ]
Jake Mannix commented on MAHOUT-206:
------------------------------------
*bump*
Hey Grant, have you tried out this version of the patch? Looks like it could resolve MAHOUT-201, MAHOUT-206, and MAHOUT-207 all in one go if it's looks good.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Grant Ingersoll
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jake Mannix updated MAHOUT-206:
-------------------------------
Attachment: MAHOUT-206.patch
This adds back SparseVector now as the parent of both sparse impls, and extending AbstractVector.
There are certainly optimizations which can still be done, and again, no work has been done to dig through all the places where RandomAccessSparseVectors are created and choose whether it would be better to have SequentialAccessSparseVector.
This patch also has the fix for MAHOUT-207.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Grant Ingersoll
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800273#action_12800273 ]
Jake Mannix commented on MAHOUT-206:
------------------------------------
bq. I had to delete another instance of TestVectorWritable in utils - does that make sense?
It does - I moved it and deleted the old one, but it must have snuck back into the patch.
If you've got it cleaned up and ready to commit, sure, thanks!
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-206.
------------------------------
Resolution: Fixed
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800271#action_12800271 ]
Sean Owen commented on MAHOUT-206:
----------------------------------
It all passes for me. I had to delete another instance of TestVectorWritable in utils -- does that make sense? I admit I still had patch problems here but thought I manually fixed them all.
I can handle issues like copyright headers.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Grant Ingersoll reassigned MAHOUT-206:
--------------------------------------
Assignee: Grant Ingersoll
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Grant Ingersoll
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800188#action_12800188 ]
Sean Owen commented on MAHOUT-206:
----------------------------------
You want me to try this and commit it?
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jake Mannix updated MAHOUT-206:
-------------------------------
Attachment: MAHOUT-206.patch
Patch renames the hash-based SparseVector with RandomAccessSparseVector, and adds back the old form of SparseVector (backed by OrderedIntDoubleMapping) named now as SequentialAccessSparseVector.
Unit tests added testing both forms of sparse vector.
BUT, I have not gone through all the places where we use SparseVector instances and deliberately picked SequetialAccess or RandomAccess choices for the sparse impl. All usage is done as RandomAccessSparseVector (ie. by the refactor-rename on trunk of SparseVector to RandomAccessSparseVector), and we will need to intentionally swap in SequentialAccessSparseVector in the primarily immutable/read-only case.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12782054#action_12782054 ]
Grant Ingersoll commented on MAHOUT-206:
----------------------------------------
Jake, there's something weird in this patch in regards to SparseVector. It didn't delete the file, but instead left it empty.
It seems like there is still some commonality between the two implementations (size, cardinality, etc.) that I think it would be worthwhile to keep SparseVector as an abstract class which the other two extend.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Grant Ingersoll
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jake Mannix reassigned MAHOUT-206:
----------------------------------
Assignee: Jake Mannix (was: Grant Ingersoll)
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12781701#action_12781701 ]
Grant Ingersoll commented on MAHOUT-206:
----------------------------------------
Sorry, yes, I missed that and I agree we do need it.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Fix For: 0.3
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12800268#action_12800268 ]
Jake Mannix commented on MAHOUT-206:
------------------------------------
If you (or anyone else) wants to try this out for sanity checking, I can commit it later too. I think I may have missed the license headers on the new files though, I'll need to add that before committing.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787596#action_12787596 ]
Jake Mannix commented on MAHOUT-206:
------------------------------------
Grant, I can take over this one, since it's my changes.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Matrix
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795557#action_12795557 ]
Jake Mannix commented on MAHOUT-206:
------------------------------------
Ugh, ok I've revived this patch, and it compiles again, after all the math / vector changes, but it's not passing all tests. I'll post it here as a progress indicator tomorrow anyways.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAHOUT-206) Separate and clearly label different
SparseVector implementations
Posted by "Jake Mannix (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jake Mannix updated MAHOUT-206:
-------------------------------
Attachment: MAHOUT-206.patch
Patch brought up to date with trunk, again fairly invasive, because it kills SparseVector, replaces with the two subclasses optimized for different use cases. In basically all cases, SparseVector -> RandomAccessSparseVector throughout user code, but this is probably *not* optimal. Cases where the vector being used is treated in a read-only way, it is more efficient to use SequentialAccessSparseVector, but I did not hunt down all those situations.
> Separate and clearly label different SparseVector implementations
> -----------------------------------------------------------------
>
> Key: MAHOUT-206
> URL: https://issues.apache.org/jira/browse/MAHOUT-206
> Project: Mahout
> Issue Type: Improvement
> Components: Math
> Affects Versions: 0.2
> Environment: all
> Reporter: Jake Mannix
> Assignee: Jake Mannix
> Fix For: 0.3
>
> Attachments: MAHOUT-206.patch, MAHOUT-206.patch, MAHOUT-206.patch
>
>
> Shashi's last patch on MAHOUT-165 swapped out the int/double parallel array impl of SparseVector for an OpenIntDoubleMap (hash-based) one. We actually need both, as I think I've mentioned a gazillion times.
> There was a patch, long ago, on MAHOUT-165, in which Ted had OrderedIntDoubleVector, and OpenIntDoubleHashVector (or something to that effect), and neither of them are called SparseVector. I like this, because it forces people to choose what kind of SparseVector they want (and they should: sparse is an optimization, and the client should make a conscious decision what they're optimizing for).
> We could call them RandomAccessSparseVector and SequentialAccessSparseVector, to be really obvious.
> But really, the important part is we have both.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.