You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Robin Anil <ro...@gmail.com> on 2010/10/02 20:23:48 UTC

Move DictionaryVectorizer to Core

How do you feel like moving the DictionaryVectorizer and Colloc generator to
the Core under vectorizers package instead of keeping them under utils.
FeatureEncoders will also be moved under vectorizers. I want to add a
Wrapper which takes a Vectorizer and converts input data to vectors. Its the
missing piece of the Classifier puzzle

o.a.m.vectorizer.dictionary
o.a.m.vectorizer.hashed or something funkier?

What do you think about this?

Robin

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
First step I am commiting the move. Will extend TextValueEncoder as you
suggest in a later patch


On Tue, Oct 5, 2010 at 3:43 AM, Ted Dunning <te...@gmail.com> wrote:

> I did.  Looked wrong, but I may have missed where the files were added back
> in.
>
> On Mon, Oct 4, 2010 at 3:07 PM, Robin Anil <ro...@gmail.com> wrote:
>
> > > Moving them will be a momentary pain, but not so bad.
> > >
> > Cool. Check the patch.
>

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
I did.  Looked wrong, but I may have missed where the files were added back
in.

On Mon, Oct 4, 2010 at 3:07 PM, Robin Anil <ro...@gmail.com> wrote:

> > Moving them will be a momentary pain, but not so bad.
> >
> Cool. Check the patch.

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
The basic idea of this hierarchy is that an encoder mutates a vector to
represent a feature.

That is the basic addToVector function which can take a byte[] or String and
a weight.   There isn't a byte => int function.

The hashed feature style does this by hashed probing.  The non-hashed style
does it by dictionary directed single probing.

On Mon, Oct 4, 2010 at 3:07 PM, Robin Anil <ro...@gmail.com> wrote:

> > I don't understand what these do.  Are they really just a dictionary
> > interface?
> >
> I assumed the basic encoder is one which maps a byte to an int.  (Probing
> is
> a overlay technique over it)

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
I would be more comfortable with this, actually.  It avoids a rename of the
FeatureVectorEncoder and
re-uses lots more existing code.  Having probes() throw
UnimplementedOperationException is fine by
me.

On Mon, Oct 4, 2010 at 3:07 PM, Robin Anil <ro...@gmail.com> wrote:

> > It seems to me that a dictionary based encoder is really no different
> from
> > any
> > hashed feature except that the hash function is based on the dictionary
> > rather than
> > a hash function, the weight is derived from the dictionary and the
> encoder
> > really only
> > supports a single probe.
> >
> > All of this seems doable with one or two sub-classes of the current
> > TextValueEncoder.
> > If you want to roll in Lucene based analysis, then sub-classing
> > LuceneTextValueEncoder
> > would be better.
> >
> I can proceed this way as well. Just need to move it around.  So are you
> more comfortable this way i.e. By throwing exception with probes?

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
On Tue, Oct 5, 2010 at 3:00 AM, Ted Dunning <te...@gmail.com> wrote:

> I would like to review these changes before they happen.  I have some folks
> using these in production
> and changes that surprise them would not be good.  (having 0.4 right now
> may
> be a god-send)


OK.

>

 On Mon, Oct 4, 2010 at 12:49 PM, Robin Anil <ro...@gmail.com> wrote:
>
> > I am ready to move FeatureVectorEncoders under vectorizer.encoder.* . I
> > wanted to move them under vectorizer but it seems these are homogenous
> and
> > should be kept separate from jobs and other classes.
> >
>
> Moving them will be a momentary pain, but not so bad.
>
Cool. Check the patch.

>
>
> > I need to create a Dictionary Based FeatureEncoder for that I am thinking
> > about the following.
> >
> > I will be renaming FeatureVectorEncoder as ProbedFeatureVectorEncoder
> >
>
> I would rather HashedFeatureVectorEncoder if a rename is really necessary.
> I
> am not convinced that it is.
>
Yeah HashedFeatureVectorEncoder is clearer.

>
>
> > abstract class
> > This abstract class will extend a FeatureEncoder interface having two
> > functions int encode(String) and int encode(byte[])
> >
>
> I don't understand what these do.  Are they really just a dictionary
> interface?
>
I assumed the basic encoder is one which maps a byte to an int.  (Probing is
a overlay technique over it)

>
>
> >
> > I will implement this interface in two FeatureEncoders: TFTextEncoder and
> > TFIDFTextEncoder
> >
>
> What about looking at the current TextValueEncoder and simply replacing the
> hash
> function with a dictionary lookup?  In fact, this might be done at the
> WordValueEncoder
> level.  setProbes would call IllegalSomethingOrOtherException.


> It seems to me that a dictionary based encoder is really no different from
> any
> hashed feature except that the hash function is based on the dictionary
> rather than
> a hash function, the weight is derived from the dictionary and the encoder
> really only
> supports a single probe.
>
> All of this seems doable with one or two sub-classes of the current
> TextValueEncoder.
> If you want to roll in Lucene based analysis, then sub-classing
> LuceneTextValueEncoder
> would be better.
>
I can proceed this way as well. Just need to move it around.  So are you
more comfortable this way i.e. By throwing exception with probes?

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
I would like to review these changes before they happen.  I have some folks
using these in production
and changes that surprise them would not be good.  (having 0.4 right now may
be a god-send)

On Mon, Oct 4, 2010 at 12:49 PM, Robin Anil <ro...@gmail.com> wrote:

> I am ready to move FeatureVectorEncoders under vectorizer.encoder.* . I
> wanted to move them under vectorizer but it seems these are homogenous and
> should be kept separate from jobs and other classes.
>

Moving them will be a momentary pain, but not so bad.


> I need to create a Dictionary Based FeatureEncoder for that I am thinking
> about the following.
>
> I will be renaming FeatureVectorEncoder as ProbedFeatureVectorEncoder
>

I would rather HashedFeatureVectorEncoder if a rename is really necessary. I
am not convinced that it is.


> abstract class
> This abstract class will extend a FeatureEncoder interface having two
> functions int encode(String) and int encode(byte[])
>

I don't understand what these do.  Are they really just a dictionary
interface?


>
> I will implement this interface in two FeatureEncoders: TFTextEncoder and
> TFIDFTextEncoder
>

What about looking at the current TextValueEncoder and simply replacing the
hash
function with a dictionary lookup?  In fact, this might be done at the
WordValueEncoder
level.  setProbes would call IllegalSomethingOrOtherException.

It seems to me that a dictionary based encoder is really no different from
any
hashed feature except that the hash function is based on the dictionary
rather than
a hash function, the weight is derived from the dictionary and the encoder
really only
supports a single probe.

All of this seems doable with one or two sub-classes of the current
TextValueEncoder.
If you want to roll in Lucene based analysis, then sub-classing
LuceneTextValueEncoder
would be better.

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
I am ready to move FeatureVectorEncoders under vectorizer.encoder.* . I
wanted to move them under vectorizer but it seems these are homogenous and
should be kept separate from jobs and other classes.

I need to create a Dictionary Based FeatureEncoder for that I am thinking
about the following.

I will be renaming FeatureVectorEncoder as ProbedFeatureVectorEncoder
abstract class
This abstract class will extend a FeatureEncoder interface having two
functions int encode(String) and int encode(byte[])

I will implement this interface in two FeatureEncoders: TFTextEncoder and
TFIDFTextEncoder

Ted, and others,  how does this sound?

On Mon, Oct 4, 2010 at 6:30 AM, Grant Ingersoll <gs...@apache.org> wrote:

>
> On Oct 2, 2010, at 2:23 PM, Robin Anil wrote:
>
> > How do you feel like moving the DictionaryVectorizer and Colloc generator
> to
> > the Core under vectorizers package instead of keeping them under utils.
> > FeatureEncoders will also be moved under vectorizers. I want to add a
> > Wrapper which takes a Vectorizer and converts input data to vectors. Its
> the
> > missing piece of the Classifier puzzle
>
> Why does that require Vectorizer be moved to Core?  Vectorizer seems like a
> util to me.
>
> >
> > o.a.m.vectorizer.dictionary
> > o.a.m.vectorizer.hashed or something funkier?
> >
> > What do you think about this?
> >
> > Robin
>
> --------------------------
> Grant Ingersoll
> http://lucenerevolution.org Apache Lucene/Solr Conference, Boston Oct 7-8
>
>

Re: Move DictionaryVectorizer to Core

Posted by Grant Ingersoll <gs...@apache.org>.
On Oct 2, 2010, at 2:23 PM, Robin Anil wrote:

> How do you feel like moving the DictionaryVectorizer and Colloc generator to
> the Core under vectorizers package instead of keeping them under utils.
> FeatureEncoders will also be moved under vectorizers. I want to add a
> Wrapper which takes a Vectorizer and converts input data to vectors. Its the
> missing piece of the Classifier puzzle

Why does that require Vectorizer be moved to Core?  Vectorizer seems like a util to me.

> 
> o.a.m.vectorizer.dictionary
> o.a.m.vectorizer.hashed or something funkier?
> 
> What do you think about this?
> 
> Robin

--------------------------
Grant Ingersoll
http://lucenerevolution.org Apache Lucene/Solr Conference, Boston Oct 7-8


Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
Does it makes sense with our current definition of core utils and examples.
My argument is that tfidf jobs shouldn't be in utils.

On Sat, Oct 2, 2010 at 11:53 PM, Robin Anil <ro...@gmail.com> wrote:

> How do you feel like moving the DictionaryVectorizer and Colloc generator
> to the Core under vectorizers package instead of keeping them under utils.
> FeatureEncoders will also be moved under vectorizers. I want to add a
> Wrapper which takes a Vectorizer and converts input data to vectors. Its the
> missing piece of the Classifier puzzle
>
> o.a.m.vectorizer.dictionary
> o.a.m.vectorizer.hashed or something funkier?
>
> What do you think about this?
>
> Robin
>
>
>

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
Yes.

On Sun, Oct 3, 2010 at 7:37 AM, Robin Anil <ro...@gmail.com> wrote:

> So how about it? If I keep the patch longer I will get conflicts. Its a
> major move. Say yes or no
>
>
> Robin
>
>
> On Sun, Oct 3, 2010 at 4:05 AM, Robin Anil <ro...@gmail.com> wrote:
>
> > I just uploaded a patch for moving DictionaryVectorizer. Take a look
> >
> >
> > On Sun, Oct 3, 2010 at 1:09 AM, Robin Anil <ro...@gmail.com> wrote:
> >
> >> Ok.
> >>
> >>
> >> On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <ted.dunning@gmail.com
> >wrote:
> >>
> >>> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we are
> >>> talking about.
> >>>
> >>> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com>
> >>> wrote:
> >>>
> >>> > i.e
> >>> > o.a.m.vectorizer.dictionary
> >>> > o.a.m.vectorizer.hashencoder
> >>> >
> >>> >
> >>> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > Same question applies to the FeatureValueEncoder family currently
> >>> under
> >>> > > classifier.vectors.*
> >>> > >
> >>> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com>
> >>> > wrote:
> >>> > >
> >>> > > > How do you feel like moving the DictionaryVectorizer and Colloc
> >>> > generator
> >>> > > > to
> >>> > > > the Core under vectorizers package instead of keeping them under
> >>> utils.
> >>> > > > FeatureEncoders will also be moved under vectorizers. I want to
> add
> >>> a
> >>> > > > Wrapper which takes a Vectorizer and converts input data to
> >>> vectors.
> >>> > Its
> >>> > > > the
> >>> > > > missing piece of the Classifier puzzle
> >>> > > >
> >>> > > > o.a.m.vectorizer.dictionary
> >>> > > > o.a.m.vectorizer.hashed or something funkier?
> >>> > > >
> >>> > > > What do you think about this?
> >>> > > >
> >>> > > > Robin
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
So how about it? If I keep the patch longer I will get conflicts. Its a
major move. Say yes or no


Robin


On Sun, Oct 3, 2010 at 4:05 AM, Robin Anil <ro...@gmail.com> wrote:

> I just uploaded a patch for moving DictionaryVectorizer. Take a look
>
>
> On Sun, Oct 3, 2010 at 1:09 AM, Robin Anil <ro...@gmail.com> wrote:
>
>> Ok.
>>
>>
>> On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <te...@gmail.com>wrote:
>>
>>> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we are
>>> talking about.
>>>
>>> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com>
>>> wrote:
>>>
>>> > i.e
>>> > o.a.m.vectorizer.dictionary
>>> > o.a.m.vectorizer.hashencoder
>>> >
>>> >
>>> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com>
>>> > wrote:
>>> >
>>> > > Same question applies to the FeatureValueEncoder family currently
>>> under
>>> > > classifier.vectors.*
>>> > >
>>> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com>
>>> > wrote:
>>> > >
>>> > > > How do you feel like moving the DictionaryVectorizer and Colloc
>>> > generator
>>> > > > to
>>> > > > the Core under vectorizers package instead of keeping them under
>>> utils.
>>> > > > FeatureEncoders will also be moved under vectorizers. I want to add
>>> a
>>> > > > Wrapper which takes a Vectorizer and converts input data to
>>> vectors.
>>> > Its
>>> > > > the
>>> > > > missing piece of the Classifier puzzle
>>> > > >
>>> > > > o.a.m.vectorizer.dictionary
>>> > > > o.a.m.vectorizer.hashed or something funkier?
>>> > > >
>>> > > > What do you think about this?
>>> > > >
>>> > > > Robin
>>> > > >
>>> > >
>>> >
>>>
>>
>>
>

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
I have checked through top half of the patch(till AbstractJob). the Patch
looks good.


On Sun, Oct 3, 2010 at 4:48 AM, Robin Anil <ro...@gmail.com> wrote:

> I checked the patch. It has nothing to do with utils/ folder and the files
> I moved to core. So MAHOUT-521 is in the clear
>
>
> On Sun, Oct 3, 2010 at 4:21 AM, Ted Dunning <te...@gmail.com> wrote:
>
>> Robin, (and everybody)
>>
>> Watch out for pending wide-scale changes from MAHOUT-480.
>>
>> Anybody who would like to avoid those changes hanging fire can do a quick
>> (right) review of the pending patch.  Only 93
>> files are changed so it should be a breeze!
>>
>> Eugen did a ton of work to get all those changes done in the first place.
>>  I
>> found a few discrepancies and would
>> like somebody to check my checking.  I will commit the changes as soon as
>> I
>> am sure we got these right.
>>
>> On Sat, Oct 2, 2010 at 3:35 PM, Robin Anil <ro...@gmail.com> wrote:
>>
>> > I just uploaded a patch for moving DictionaryVectorizer. Take a look
>> >
>> > On Sun, Oct 3, 2010 at 1:09 AM, Robin Anil <ro...@gmail.com>
>> wrote:
>> >
>> > > Ok.
>> > >
>> > >
>> > > On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <ted.dunning@gmail.com
>> > >wrote:
>> > >
>> > >> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we
>> are
>> > >> talking about.
>> > >>
>> > >> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com>
>> > wrote:
>> > >>
>> > >> > i.e
>> > >> > o.a.m.vectorizer.dictionary
>> > >> > o.a.m.vectorizer.hashencoder
>> > >> >
>> > >> >
>> > >> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <
>> ted.dunning@gmail.com>
>> > >> > wrote:
>> > >> >
>> > >> > > Same question applies to the FeatureValueEncoder family currently
>> > >> under
>> > >> > > classifier.vectors.*
>> > >> > >
>> > >> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <
>> robin.anil@gmail.com>
>> > >> > wrote:
>> > >> > >
>> > >> > > > How do you feel like moving the DictionaryVectorizer and Colloc
>> > >> > generator
>> > >> > > > to
>> > >> > > > the Core under vectorizers package instead of keeping them
>> under
>> > >> utils.
>> > >> > > > FeatureEncoders will also be moved under vectorizers. I want to
>> > add
>> > >> a
>> > >> > > > Wrapper which takes a Vectorizer and converts input data to
>> > vectors.
>> > >> > Its
>> > >> > > > the
>> > >> > > > missing piece of the Classifier puzzle
>> > >> > > >
>> > >> > > > o.a.m.vectorizer.dictionary
>> > >> > > > o.a.m.vectorizer.hashed or something funkier?
>> > >> > > >
>> > >> > > > What do you think about this?
>> > >> > > >
>> > >> > > > Robin
>> > >> > > >
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> >
>>
>
>

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
I checked the patch. It has nothing to do with utils/ folder and the files I
moved to core. So MAHOUT-521 is in the clear

On Sun, Oct 3, 2010 at 4:21 AM, Ted Dunning <te...@gmail.com> wrote:

> Robin, (and everybody)
>
> Watch out for pending wide-scale changes from MAHOUT-480.
>
> Anybody who would like to avoid those changes hanging fire can do a quick
> (right) review of the pending patch.  Only 93
> files are changed so it should be a breeze!
>
> Eugen did a ton of work to get all those changes done in the first place.
>  I
> found a few discrepancies and would
> like somebody to check my checking.  I will commit the changes as soon as I
> am sure we got these right.
>
> On Sat, Oct 2, 2010 at 3:35 PM, Robin Anil <ro...@gmail.com> wrote:
>
> > I just uploaded a patch for moving DictionaryVectorizer. Take a look
> >
> > On Sun, Oct 3, 2010 at 1:09 AM, Robin Anil <ro...@gmail.com> wrote:
> >
> > > Ok.
> > >
> > >
> > > On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <ted.dunning@gmail.com
> > >wrote:
> > >
> > >> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we
> are
> > >> talking about.
> > >>
> > >> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com>
> > wrote:
> > >>
> > >> > i.e
> > >> > o.a.m.vectorizer.dictionary
> > >> > o.a.m.vectorizer.hashencoder
> > >> >
> > >> >
> > >> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <ted.dunning@gmail.com
> >
> > >> > wrote:
> > >> >
> > >> > > Same question applies to the FeatureValueEncoder family currently
> > >> under
> > >> > > classifier.vectors.*
> > >> > >
> > >> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <robin.anil@gmail.com
> >
> > >> > wrote:
> > >> > >
> > >> > > > How do you feel like moving the DictionaryVectorizer and Colloc
> > >> > generator
> > >> > > > to
> > >> > > > the Core under vectorizers package instead of keeping them under
> > >> utils.
> > >> > > > FeatureEncoders will also be moved under vectorizers. I want to
> > add
> > >> a
> > >> > > > Wrapper which takes a Vectorizer and converts input data to
> > vectors.
> > >> > Its
> > >> > > > the
> > >> > > > missing piece of the Classifier puzzle
> > >> > > >
> > >> > > > o.a.m.vectorizer.dictionary
> > >> > > > o.a.m.vectorizer.hashed or something funkier?
> > >> > > >
> > >> > > > What do you think about this?
> > >> > > >
> > >> > > > Robin
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
Robin, (and everybody)

Watch out for pending wide-scale changes from MAHOUT-480.

Anybody who would like to avoid those changes hanging fire can do a quick
(right) review of the pending patch.  Only 93
files are changed so it should be a breeze!

Eugen did a ton of work to get all those changes done in the first place.  I
found a few discrepancies and would
like somebody to check my checking.  I will commit the changes as soon as I
am sure we got these right.

On Sat, Oct 2, 2010 at 3:35 PM, Robin Anil <ro...@gmail.com> wrote:

> I just uploaded a patch for moving DictionaryVectorizer. Take a look
>
> On Sun, Oct 3, 2010 at 1:09 AM, Robin Anil <ro...@gmail.com> wrote:
>
> > Ok.
> >
> >
> > On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <ted.dunning@gmail.com
> >wrote:
> >
> >> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we are
> >> talking about.
> >>
> >> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com>
> wrote:
> >>
> >> > i.e
> >> > o.a.m.vectorizer.dictionary
> >> > o.a.m.vectorizer.hashencoder
> >> >
> >> >
> >> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com>
> >> > wrote:
> >> >
> >> > > Same question applies to the FeatureValueEncoder family currently
> >> under
> >> > > classifier.vectors.*
> >> > >
> >> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com>
> >> > wrote:
> >> > >
> >> > > > How do you feel like moving the DictionaryVectorizer and Colloc
> >> > generator
> >> > > > to
> >> > > > the Core under vectorizers package instead of keeping them under
> >> utils.
> >> > > > FeatureEncoders will also be moved under vectorizers. I want to
> add
> >> a
> >> > > > Wrapper which takes a Vectorizer and converts input data to
> vectors.
> >> > Its
> >> > > > the
> >> > > > missing piece of the Classifier puzzle
> >> > > >
> >> > > > o.a.m.vectorizer.dictionary
> >> > > > o.a.m.vectorizer.hashed or something funkier?
> >> > > >
> >> > > > What do you think about this?
> >> > > >
> >> > > > Robin
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
I just uploaded a patch for moving DictionaryVectorizer. Take a look

On Sun, Oct 3, 2010 at 1:09 AM, Robin Anil <ro...@gmail.com> wrote:

> Ok.
>
>
> On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <te...@gmail.com>wrote:
>
>> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we are
>> talking about.
>>
>> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com> wrote:
>>
>> > i.e
>> > o.a.m.vectorizer.dictionary
>> > o.a.m.vectorizer.hashencoder
>> >
>> >
>> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com>
>> > wrote:
>> >
>> > > Same question applies to the FeatureValueEncoder family currently
>> under
>> > > classifier.vectors.*
>> > >
>> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com>
>> > wrote:
>> > >
>> > > > How do you feel like moving the DictionaryVectorizer and Colloc
>> > generator
>> > > > to
>> > > > the Core under vectorizers package instead of keeping them under
>> utils.
>> > > > FeatureEncoders will also be moved under vectorizers. I want to add
>> a
>> > > > Wrapper which takes a Vectorizer and converts input data to vectors.
>> > Its
>> > > > the
>> > > > missing piece of the Classifier puzzle
>> > > >
>> > > > o.a.m.vectorizer.dictionary
>> > > > o.a.m.vectorizer.hashed or something funkier?
>> > > >
>> > > > What do you think about this?
>> > > >
>> > > > Robin
>> > > >
>> > >
>> >
>>
>
>

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
Ok.


On Sun, Oct 3, 2010 at 12:58 AM, Ted Dunning <te...@gmail.com> wrote:

> Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we are
> talking about.
>
> On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com> wrote:
>
> > i.e
> > o.a.m.vectorizer.dictionary
> > o.a.m.vectorizer.hashencoder
> >
> >
> > On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com>
> > wrote:
> >
> > > Same question applies to the FeatureValueEncoder family currently under
> > > classifier.vectors.*
> > >
> > > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com>
> > wrote:
> > >
> > > > How do you feel like moving the DictionaryVectorizer and Colloc
> > generator
> > > > to
> > > > the Core under vectorizers package instead of keeping them under
> utils.
> > > > FeatureEncoders will also be moved under vectorizers. I want to add a
> > > > Wrapper which takes a Vectorizer and converts input data to vectors.
> > Its
> > > > the
> > > > missing piece of the Classifier puzzle
> > > >
> > > > o.a.m.vectorizer.dictionary
> > > > o.a.m.vectorizer.hashed or something funkier?
> > > >
> > > > What do you think about this?
> > > >
> > > > Robin
> > > >
> > >
> >
>

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
Or just flat in o.a.m.vectorizer.  There aren't a lot of classes we are
talking about.

On Sat, Oct 2, 2010 at 12:26 PM, Robin Anil <ro...@gmail.com> wrote:

> i.e
> o.a.m.vectorizer.dictionary
> o.a.m.vectorizer.hashencoder
>
>
> On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com>
> wrote:
>
> > Same question applies to the FeatureValueEncoder family currently under
> > classifier.vectors.*
> >
> > On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com>
> wrote:
> >
> > > How do you feel like moving the DictionaryVectorizer and Colloc
> generator
> > > to
> > > the Core under vectorizers package instead of keeping them under utils.
> > > FeatureEncoders will also be moved under vectorizers. I want to add a
> > > Wrapper which takes a Vectorizer and converts input data to vectors.
> Its
> > > the
> > > missing piece of the Classifier puzzle
> > >
> > > o.a.m.vectorizer.dictionary
> > > o.a.m.vectorizer.hashed or something funkier?
> > >
> > > What do you think about this?
> > >
> > > Robin
> > >
> >
>

Re: Move DictionaryVectorizer to Core

Posted by Robin Anil <ro...@gmail.com>.
i.e
o.a.m.vectorizer.dictionary
o.a.m.vectorizer.hashencoder


On Sun, Oct 3, 2010 at 12:49 AM, Ted Dunning <te...@gmail.com> wrote:

> Same question applies to the FeatureValueEncoder family currently under
> classifier.vectors.*
>
> On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com> wrote:
>
> > How do you feel like moving the DictionaryVectorizer and Colloc generator
> > to
> > the Core under vectorizers package instead of keeping them under utils.
> > FeatureEncoders will also be moved under vectorizers. I want to add a
> > Wrapper which takes a Vectorizer and converts input data to vectors. Its
> > the
> > missing piece of the Classifier puzzle
> >
> > o.a.m.vectorizer.dictionary
> > o.a.m.vectorizer.hashed or something funkier?
> >
> > What do you think about this?
> >
> > Robin
> >
>

Re: Move DictionaryVectorizer to Core

Posted by Ted Dunning <te...@gmail.com>.
Same question applies to the FeatureValueEncoder family currently under
classifier.vectors.*

On Sat, Oct 2, 2010 at 11:23 AM, Robin Anil <ro...@gmail.com> wrote:

> How do you feel like moving the DictionaryVectorizer and Colloc generator
> to
> the Core under vectorizers package instead of keeping them under utils.
> FeatureEncoders will also be moved under vectorizers. I want to add a
> Wrapper which takes a Vectorizer and converts input data to vectors. Its
> the
> missing piece of the Classifier puzzle
>
> o.a.m.vectorizer.dictionary
> o.a.m.vectorizer.hashed or something funkier?
>
> What do you think about this?
>
> Robin
>