You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mahout.apache.org by Patrick Collins <pa...@ready2sign.com> on 2011/06/28 02:51:13 UTC
Fuzzy logic and Heuristics vs Classification
Has anyone got any advice on how to combine heuristics and classification?
When preparing my data to build out the features to feed into my
classification model I keep noticing patterns of text which I know with
99.99% probability implies a certain outcome.
How would you construct the data/features in order to pre-classify this
data to provide much more likelihood that the classifier comes to the
"correct" conclusion?
For example, I remember seeing an anti-spam machine which used a
combination of fuzzy logic and then classification to build a better
outcome (but he did not detail out how it was actually implemented). He
used a whole range of heuristics to determine that a certain sender is
known to be a spammer rather than just blindly passing this data in to
the classifier.
In my dataset I have a LOT of patterns like this that I can identify and
then determine with very high probability the outcome. I say high
probability, but I cannot say absolutely. Ideally if I could pre compute
a lot of this data using heuristics I could feed this information in to
the classifier to greatly reduce the number of features. But the
classifiers do not allow me the ability to provide a "weight" to a
certain feature.
Other than "well just try and see what works", I was wondering how do
people deal with this problem? Do they just leave it to the classifier
and hope that the classifier picks up the same patterns?
I'm a bit new to mahout and classification algorithms and so am just
trying to get some input from how others might see this problem and
whether I'm barking up the wrong tree.
Patrick.
Re: Fuzzy logic and Heuristics vs Classification
Posted by Patrick Collins <pa...@ready2sign.com>.
Thanks Ted and Hector. Makes sense.
On 6/27/11 9:55 PM, Ted Dunning wrote:
> Yeah... what Hector says.
>
> You can even make the output of preliminary classifiers be features for new
> classifiers.
>
> Or if you have two different target variables, you can make a model that
> predicts one target be a feature in the model that predicts the other.
>
> Feature extraction generally has more potential for performance improvements
> than any algorithm changes.
>
> On Mon, Jun 27, 2011 at 7:22 PM, Hector Yee<he...@gmail.com> wrote:
>
>> Redacted to pass the overly aggressive spam filter.
>>
>> On Mon, Jun 27, 2011 at 7:19 PM, Hector Yee<he...@gmail.com> wrote:
>>
>>> Just make the pattern a feature and feed it into the machine learning.
>>>
>>> e.g. if its a spam model and you notice v**gra is a spam term just make
>>> feature 0 = "v**gra count" and the rest your regular bag of words.
>>>
>>> The only thing you have to be careful of is the relative weights between
>>> each feature category. Typical normalizations is to L2 norm each feature
>>> category separately before concatenation.
>>> Another option is to use a "scale free" classification algorithm like
>>> adaboost.
>>>
>>>
>>> On Mon, Jun 27, 2011 at 5:51 PM, Patrick Collins<
>>> patrick.collins@ready2sign.com> wrote:
>>>
>>>> Has anyone got any advice on how to combine heuristics and
>> classification?
>>>> When preparing my data to build out the features to feed into my
>>>> classification model I keep noticing patterns of text which I know with
>>>> 99.99% probability implies a certain outcome.
>>>>
>>>> How would you construct the data/features in order to pre-classify this
>>>> data to provide much more likelihood that the classifier comes to the
>>>> "correct" conclusion?
>>>>
>>>> For example, I remember seeing an anti-spam machine which used a
>>>> combination of fuzzy logic and then classification to build a better
>> outcome
>>>> (but he did not detail out how it was actually implemented). He used a
>> whole
>>>> range of heuristics to determine that a certain sender is known to be a
>>>> spammer rather than just blindly passing this data in to the classifier.
>>>>
>>>> In my dataset I have a LOT of patterns like this that I can identify and
>>>> then determine with very high probability the outcome. I say high
>>>> probability, but I cannot say absolutely. Ideally if I could pre compute
>> a
>>>> lot of this data using heuristics I could feed this information in to
>> the
>>>> classifier to greatly reduce the number of features. But the classifiers
>> do
>>>> not allow me the ability to provide a "weight" to a certain feature.
>>>>
>>>> Other than "well just try and see what works", I was wondering how do
>>>> people deal with this problem? Do they just leave it to the classifier
>> and
>>>> hope that the classifier picks up the same patterns?
>>>>
>>>> I'm a bit new to mahout and classification algorithms and so am just
>>>> trying to get some input from how others might see this problem and
>> whether
>>>> I'm barking up the wrong tree.
>>>>
>>>> Patrick.
>>>>
>>>
>>>
>>> --
>>> Yee Yang Li Hector
>>> http://hectorgon.blogspot.com/ (tech + travel)
>>> http://hectorgon.com (book reviews)
>>>
>>>
>>
>> --
>> Yee Yang Li Hector
>> http://hectorgon.blogspot.com/ (tech + travel)
>> http://hectorgon.com (book reviews)
>>
Re: Fuzzy logic and Heuristics vs Classification
Posted by Ted Dunning <te...@gmail.com>.
Yeah... what Hector says.
You can even make the output of preliminary classifiers be features for new
classifiers.
Or if you have two different target variables, you can make a model that
predicts one target be a feature in the model that predicts the other.
Feature extraction generally has more potential for performance improvements
than any algorithm changes.
On Mon, Jun 27, 2011 at 7:22 PM, Hector Yee <he...@gmail.com> wrote:
> Redacted to pass the overly aggressive spam filter.
>
> On Mon, Jun 27, 2011 at 7:19 PM, Hector Yee <he...@gmail.com> wrote:
>
> > Just make the pattern a feature and feed it into the machine learning.
> >
> > e.g. if its a spam model and you notice v**gra is a spam term just make
> > feature 0 = "v**gra count" and the rest your regular bag of words.
> >
> > The only thing you have to be careful of is the relative weights between
> > each feature category. Typical normalizations is to L2 norm each feature
> > category separately before concatenation.
> > Another option is to use a "scale free" classification algorithm like
> > adaboost.
> >
> >
> > On Mon, Jun 27, 2011 at 5:51 PM, Patrick Collins <
> > patrick.collins@ready2sign.com> wrote:
> >
> >> Has anyone got any advice on how to combine heuristics and
> classification?
> >>
> >> When preparing my data to build out the features to feed into my
> >> classification model I keep noticing patterns of text which I know with
> >> 99.99% probability implies a certain outcome.
> >>
> >> How would you construct the data/features in order to pre-classify this
> >> data to provide much more likelihood that the classifier comes to the
> >> "correct" conclusion?
> >>
> >> For example, I remember seeing an anti-spam machine which used a
> >> combination of fuzzy logic and then classification to build a better
> outcome
> >> (but he did not detail out how it was actually implemented). He used a
> whole
> >> range of heuristics to determine that a certain sender is known to be a
> >> spammer rather than just blindly passing this data in to the classifier.
> >>
> >> In my dataset I have a LOT of patterns like this that I can identify and
> >> then determine with very high probability the outcome. I say high
> >> probability, but I cannot say absolutely. Ideally if I could pre compute
> a
> >> lot of this data using heuristics I could feed this information in to
> the
> >> classifier to greatly reduce the number of features. But the classifiers
> do
> >> not allow me the ability to provide a "weight" to a certain feature.
> >>
> >> Other than "well just try and see what works", I was wondering how do
> >> people deal with this problem? Do they just leave it to the classifier
> and
> >> hope that the classifier picks up the same patterns?
> >>
> >> I'm a bit new to mahout and classification algorithms and so am just
> >> trying to get some input from how others might see this problem and
> whether
> >> I'm barking up the wrong tree.
> >>
> >> Patrick.
> >>
> >
> >
> >
> > --
> > Yee Yang Li Hector
> > http://hectorgon.blogspot.com/ (tech + travel)
> > http://hectorgon.com (book reviews)
> >
> >
>
>
> --
> Yee Yang Li Hector
> http://hectorgon.blogspot.com/ (tech + travel)
> http://hectorgon.com (book reviews)
>
Re: Fuzzy logic and Heuristics vs Classification
Posted by Hector Yee <he...@gmail.com>.
Redacted to pass the overly aggressive spam filter.
On Mon, Jun 27, 2011 at 7:19 PM, Hector Yee <he...@gmail.com> wrote:
> Just make the pattern a feature and feed it into the machine learning.
>
> e.g. if its a spam model and you notice v**gra is a spam term just make
> feature 0 = "v**gra count" and the rest your regular bag of words.
>
> The only thing you have to be careful of is the relative weights between
> each feature category. Typical normalizations is to L2 norm each feature
> category separately before concatenation.
> Another option is to use a "scale free" classification algorithm like
> adaboost.
>
>
> On Mon, Jun 27, 2011 at 5:51 PM, Patrick Collins <
> patrick.collins@ready2sign.com> wrote:
>
>> Has anyone got any advice on how to combine heuristics and classification?
>>
>> When preparing my data to build out the features to feed into my
>> classification model I keep noticing patterns of text which I know with
>> 99.99% probability implies a certain outcome.
>>
>> How would you construct the data/features in order to pre-classify this
>> data to provide much more likelihood that the classifier comes to the
>> "correct" conclusion?
>>
>> For example, I remember seeing an anti-spam machine which used a
>> combination of fuzzy logic and then classification to build a better outcome
>> (but he did not detail out how it was actually implemented). He used a whole
>> range of heuristics to determine that a certain sender is known to be a
>> spammer rather than just blindly passing this data in to the classifier.
>>
>> In my dataset I have a LOT of patterns like this that I can identify and
>> then determine with very high probability the outcome. I say high
>> probability, but I cannot say absolutely. Ideally if I could pre compute a
>> lot of this data using heuristics I could feed this information in to the
>> classifier to greatly reduce the number of features. But the classifiers do
>> not allow me the ability to provide a "weight" to a certain feature.
>>
>> Other than "well just try and see what works", I was wondering how do
>> people deal with this problem? Do they just leave it to the classifier and
>> hope that the classifier picks up the same patterns?
>>
>> I'm a bit new to mahout and classification algorithms and so am just
>> trying to get some input from how others might see this problem and whether
>> I'm barking up the wrong tree.
>>
>> Patrick.
>>
>
>
>
> --
> Yee Yang Li Hector
> http://hectorgon.blogspot.com/ (tech + travel)
> http://hectorgon.com (book reviews)
>
>
--
Yee Yang Li Hector
http://hectorgon.blogspot.com/ (tech + travel)
http://hectorgon.com (book reviews)