You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@opennlp.apache.org by Jörn Kottmann <ko...@gmail.com> on 2014/01/20 10:48:09 UTC
Support for sequence models
Hi all,
in OpenNLP we have a couple of components which rely on sequence tagging.
Right now they are using a normal classifier and search for a good
sequence via beam search.
I would like to propose that we change that a bit, all components which
are based on sequence
tagging should use a Sequence Classification Model instead of directly
using an
Event Classification Model (currently named MaxentModel).
The change will have two advantages, It will be possible to integrate ml
algorithm which operate on a sequence
level (e.g. CRF) and it would be easy to exchange beam search against a
similar (maybe enhanced) algorithm.
On the training side we already have support for training on sequences.
Anyway the current implementation is a bit
unlucky because the sequence training class can only return an Event
Classification Model. I will change that so that
a Sequence Classification Model has to be returned, and the Perceptron
Sequence Model will be returned as a
Sequence Classification Model instead.
Any thoughts?
Jörn
Re: Support for sequence models
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
I don't have any particular use case at hand, nope.
Cheers,
-- Richard
On 29.01.2014, at 20:33, Jörn Kottmann <ko...@gmail.com> wrote:
> On 01/29/2014 07:33 PM, Richard Eckart de Castilho wrote:
>> If I understand the SequenceClassificationModel interface correctly,
>> the input data to be classified is passed as an array T[].
>>
>> What about data that is very large? I think it would be nice if
>> the new interface would support sequence classifications on streams,
>> e.g. by passing an Iterator<T> or an actual stream to the classifier.
>
> Exactly, the sequence to be classified is passed in as an array.
>
> The current interface will support passing in quite long sequences (only limited by memory)
> of probably easily a few ten thousand elements.
>
> Do you have a use case where this would not be good enough? In the OpenNLP components the
> sequences are usually a sentence, but even a really long document should work.
>
> Supporting streams would add quite some complexity, and I wonder if it is really necessary,
> e.g. looking back or forward in the sequence during feature generation.
>
> Thanks,
> Jörn
Re: Support for sequence models
Posted by Jörn Kottmann <ko...@gmail.com>.
On 01/29/2014 07:33 PM, Richard Eckart de Castilho wrote:
> If I understand the SequenceClassificationModel interface correctly,
> the input data to be classified is passed as an array T[].
>
> What about data that is very large? I think it would be nice if
> the new interface would support sequence classifications on streams,
> e.g. by passing an Iterator<T> or an actual stream to the classifier.
Exactly, the sequence to be classified is passed in as an array.
The current interface will support passing in quite long sequences (only
limited by memory)
of probably easily a few ten thousand elements.
Do you have a use case where this would not be good enough? In the
OpenNLP components the
sequences are usually a sentence, but even a really long document should
work.
Supporting streams would add quite some complexity, and I wonder if it
is really necessary,
e.g. looking back or forward in the sequence during feature generation.
Thanks,
Jörn
Re: Support for sequence models
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
If I understand the SequenceClassificationModel interface correctly,
the input data to be classified is passed as an array T[].
What about data that is very large? I think it would be nice if
the new interface would support sequence classifications on streams,
e.g. by passing an Iterator<T> or an actual stream to the classifier.
Just my 2 cents.
Cheers,
-- Richard
On 29.01.2014, at 19:22, Jörn Kottmann <ko...@gmail.com> wrote:
> On 01/20/2014 10:48 AM, Jörn Kottmann wrote:
>>
>> On the training side we already have support for training on sequences. Anyway the current implementation is a bit
>> unlucky because the sequence training class can only return an Event Classification Model. I will change that so that
>> a Sequence Classification Model has to be returned, and the Perceptron Sequence Model will be returned as a
>> Sequence Classification Model instead.
>
> After looking further into things it seems good to support all three cases:
> - A Classification Model trained on non-sequential events
> - A Classification Model trained on sequential events
> - A Sequence Classification Model trained on sequential events
>
> Jörn
Re: Support for sequence models
Posted by Jörn Kottmann <ko...@gmail.com>.
On 01/20/2014 10:48 AM, Jörn Kottmann wrote:
>
> On the training side we already have support for training on
> sequences. Anyway the current implementation is a bit
> unlucky because the sequence training class can only return an Event
> Classification Model. I will change that so that
> a Sequence Classification Model has to be returned, and the Perceptron
> Sequence Model will be returned as a
> Sequence Classification Model instead.
After looking further into things it seems good to support all three cases:
- A Classification Model trained on non-sequential events
- A Classification Model trained on sequential events
- A Sequence Classification Model trained on sequential events
Jörn
Re: Support for sequence models
Posted by Jörn Kottmann <ko...@gmail.com>.
On 01/20/2014 03:17 PM, Jörn Kottmann wrote:
> On 01/20/2014 02:55 PM, Mark G wrote:
>> Sounds like a good change, but I have two questions: will this affect
>> the
>> current APIs? Will people's maxent models still work if they are using a
>> Maxent model now in a component that will soon require a Seq tagging
>> model
>> after the change?
>
> Yes, the current Maxent Model impls will then be turned into a
> Sequence Classification Model by
> applying beam search to it.
>
> All existing models will continue to work.
>
> Anyway, classes which extend NameFinderME will very likely break
> (depending on what is modified), but they were
> broken through the maxent refactoring anyway already.
I just committed the interface Sequence Classification Model.
Any comments?
Jörn
Re: Support for sequence models
Posted by Jörn Kottmann <ko...@gmail.com>.
On 01/20/2014 02:55 PM, Mark G wrote:
> Sounds like a good change, but I have two questions: will this affect the
> current APIs? Will people's maxent models still work if they are using a
> Maxent model now in a component that will soon require a Seq tagging model
> after the change?
Yes, the current Maxent Model impls will then be turned into a Sequence
Classification Model by
applying beam search to it.
All existing models will continue to work.
Anyway, classes which extend NameFinderME will very likely break
(depending on what is modified), but they were
broken through the maxent refactoring anyway already.
Jörn
Re: Support for sequence models
Posted by Mark G <gi...@gmail.com>.
Sounds like a good change, but I have two questions: will this affect the
current APIs? Will people's maxent models still work if they are using a
Maxent model now in a component that will soon require a Seq tagging model
after the change?
On Mon, Jan 20, 2014 at 8:44 AM, Richard Eckart de Castilho <
richard.eckart@gmail.com> wrote:
> Would it still be possible to use the current 1.5.x models with OpenNLP
> after the change?
>
> -- Richard
>
> On 20.01.2014, at 07:48, Jörn Kottmann <ko...@gmail.com> wrote:
>
> > Hi all,
> >
> > in OpenNLP we have a couple of components which rely on sequence tagging.
> > Right now they are using a normal classifier and search for a good
> sequence via beam search.
> >
> > I would like to propose that we change that a bit, all components which
> are based on sequence
> > tagging should use a Sequence Classification Model instead of directly
> using an
> > Event Classification Model (currently named MaxentModel).
> >
> > The change will have two advantages, It will be possible to integrate ml
> algorithm which operate on a sequence
> > level (e.g. CRF) and it would be easy to exchange beam search against a
> similar (maybe enhanced) algorithm.
> >
> > On the training side we already have support for training on sequences.
> Anyway the current implementation is a bit
> > unlucky because the sequence training class can only return an Event
> Classification Model. I will change that so that
> > a Sequence Classification Model has to be returned, and the Perceptron
> Sequence Model will be returned as a
> > Sequence Classification Model instead.
> >
> > Any thoughts?
> >
> > Jörn
>
Re: Support for sequence models
Posted by Jörn Kottmann <ko...@gmail.com>.
On 01/20/2014 02:44 PM, Richard Eckart de Castilho wrote:
> Would it still be possible to use the current 1.5.x models with OpenNLP after the change?
Yes, they will continue to work as they do now. In 1.6.0 only support
for 1.4.x and earlier models will be removed.
Jörn
Re: Support for sequence models
Posted by Richard Eckart de Castilho <ri...@gmail.com>.
Would it still be possible to use the current 1.5.x models with OpenNLP after the change?
-- Richard
On 20.01.2014, at 07:48, Jörn Kottmann <ko...@gmail.com> wrote:
> Hi all,
>
> in OpenNLP we have a couple of components which rely on sequence tagging.
> Right now they are using a normal classifier and search for a good sequence via beam search.
>
> I would like to propose that we change that a bit, all components which are based on sequence
> tagging should use a Sequence Classification Model instead of directly using an
> Event Classification Model (currently named MaxentModel).
>
> The change will have two advantages, It will be possible to integrate ml algorithm which operate on a sequence
> level (e.g. CRF) and it would be easy to exchange beam search against a similar (maybe enhanced) algorithm.
>
> On the training side we already have support for training on sequences. Anyway the current implementation is a bit
> unlucky because the sequence training class can only return an Event Classification Model. I will change that so that
> a Sequence Classification Model has to be returned, and the Perceptron Sequence Model will be returned as a
> Sequence Classification Model instead.
>
> Any thoughts?
>
> Jörn