You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@opennlp.apache.org by "Manoj B. Narayanan" <ma...@gmail.com> on 2017/12/21 14:01:39 UTC

Feature Addition in POS Tagger

Hi,

Is it possible to add custom features for POS Tagger? If so please let me
know how to do it.

-- 
Regards,
Manoj.

Re: Feature Addition in POS Tagger

Posted by "Manoj B. Narayanan" <ma...@gmail.com>.

Thanks Dan.

On Thu, Dec 21, 2017 at 7:51 PM, Dan Russ <da...@gmail.com> wrote:

> Hi Manoj,
>
> Yes,   In that documentation under the NameFinder training api (
> https://opennlp.apache.org/docs/1.8.3/manual/opennlp.
> html#tools.namefind.training.featuregen <https://opennlp.apache.org/
> docs/1.8.3/manual/opennlp.html#tools.namefind.training.featuregen>  ), it
> explains how to create custom features.  If you are using the command line
> took, an important thing, WHICH IS NOT MENTIONED IN THE DOCUMENTATION,  is
> that you must create a jar file with your custom feature generator and put
> it in the DIR/apache-opennlp-x.y.x/lib directory so that it is in the class
> path.
>
> Here is an example xml file to define the features ( I use the NameFinder,
> but the POSTagger should be similar)
>
> <generators>
>   <cache>
>     <generators>
>       <window prevLength = "2" nextLength = "2">
>         <token/>
>       </window>
>       <window prevLength = "2" nextLength = "2">
>         <custom class = "sequence.featuregenerators.JobSuffixFeatureGenerator"
> />
>       </window>
>       <custom class = "sequence.featuregenerators.KnownTaskFeatureGenerator"
> dict = "/trainingdata/taskfinder/knowntasks.dict" />
>     </generators>
>   </cache>
>
>
> opennlp TokenNameFinderTrainer.brat -featuregen fg.xml -lang en -model
> taskfinder.bin -tokenizerModel /home/me/apache-opennlp-models/en-token.bin
> -annotationConfig annotation.conf -bratDataDir /home/me/trainingdata/
> taskfinder/train
>
>
> Best of luck,
> Daniel
>
>
> > On Dec 21, 2017, at 9:01 AM, Manoj B. Narayanan <
> manojb.narayanan2011@gmail.com> wrote:
> >
> > Hi,
> >
> > Is it possible to add custom features for POS Tagger? If so please let me
> > know how to do it.
> >
> > --
> > Regards,
> > Manoj.
>
>


-- 
Regards,
Manoj.

Re: Feature Addition in POS Tagger

Posted by Dan Russ <da...@gmail.com>.

Hi Manoj,

Yes,   In that documentation under the NameFinder training api ( https://opennlp.apache.org/docs/1.8.3/manual/opennlp.html#tools.namefind.training.featuregen <https://opennlp.apache.org/docs/1.8.3/manual/opennlp.html#tools.namefind.training.featuregen>  ), it explains how to create custom features.  If you are using the command line took, an important thing, WHICH IS NOT MENTIONED IN THE DOCUMENTATION,  is that you must create a jar file with your custom feature generator and put it in the DIR/apache-opennlp-x.y.x/lib directory so that it is in the class path. 

Here is an example xml file to define the features ( I use the NameFinder, but the POSTagger should be similar)

<generators>
  <cache>
    <generators>
      <window prevLength = "2" nextLength = "2">
        <token/>
      </window>
      <window prevLength = "2" nextLength = "2">
        <custom class = "sequence.featuregenerators.JobSuffixFeatureGenerator" />
      </window>
      <custom class = "sequence.featuregenerators.KnownTaskFeatureGenerator" dict = "/trainingdata/taskfinder/knowntasks.dict" />
    </generators>
  </cache>

opennlp TokenNameFinderTrainer.brat -featuregen fg.xml -lang en -model taskfinder.bin -tokenizerModel /home/me/apache-opennlp-models/en-token.bin -annotationConfig annotation.conf -bratDataDir /home/me/trainingdata/taskfinder/train 

Best of luck,
Daniel

> On Dec 21, 2017, at 9:01 AM, Manoj B. Narayanan <ma...@gmail.com> wrote:
> 
> Hi,
> 
> Is it possible to add custom features for POS Tagger? If so please let me
> know how to do it.
> 
> -- 
> Regards,
> Manoj.