You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@opennlp.apache.org by György Chityil <gy...@gmail.com> on 2011/09/15 14:25:07 UTC
Is there anyway to speed up the loading of the POSTagger model?
Hello,
On my comp it takes on average 2-3 secs to load the large POSTaggger model
(en, circa 70MB)
Here is a piece of the output:
Loading POS Tagger model ... done (2.814s)
Is there anyway to speed this up?
Re: Is there anyway to speed up the loading of the POSTagger model?
Posted by György Chityil <gy...@gmail.com>.
Sorry, I realized you were referring to the API found here
http://incubator.apache.org/opennlp/documentation/manual/opennlp.html#opennlp
Will be checking it.
On Thu, Sep 15, 2011 at 2:48 PM, György Chityil <gy...@gmail.com>wrote:
> Thanks Jörn, this sounds interesting "you only need to do it once at start
> up. And then the model can be shared between all POS Tagger instances." Is
> there some kind of documentation on how to run multiple POS Tagger
> instances? So far it seemed to me (on linux) that I have to start a new
> instance for every tagging, meaning I have to execute the command "opennlp
> POSTagger en-maxent-pos.bin < myfile.txt > result.txt"
>
> Or perhaps what I just thought of it there is a way to load opennlp (and
> the tagger) with nohup command on linux so it stays active in the background
> waiting for requests.
>
>
> On Thu, Sep 15, 2011 at 2:42 PM, Jörn Kottmann <ko...@gmail.com> wrote:
>
>> On 9/15/11 2:25 PM, György Chityil wrote:
>>
>>> Hello,
>>>
>>> On my comp it takes on average 2-3 secs to load the large POSTaggger
>>> model
>>> (en, circa 70MB)
>>>
>>> Here is a piece of the output:
>>> Loading POS Tagger model ... done (2.814s)
>>>
>>>
>>> Is there anyway to speed this up?
>>>
>>
>> No not really. We would need to optimize the code which is loading the
>> model.
>> You are invited to submit a patch which does that, maybe there are a few
>> easy ways
>> to make it faster, not sure.
>>
>> Which loading time would you like to have?
>>
>> In all the applications on which I worked the loading time didn't matter
>> because
>> you only need to do it once at start up. And then the model can be shared
>> between
>> all POS Tagger instances.
>>
>> Jörn
>>
>>
>
>
> --
> Gyuri
> 274 44 98
> 06 30 5888 744
>
>
--
Gyuri
274 44 98
06 30 5888 744
Re: Is there anyway to speed up the loading of the POSTagger model?
Posted by Jörn Kottmann <ko...@gmail.com>.
On 9/15/11 2:48 PM, György Chityil wrote:
> Thanks Jörn, this sounds interesting "you only need to do it once at start
> up. And then the model can be shared between all POS Tagger instances." Is
> there some kind of documentation on how to run multiple POS Tagger
> instances? So far it seemed to me (on linux) that I have to start a new
> instance for every tagging, meaning I have to execute the command "opennlp
> POSTagger en-maxent-pos.bin< myfile.txt> result.txt"
>
> Or perhaps what I just thought of it there is a way to load opennlp (and the
> tagger) with nohup command on linux so it stays active in the background
> waiting for requests.
The command line tools to process data are only there for testing and
demonstration.
If you need to do some serious processing you should either use our API
or use OpenNLP
through some kind of framework, e.g. UIMA.
Jörn
Re: Is there anyway to speed up the loading of the POSTagger model?
Posted by György Chityil <gy...@gmail.com>.
Thanks Jörn, this sounds interesting "you only need to do it once at start
up. And then the model can be shared between all POS Tagger instances." Is
there some kind of documentation on how to run multiple POS Tagger
instances? So far it seemed to me (on linux) that I have to start a new
instance for every tagging, meaning I have to execute the command "opennlp
POSTagger en-maxent-pos.bin < myfile.txt > result.txt"
Or perhaps what I just thought of it there is a way to load opennlp (and the
tagger) with nohup command on linux so it stays active in the background
waiting for requests.
On Thu, Sep 15, 2011 at 2:42 PM, Jörn Kottmann <ko...@gmail.com> wrote:
> On 9/15/11 2:25 PM, György Chityil wrote:
>
>> Hello,
>>
>> On my comp it takes on average 2-3 secs to load the large POSTaggger model
>> (en, circa 70MB)
>>
>> Here is a piece of the output:
>> Loading POS Tagger model ... done (2.814s)
>>
>>
>> Is there anyway to speed this up?
>>
>
> No not really. We would need to optimize the code which is loading the
> model.
> You are invited to submit a patch which does that, maybe there are a few
> easy ways
> to make it faster, not sure.
>
> Which loading time would you like to have?
>
> In all the applications on which I worked the loading time didn't matter
> because
> you only need to do it once at start up. And then the model can be shared
> between
> all POS Tagger instances.
>
> Jörn
>
>
--
Gyuri
274 44 98
06 30 5888 744
Re: Is there anyway to speed up the loading of the POSTagger model?
Posted by Jörn Kottmann <ko...@gmail.com>.
On 9/15/11 2:25 PM, György Chityil wrote:
> Hello,
>
> On my comp it takes on average 2-3 secs to load the large POSTaggger model
> (en, circa 70MB)
>
> Here is a piece of the output:
> Loading POS Tagger model ... done (2.814s)
>
>
> Is there anyway to speed this up?
No not really. We would need to optimize the code which is loading the
model.
You are invited to submit a patch which does that, maybe there are a few
easy ways
to make it faster, not sure.
Which loading time would you like to have?
In all the applications on which I worked the loading time didn't matter
because
you only need to do it once at start up. And then the model can be
shared between
all POS Tagger instances.
Jörn