You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@opennlp.apache.org by Jörn Kottmann <ko...@gmail.com> on 2012/04/27 10:53:32 UTC
Model loading should be easier
Hi all,
currently a model can only be loaded from an InputStream.
I suggest that we extend this a bit and give all models
two additional constructors, one to load from a URL and one
to load from a File.
This makes it easier to instantiate a model. The URL constructor
can be easily used by a dependency injection framework to
create an instance.
Many users just load a model from a file for testing, having a constructor
which takes a File will ensure that the InputStream handling is done
correctly.
Any opinions?
Jörn
Re: Model loading should be easier
Posted by Aliaksandr Autayeu <al...@autayeu.com>.
Yes, I also thought along these lines, this simplifies customization a lot.
And +1 to Jörn comment below about security.
Aliaksandr
On Fri, Apr 27, 2012 at 3:03 PM, william.colen@gmail.com <
william.colen@gmail.com> wrote:
> To bundle models inside a jar file would be nice also because of the
> customization factories. While using maven we can make a model depend on
> external jar that provides the factory.
>
>
> On Fri, Apr 27, 2012 at 9:08 AM, Jörn Kottmann <ko...@gmail.com> wrote:
>
> > On 04/27/2012 01:44 PM, Aliaksandr Autayeu wrote:
> >
> >> Ah, I see, it is for the stuff you do in sandbox. Looks very reasonable
> to
> >> me. But I would test for resource loading - we use jar artifacts to move
> >> our models along with the library, but now they're manually handled.
> With
> >> this new method it'll be easier.
> >>
> >
> > Placing models in jars and them somehow distribute them seems
> > to become a new trend.
> >
> > Its actually quite amazing how many possibilities you then suddenly
> > get.
> >
> > - The jar might be hosted in local repository. Then it can be referenced
> > by URL like this: jar:http://www.foo.com/bar/**
> > baz.jar!/com/foo/models/ner/**person.bin<
> http://www.foo.com/bar/baz.jar!/com/foo/models/ner/person.bin>
> > ,
> > or http:t//www.foo.com/person.bin**.
> >
> > - The jar could contain a blueprint xml and then the models will
> > automatically
> > made available in an OSGi environment.
> >
> > - The jar can be retrieved during build time (e.g. via maven) and put
> > along with
> > some other code in a distribution.
> >
> > - Models which need dependencies to run can ship them in the jar.
> > When using maven, they can depend on jars which contain the extension.
> > Especially handy if we provide even more extensibility e.g. a MALLET
> > support.
> >
> > Jörn
> >
>
Re: Model loading should be easier
Posted by Jörn Kottmann <ko...@gmail.com>.
I opened a jira for it:
https://issues.apache.org/jira/browse/OPENNLP-505
Jörn
On 04/27/2012 03:19 PM, Jörn Kottmann wrote:
> On 04/27/2012 03:03 PM, william.colen@gmail.com wrote:
>> To bundle models inside a jar file would be nice also because of the
>> customization factories. While using maven we can make a model depend on
>> external jar that provides the factory.
>
> One of our first ideas to make this work was, just to include
> some code in our models directly, but then people might execute
> code from non-trustworthy sources on their machines without
> even noticing it.
>
> Anyway if someone downloads a jar file and puts it on the classpath
> he must be aware of the potential security issue and we cannot
> be blamed.
>
> Jörn
Re: Model loading should be easier
Posted by Jörn Kottmann <ko...@gmail.com>.
On 04/27/2012 03:03 PM, william.colen@gmail.com wrote:
> To bundle models inside a jar file would be nice also because of the
> customization factories. While using maven we can make a model depend on
> external jar that provides the factory.
One of our first ideas to make this work was, just to include
some code in our models directly, but then people might execute
code from non-trustworthy sources on their machines without
even noticing it.
Anyway if someone downloads a jar file and puts it on the classpath
he must be aware of the potential security issue and we cannot
be blamed.
Jörn
Re: Model loading should be easier
Posted by "william.colen@gmail.com" <wi...@gmail.com>.
To bundle models inside a jar file would be nice also because of the
customization factories. While using maven we can make a model depend on
external jar that provides the factory.
On Fri, Apr 27, 2012 at 9:08 AM, Jörn Kottmann <ko...@gmail.com> wrote:
> On 04/27/2012 01:44 PM, Aliaksandr Autayeu wrote:
>
>> Ah, I see, it is for the stuff you do in sandbox. Looks very reasonable to
>> me. But I would test for resource loading - we use jar artifacts to move
>> our models along with the library, but now they're manually handled. With
>> this new method it'll be easier.
>>
>
> Placing models in jars and them somehow distribute them seems
> to become a new trend.
>
> Its actually quite amazing how many possibilities you then suddenly
> get.
>
> - The jar might be hosted in local repository. Then it can be referenced
> by URL like this: jar:http://www.foo.com/bar/**
> baz.jar!/com/foo/models/ner/**person.bin<http://www.foo.com/bar/baz.jar!/com/foo/models/ner/person.bin>
> ,
> or http:t//www.foo.com/person.bin**.
>
> - The jar could contain a blueprint xml and then the models will
> automatically
> made available in an OSGi environment.
>
> - The jar can be retrieved during build time (e.g. via maven) and put
> along with
> some other code in a distribution.
>
> - Models which need dependencies to run can ship them in the jar.
> When using maven, they can depend on jars which contain the extension.
> Especially handy if we provide even more extensibility e.g. a MALLET
> support.
>
> Jörn
>
Re: Model loading should be easier
Posted by Jörn Kottmann <ko...@gmail.com>.
On 04/27/2012 01:44 PM, Aliaksandr Autayeu wrote:
> Ah, I see, it is for the stuff you do in sandbox. Looks very reasonable to
> me. But I would test for resource loading - we use jar artifacts to move
> our models along with the library, but now they're manually handled. With
> this new method it'll be easier.
Placing models in jars and them somehow distribute them seems
to become a new trend.
Its actually quite amazing how many possibilities you then suddenly
get.
- The jar might be hosted in local repository. Then it can be referenced
by URL like this:
jar:http://www.foo.com/bar/baz.jar!/com/foo/models/ner/person.bin,
or http:t//www.foo.com/person.bin.
- The jar could contain a blueprint xml and then the models will
automatically
made available in an OSGi environment.
- The jar can be retrieved during build time (e.g. via maven) and put
along with
some other code in a distribution.
- Models which need dependencies to run can ship them in the jar.
When using maven, they can depend on jars which contain the extension.
Especially handy if we provide even more extensibility e.g. a MALLET
support.
Jörn
Re: Model loading should be easier
Posted by Aliaksandr Autayeu <al...@autayeu.com>.
Ah, I see, it is for the stuff you do in sandbox. Looks very reasonable to
me. But I would test for resource loading - we use jar artifacts to move
our models along with the library, but now they're manually handled. With
this new method it'll be easier.
Aliaksandr
On Fri, Apr 27, 2012 at 12:05 PM, Jörn Kottmann <ko...@gmail.com> wrote:
> With the URL constructor the model bean can be created like this:
> <bean id="en-ner-person-model"
> class="opennlp.t.namefind.**TokenNameFinderModel">
>
> <argument value="file:///home/xyz/**models/en-ner-person.bin"/>
> </bean>
>
> Jörn
>
>
> On 04/27/2012 12:02 PM, Jörn Kottmann wrote:
>
>> On 04/27/2012 11:47 AM, Aliaksandr Autayeu wrote:
>>
>>> Seems to be a good idea. Can you shed some light on this: "URL
>>> constructor can be easily used by a dependency injection framework
>>> to create an instance."?
>>>
>>
>> Sure.
>>
>> In the Tagging Server I did this:
>> public static TokenNamerFinderModel createModel(URL modelURL) {...}
>>
>> This factory method loads a model from a URL. I needed to create it
>> because
>> our models do not have such a constructor, with a URL constructor the
>> factory
>> is not necessary.
>>
>> Now with this factory you can create a bean like this:
>> <bean id="en-ner-person-model"
>> class="org.apache.opennlp.**tagging_server.namefind.**
>> TokenNameFinderModelFactory"
>> factory-method="createModel">
>> <argument value="file:///home/xyz/**models/en-ner-person.bin"/>
>> </bean>
>>
>> The url could also be http, or as far as I know in spring even a resource
>> on the
>> classpath, e.g. something like classpath:person.bin (I didn't test that,
>> just heard its possible).
>>
>> To finally use the model it still needs to be registered as a service (at
>> least in the Tagging Server):
>> <service id="en-ner-person-model-**service" ref="en-ner-person-model"
>> interface="opennlp.tools.**namefind.TokenNameFinderModel"**>
>> <service-properties>
>> <entry key="MODEL_LANG" value="en"/>
>> <entry key="MODEL_NAME" value="person"/>
>> </service-properties>
>> </service>
>>
>> Jörn
>>
>>
>>
>
Re: Model loading should be easier
Posted by Jörn Kottmann <ko...@gmail.com>.
With the URL constructor the model bean can be created like this:
<bean id="en-ner-person-model"
class="opennlp.t.namefind.TokenNameFinderModel">
<argument value="file:///home/xyz/models/en-ner-person.bin"/>
</bean>
Jörn
On 04/27/2012 12:02 PM, Jörn Kottmann wrote:
> On 04/27/2012 11:47 AM, Aliaksandr Autayeu wrote:
>> Seems to be a good idea. Can you shed some light on this: "URL
>> constructor can be easily used by a dependency injection framework
>> to create an instance."?
>
> Sure.
>
> In the Tagging Server I did this:
> public static TokenNamerFinderModel createModel(URL modelURL) {...}
>
> This factory method loads a model from a URL. I needed to create it
> because
> our models do not have such a constructor, with a URL constructor the
> factory
> is not necessary.
>
> Now with this factory you can create a bean like this:
> <bean id="en-ner-person-model"
>
> class="org.apache.opennlp.tagging_server.namefind.TokenNameFinderModelFactory"
> factory-method="createModel">
> <argument value="file:///home/xyz/models/en-ner-person.bin"/>
> </bean>
>
> The url could also be http, or as far as I know in spring even a
> resource on the
> classpath, e.g. something like classpath:person.bin (I didn't test
> that, just heard its possible).
>
> To finally use the model it still needs to be registered as a service
> (at least in the Tagging Server):
> <service id="en-ner-person-model-service" ref="en-ner-person-model"
> interface="opennlp.tools.namefind.TokenNameFinderModel">
> <service-properties>
> <entry key="MODEL_LANG" value="en"/>
> <entry key="MODEL_NAME" value="person"/>
> </service-properties>
> </service>
>
> Jörn
>
>
Re: Model loading should be easier
Posted by Jörn Kottmann <ko...@gmail.com>.
On 04/27/2012 11:47 AM, Aliaksandr Autayeu wrote:
> Seems to be a good idea. Can you shed some light on this: "URL
> constructor can be easily used by a dependency injection framework
> to create an instance."?
Sure.
In the Tagging Server I did this:
public static TokenNamerFinderModel createModel(URL modelURL) {...}
This factory method loads a model from a URL. I needed to create it because
our models do not have such a constructor, with a URL constructor the
factory
is not necessary.
Now with this factory you can create a bean like this:
<bean id="en-ner-person-model"
class="org.apache.opennlp.tagging_server.namefind.TokenNameFinderModelFactory"
factory-method="createModel">
<argument value="file:///home/xyz/models/en-ner-person.bin"/>
</bean>
The url could also be http, or as far as I know in spring even a
resource on the
classpath, e.g. something like classpath:person.bin (I didn't test that,
just heard its possible).
To finally use the model it still needs to be registered as a service
(at least in the Tagging Server):
<service id="en-ner-person-model-service" ref="en-ner-person-model"
interface="opennlp.tools.namefind.TokenNameFinderModel">
<service-properties>
<entry key="MODEL_LANG" value="en"/>
<entry key="MODEL_NAME" value="person"/>
</service-properties>
</service>
Jörn
Re: Model loading should be easier
Posted by Aliaksandr Autayeu <al...@autayeu.com>.
Seems to be a good idea. Can you shed some light on this: "URL
constructor can be easily used by a dependency injection framework
to create an instance."?
Aliaksandr
On Fri, Apr 27, 2012 at 10:53 AM, Jörn Kottmann <ko...@gmail.com> wrote:
> Hi all,
>
> currently a model can only be loaded from an InputStream.
> I suggest that we extend this a bit and give all models
> two additional constructors, one to load from a URL and one
> to load from a File.
>
> This makes it easier to instantiate a model. The URL constructor
> can be easily used by a dependency injection framework to
> create an instance.
>
> Many users just load a model from a file for testing, having a constructor
> which takes a File will ensure that the InputStream handling is done
> correctly.
>
> Any opinions?
>
> Jörn
>