You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@opennlp.apache.org by Lance Norskog <go...@gmail.com> on 2012/07/09 05:56:27 UTC
Apache "Text Analysis" top-level project?
Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
top-level "Text Analysis" project? There are already cross-project
connections between UIMA and OpenNLP. ORP seems dormant. It also seems
a more natural place than OpenNLP for a database of tagged text.
--
Lance Norskog
goksron@gmail.com
Re: Apache "Text Analysis" top-level project?
Posted by Michael Schmitz <sc...@cs.washington.edu>.
I like OpenNLP for this reason too. There are too many things that
can be done wrong when making a framework, especially from my narrow
perspective!
Peace. Michael
On Mon, Jul 9, 2012 at 11:57 AM, John Stewart <ca...@gmail.com> wrote:
> FWIW I agree with this perspective. So many Java-based tools grow
> into configuration monsters (cough! Hadoop). I very much appreciate
> the simplicity and self-containedness of OpenNLP, which makes it
> straightforward to integrate into many different architectures, even
> non-Java. In fact, I'd argue for making this aspect of the project
> one of its defining characteristics.
>
> jds
>
> On Mon, Jul 9, 2012 at 2:18 PM, Jeyendran Balakrishnan
> <jp...@personaltube.com> wrote:
>> It is exactly the "production"-oriented aspects of UIMA that, IMHO, make it unattractive for many users.
>> Almost all commercial implementations of NLP use NLP as one of the components in their overall software stack,
>> i.e., they typically have their own custom framework, platform, and architecture that do many things other than NLP.
>> What these commercial systems need are open source libraries to do specific things, leaving their developers free to put them together according to their own requirements and tradeoffs.
>> Any NLP system that forces the developer to use one particular way (eg UIMA) of putting things together will not be attractive,
>> and might steer them away from otherwise great algorithm implementations due to the significant additional baggage that the algorithms come with.
>>
>> Today, there are so many ways of connecting components together, including workflows, platforms, configuration management, parallel batch processing (Hadoop, anyone?), parallel stream processing (Storm), etc. Almost the entire code-base of UIMA has nothing to do with NLP. There's a big reason why IBM gave away all this code to Apache, and kept their core algorithms to themselves - it was clear to them where the value is. At least to me, it is clear where the value is - algorithms, and not frameworks.
>>
>> Software developers in industry are very capable of easily putting together their own frameworks. What they need help with are core NLP algorithms that they don't have the background to do themselves. One example I would suggest (at least according my view), is the difference between Lucene and Nutch. Being a library, Lucene has pretty much taken over search engine software development. Nutch, on the other hand, tries to be a full-fledged platform for crawling, indexing and search, and has not gathered anywhere near the same usage levels.
>>
>> My vote is to please keep OpenNLP clean, smart, algorithm-centered, user-focused.
>> Keep it simple.
>> Math, stat, and algos.
>> And excel at it.
>>
>> Please don’t dumb OpenNLP down with unnecessary bloat that any decent software team can do easily, and might often prefer to implement in a different way.
>> Connectors, not merging.
>>
>> My two bits... :-)
>>
>> Cheers,
>> Jeyendran
>>
>>
>> -----Original Message-----
>> From: Jörn Kottmann [mailto:kottmann@gmail.com]
>> Sent: Monday, July 09, 2012 1:50 AM
>> To: users@opennlp.apache.org
>> Subject: Re: Apache "Text Analysis" top-level project?
>>
>> On 07/09/2012 05:56 AM, Lance Norskog wrote:
>>> Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
>>> top-level "Text Analysis" project? There are already cross-project
>>> connections between UIMA and OpenNLP. ORP seems dormant. It also seems
>>> a more natural place than OpenNLP for a database of tagged text.
>>>
>>>
>>
>> OpenNLP and UIMA align nicely in my opinion. OpenNLP just implements engines for various NLP tasks without any further support.
>> UIMA on the other side can do a lot of these additional things you need to run OpenNLP in a production system e.g. scaling the engines to many machines, providing workflow support, resource loading and management, etc.
>> So there is not really an overlap between the two.
>>
>> UIMA has some NLP related addons in their sandbox, some of them duplicate functionality which is also provided by OpenNLP e.g. pos tagging, or the dictionary annotator, but that does not seem to be that much.
>>
>> Lucene contains a lot of NLP code for stemming and word segmentation in different languages. Thats probably the biggest NLP related code base next to OpenNLP at Apache.
>>
>> Jörn
>>
>>
>>
Re: Apache "Text Analysis" top-level project?
Posted by John Stewart <ca...@gmail.com>.
FWIW I agree with this perspective. So many Java-based tools grow
into configuration monsters (cough! Hadoop). I very much appreciate
the simplicity and self-containedness of OpenNLP, which makes it
straightforward to integrate into many different architectures, even
non-Java. In fact, I'd argue for making this aspect of the project
one of its defining characteristics.
jds
On Mon, Jul 9, 2012 at 2:18 PM, Jeyendran Balakrishnan
<jp...@personaltube.com> wrote:
> It is exactly the "production"-oriented aspects of UIMA that, IMHO, make it unattractive for many users.
> Almost all commercial implementations of NLP use NLP as one of the components in their overall software stack,
> i.e., they typically have their own custom framework, platform, and architecture that do many things other than NLP.
> What these commercial systems need are open source libraries to do specific things, leaving their developers free to put them together according to their own requirements and tradeoffs.
> Any NLP system that forces the developer to use one particular way (eg UIMA) of putting things together will not be attractive,
> and might steer them away from otherwise great algorithm implementations due to the significant additional baggage that the algorithms come with.
>
> Today, there are so many ways of connecting components together, including workflows, platforms, configuration management, parallel batch processing (Hadoop, anyone?), parallel stream processing (Storm), etc. Almost the entire code-base of UIMA has nothing to do with NLP. There's a big reason why IBM gave away all this code to Apache, and kept their core algorithms to themselves - it was clear to them where the value is. At least to me, it is clear where the value is - algorithms, and not frameworks.
>
> Software developers in industry are very capable of easily putting together their own frameworks. What they need help with are core NLP algorithms that they don't have the background to do themselves. One example I would suggest (at least according my view), is the difference between Lucene and Nutch. Being a library, Lucene has pretty much taken over search engine software development. Nutch, on the other hand, tries to be a full-fledged platform for crawling, indexing and search, and has not gathered anywhere near the same usage levels.
>
> My vote is to please keep OpenNLP clean, smart, algorithm-centered, user-focused.
> Keep it simple.
> Math, stat, and algos.
> And excel at it.
>
> Please don’t dumb OpenNLP down with unnecessary bloat that any decent software team can do easily, and might often prefer to implement in a different way.
> Connectors, not merging.
>
> My two bits... :-)
>
> Cheers,
> Jeyendran
>
>
> -----Original Message-----
> From: Jörn Kottmann [mailto:kottmann@gmail.com]
> Sent: Monday, July 09, 2012 1:50 AM
> To: users@opennlp.apache.org
> Subject: Re: Apache "Text Analysis" top-level project?
>
> On 07/09/2012 05:56 AM, Lance Norskog wrote:
>> Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
>> top-level "Text Analysis" project? There are already cross-project
>> connections between UIMA and OpenNLP. ORP seems dormant. It also seems
>> a more natural place than OpenNLP for a database of tagged text.
>>
>>
>
> OpenNLP and UIMA align nicely in my opinion. OpenNLP just implements engines for various NLP tasks without any further support.
> UIMA on the other side can do a lot of these additional things you need to run OpenNLP in a production system e.g. scaling the engines to many machines, providing workflow support, resource loading and management, etc.
> So there is not really an overlap between the two.
>
> UIMA has some NLP related addons in their sandbox, some of them duplicate functionality which is also provided by OpenNLP e.g. pos tagging, or the dictionary annotator, but that does not seem to be that much.
>
> Lucene contains a lot of NLP code for stemming and word segmentation in different languages. Thats probably the biggest NLP related code base next to OpenNLP at Apache.
>
> Jörn
>
>
>
Re: Apache "Text Analysis" top-level project?
Posted by Benson Margulies <bi...@gmail.com>.
On Wednesday, July 11, 2012, Lance Norskog wrote:
> I did not articulate it well. I meant an "umbrella" project, not
> merging the code bases. Yes, merging the code bases would be a
> disaster. The "flagship" projects draw attention to the less-active
> projects in the umbrella.
It is specifically umbrellas which have been found wanting from a
governance standpoint. Code base is not an issue. It is a two-layer pmc
that has been deemed undesirable. If several PMCs wanted to merge
governance and be a single pmc with detailed oversight over all the code, I
suppose that would be fine. I don't see Lucene/Solr doing that with other
projects; they are already large and complex. I could see opennlp and
mahout joining forces, but I would not presume to predict the opinions of
all the players.
Note that Lucene was once an umbrella and restructured as a single project,
as did WS.
>
> What prompted this is that I'm used to the activity levels of Solr and
> Mahout. These are very "alive" projects which attract new algorithms
> as well as improvements to the existing code. These projects have
> momentum: above it, the project attracts new people, and below it the
> project languishes. OpenNLP seems to be a little below it. This
> suggestion is one way to push it up.
>
> As an example, I'm playing with using LSA to summarize documents and
> create tag clouds. I planned to contribute this to Lucene/Solr, and it
> did not occur to me that it might also work in OpenNLP.
>
> I'm only coming to this project now. What is the most recent new
> algorithm suite added? When? Are the contributors committers?
>
> On Tue, Jul 10, 2012 at 12:58 AM, Benson Margulies
> <bimargulies@gmail.com <javascript:;>> wrote:
> > The board is not enthusiastic about 'umbrella' projects. Cooperate,
> > co-market -- great. Merge, not so good.
> >
> > On Tuesday, July 10, 2012, Jeyendran Balakrishnan wrote:
> >
> >> Hi Julien,
> >>
> >> My hats off to you and the rest of the Nutch developer team for
> improving
> >> Nutch over the past several years, to the level where anybody with heavy
> >> duty crawling needs can just use it off the shelf.
> >> I agree with you that Lucene vs Nutch is not as clear an analogy for the
> >> library vs framework debate.
> >>
> >> In my usage, I have tended to use Nutch more as a library (in one
> >> application only the crawling part, and in another just Fetcher2 [a
> great
> >> component], hacked up a bit to remove its dependency on the rest of
> Nutch).
> >> The point I was trying to make, not very clearly, was that Nutch
> aggregates
> >> other components (Hadoop for distributed processing, Lucene/Solr for
> >> indexing and search, Tika for parsing, etc) along with its own custom
> >> crawler component code and custom data flow design, into a platform for
> >> end-to-end crawling, indexing and search, as opposed to, for example,
> being
> >> a pure-play crawler library on top of Hadoop.
> >>
> >> I look forward with interest to follow how this debate evolves regarding
> >> OpenNLP and UIMA.
> >>
> >> Cheers,
> >> Jeyendran
> >>
> >>
> >> -----Original Message-----
> >> From: Julien Nioche [mailto:lists.digitalpebble@gmail.com<javascript:;><javascript:;>]
> >> Sent: Monday, July 09, 2012 2:01 PM
> >> To: users@opennlp.apache.org <javascript:;> <javascript:;>;
> jp@personaltube.com <javascript:;><javascript:;>
> >> Subject: Re: Apache "Text Analysis" top-level project?
> >>
> >> Jeyendran,
> >>
> >> One example I would suggest (at least according my view), is the
> difference
> >> > between Lucene and Nutch. Being a library, Lucene has pretty much
> >> > taken over search engine software development. Nutch, on the other
> >> > hand, tries to be a full-fledged platform for crawling, indexing and
> >> > search, and has not gathered anywhere near the same usage levels.
> >> >
> >>
> >> That Nutch does not have the same audience as Lucene is completely
> >> understandable given that they are quite different in scope and nature.
> Not
> >> everybody needs to crawl on a large scale, but when they do they often
> use
> >> Nutch. And by the way Nutch does not do indexing and search - it
> delegates
> >> this to other tools like SOLR so it is mostly a crawler.
> >>
> >> The comparison between UIMA and OpenNLP is a better illustration of the
> >> difference between a framework and a library IMHO
> >>
> >> Julien
> >>
> >> --
> >> *
> >> *Open Source Solutions for Text Engineering
> >>
> >> http://digitalpebble.blogspot.com/
> >> http://www.digitalpebble.com
> >> http://twitter.com/digitalpebble
> >>
> >>
>
>
>
> --
> Lance Norskog
> goksron@gmail.com <javascript:;>
>
Re: Apache "Text Analysis" top-level project?
Posted by Jörn Kottmann <ko...@gmail.com>.
On 07/11/2012 05:38 AM, Lance Norskog wrote:
> As an example, I'm playing with using LSA to summarize documents and
> create tag clouds. I planned to contribute this to Lucene/Solr, and it
> did not occur to me that it might also work in OpenNLP.
>
> I'm only coming to this project now. What is the most recent new
> algorithm suite added? When? Are the contributors committers?
We added perceptron support quite a big ago.
And yes, we give out commit rights to contributors
after we saw a handful of non-trivial patches.
Jörn
Re: Apache "Text Analysis" top-level project?
Posted by Lance Norskog <go...@gmail.com>.
I did not articulate it well. I meant an "umbrella" project, not
merging the code bases. Yes, merging the code bases would be a
disaster. The "flagship" projects draw attention to the less-active
projects in the umbrella.
What prompted this is that I'm used to the activity levels of Solr and
Mahout. These are very "alive" projects which attract new algorithms
as well as improvements to the existing code. These projects have
momentum: above it, the project attracts new people, and below it the
project languishes. OpenNLP seems to be a little below it. This
suggestion is one way to push it up.
As an example, I'm playing with using LSA to summarize documents and
create tag clouds. I planned to contribute this to Lucene/Solr, and it
did not occur to me that it might also work in OpenNLP.
I'm only coming to this project now. What is the most recent new
algorithm suite added? When? Are the contributors committers?
On Tue, Jul 10, 2012 at 12:58 AM, Benson Margulies
<bi...@gmail.com> wrote:
> The board is not enthusiastic about 'umbrella' projects. Cooperate,
> co-market -- great. Merge, not so good.
>
> On Tuesday, July 10, 2012, Jeyendran Balakrishnan wrote:
>
>> Hi Julien,
>>
>> My hats off to you and the rest of the Nutch developer team for improving
>> Nutch over the past several years, to the level where anybody with heavy
>> duty crawling needs can just use it off the shelf.
>> I agree with you that Lucene vs Nutch is not as clear an analogy for the
>> library vs framework debate.
>>
>> In my usage, I have tended to use Nutch more as a library (in one
>> application only the crawling part, and in another just Fetcher2 [a great
>> component], hacked up a bit to remove its dependency on the rest of Nutch).
>> The point I was trying to make, not very clearly, was that Nutch aggregates
>> other components (Hadoop for distributed processing, Lucene/Solr for
>> indexing and search, Tika for parsing, etc) along with its own custom
>> crawler component code and custom data flow design, into a platform for
>> end-to-end crawling, indexing and search, as opposed to, for example, being
>> a pure-play crawler library on top of Hadoop.
>>
>> I look forward with interest to follow how this debate evolves regarding
>> OpenNLP and UIMA.
>>
>> Cheers,
>> Jeyendran
>>
>>
>> -----Original Message-----
>> From: Julien Nioche [mailto:lists.digitalpebble@gmail.com <javascript:;>]
>> Sent: Monday, July 09, 2012 2:01 PM
>> To: users@opennlp.apache.org <javascript:;>; jp@personaltube.com<javascript:;>
>> Subject: Re: Apache "Text Analysis" top-level project?
>>
>> Jeyendran,
>>
>> One example I would suggest (at least according my view), is the difference
>> > between Lucene and Nutch. Being a library, Lucene has pretty much
>> > taken over search engine software development. Nutch, on the other
>> > hand, tries to be a full-fledged platform for crawling, indexing and
>> > search, and has not gathered anywhere near the same usage levels.
>> >
>>
>> That Nutch does not have the same audience as Lucene is completely
>> understandable given that they are quite different in scope and nature. Not
>> everybody needs to crawl on a large scale, but when they do they often use
>> Nutch. And by the way Nutch does not do indexing and search - it delegates
>> this to other tools like SOLR so it is mostly a crawler.
>>
>> The comparison between UIMA and OpenNLP is a better illustration of the
>> difference between a framework and a library IMHO
>>
>> Julien
>>
>> --
>> *
>> *Open Source Solutions for Text Engineering
>>
>> http://digitalpebble.blogspot.com/
>> http://www.digitalpebble.com
>> http://twitter.com/digitalpebble
>>
>>
--
Lance Norskog
goksron@gmail.com
Re: Apache "Text Analysis" top-level project?
Posted by Benson Margulies <bi...@gmail.com>.
The board is not enthusiastic about 'umbrella' projects. Cooperate,
co-market -- great. Merge, not so good.
On Tuesday, July 10, 2012, Jeyendran Balakrishnan wrote:
> Hi Julien,
>
> My hats off to you and the rest of the Nutch developer team for improving
> Nutch over the past several years, to the level where anybody with heavy
> duty crawling needs can just use it off the shelf.
> I agree with you that Lucene vs Nutch is not as clear an analogy for the
> library vs framework debate.
>
> In my usage, I have tended to use Nutch more as a library (in one
> application only the crawling part, and in another just Fetcher2 [a great
> component], hacked up a bit to remove its dependency on the rest of Nutch).
> The point I was trying to make, not very clearly, was that Nutch aggregates
> other components (Hadoop for distributed processing, Lucene/Solr for
> indexing and search, Tika for parsing, etc) along with its own custom
> crawler component code and custom data flow design, into a platform for
> end-to-end crawling, indexing and search, as opposed to, for example, being
> a pure-play crawler library on top of Hadoop.
>
> I look forward with interest to follow how this debate evolves regarding
> OpenNLP and UIMA.
>
> Cheers,
> Jeyendran
>
>
> -----Original Message-----
> From: Julien Nioche [mailto:lists.digitalpebble@gmail.com <javascript:;>]
> Sent: Monday, July 09, 2012 2:01 PM
> To: users@opennlp.apache.org <javascript:;>; jp@personaltube.com<javascript:;>
> Subject: Re: Apache "Text Analysis" top-level project?
>
> Jeyendran,
>
> One example I would suggest (at least according my view), is the difference
> > between Lucene and Nutch. Being a library, Lucene has pretty much
> > taken over search engine software development. Nutch, on the other
> > hand, tries to be a full-fledged platform for crawling, indexing and
> > search, and has not gathered anywhere near the same usage levels.
> >
>
> That Nutch does not have the same audience as Lucene is completely
> understandable given that they are quite different in scope and nature. Not
> everybody needs to crawl on a large scale, but when they do they often use
> Nutch. And by the way Nutch does not do indexing and search - it delegates
> this to other tools like SOLR so it is mostly a crawler.
>
> The comparison between UIMA and OpenNLP is a better illustration of the
> difference between a framework and a library IMHO
>
> Julien
>
> --
> *
> *Open Source Solutions for Text Engineering
>
> http://digitalpebble.blogspot.com/
> http://www.digitalpebble.com
> http://twitter.com/digitalpebble
>
>
RE: Apache "Text Analysis" top-level project?
Posted by Jeyendran Balakrishnan <jp...@personaltube.com>.
Hi Julien,
My hats off to you and the rest of the Nutch developer team for improving
Nutch over the past several years, to the level where anybody with heavy
duty crawling needs can just use it off the shelf.
I agree with you that Lucene vs Nutch is not as clear an analogy for the
library vs framework debate.
In my usage, I have tended to use Nutch more as a library (in one
application only the crawling part, and in another just Fetcher2 [a great
component], hacked up a bit to remove its dependency on the rest of Nutch).
The point I was trying to make, not very clearly, was that Nutch aggregates
other components (Hadoop for distributed processing, Lucene/Solr for
indexing and search, Tika for parsing, etc) along with its own custom
crawler component code and custom data flow design, into a platform for
end-to-end crawling, indexing and search, as opposed to, for example, being
a pure-play crawler library on top of Hadoop.
I look forward with interest to follow how this debate evolves regarding
OpenNLP and UIMA.
Cheers,
Jeyendran
-----Original Message-----
From: Julien Nioche [mailto:lists.digitalpebble@gmail.com]
Sent: Monday, July 09, 2012 2:01 PM
To: users@opennlp.apache.org; jp@personaltube.com
Subject: Re: Apache "Text Analysis" top-level project?
Jeyendran,
One example I would suggest (at least according my view), is the difference
> between Lucene and Nutch. Being a library, Lucene has pretty much
> taken over search engine software development. Nutch, on the other
> hand, tries to be a full-fledged platform for crawling, indexing and
> search, and has not gathered anywhere near the same usage levels.
>
That Nutch does not have the same audience as Lucene is completely
understandable given that they are quite different in scope and nature. Not
everybody needs to crawl on a large scale, but when they do they often use
Nutch. And by the way Nutch does not do indexing and search - it delegates
this to other tools like SOLR so it is mostly a crawler.
The comparison between UIMA and OpenNLP is a better illustration of the
difference between a framework and a library IMHO
Julien
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
Re: Apache "Text Analysis" top-level project?
Posted by Julien Nioche <li...@gmail.com>.
Jeyendran,
One example I would suggest (at least according my view), is the difference
> between Lucene and Nutch. Being a library, Lucene has pretty much taken
> over search engine software development. Nutch, on the other hand, tries to
> be a full-fledged platform for crawling, indexing and search, and has not
> gathered anywhere near the same usage levels.
>
That Nutch does not have the same audience as Lucene is completely
understandable given that they are quite different in scope and nature. Not
everybody needs to crawl on a large scale, but when they do they often use
Nutch. And by the way Nutch does not do indexing and search - it delegates
this to other tools like SOLR so it is mostly a crawler.
The comparison between UIMA and OpenNLP is a better illustration of the
difference between a framework and a library IMHO
Julien
--
*
*Open Source Solutions for Text Engineering
http://digitalpebble.blogspot.com/
http://www.digitalpebble.com
http://twitter.com/digitalpebble
RE: Apache "Text Analysis" top-level project?
Posted by Jeyendran Balakrishnan <jp...@personaltube.com>.
It is exactly the "production"-oriented aspects of UIMA that, IMHO, make it unattractive for many users.
Almost all commercial implementations of NLP use NLP as one of the components in their overall software stack,
i.e., they typically have their own custom framework, platform, and architecture that do many things other than NLP.
What these commercial systems need are open source libraries to do specific things, leaving their developers free to put them together according to their own requirements and tradeoffs.
Any NLP system that forces the developer to use one particular way (eg UIMA) of putting things together will not be attractive,
and might steer them away from otherwise great algorithm implementations due to the significant additional baggage that the algorithms come with.
Today, there are so many ways of connecting components together, including workflows, platforms, configuration management, parallel batch processing (Hadoop, anyone?), parallel stream processing (Storm), etc. Almost the entire code-base of UIMA has nothing to do with NLP. There's a big reason why IBM gave away all this code to Apache, and kept their core algorithms to themselves - it was clear to them where the value is. At least to me, it is clear where the value is - algorithms, and not frameworks.
Software developers in industry are very capable of easily putting together their own frameworks. What they need help with are core NLP algorithms that they don't have the background to do themselves. One example I would suggest (at least according my view), is the difference between Lucene and Nutch. Being a library, Lucene has pretty much taken over search engine software development. Nutch, on the other hand, tries to be a full-fledged platform for crawling, indexing and search, and has not gathered anywhere near the same usage levels.
My vote is to please keep OpenNLP clean, smart, algorithm-centered, user-focused.
Keep it simple.
Math, stat, and algos.
And excel at it.
Please don’t dumb OpenNLP down with unnecessary bloat that any decent software team can do easily, and might often prefer to implement in a different way.
Connectors, not merging.
My two bits... :-)
Cheers,
Jeyendran
-----Original Message-----
From: Jörn Kottmann [mailto:kottmann@gmail.com]
Sent: Monday, July 09, 2012 1:50 AM
To: users@opennlp.apache.org
Subject: Re: Apache "Text Analysis" top-level project?
On 07/09/2012 05:56 AM, Lance Norskog wrote:
> Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
> top-level "Text Analysis" project? There are already cross-project
> connections between UIMA and OpenNLP. ORP seems dormant. It also seems
> a more natural place than OpenNLP for a database of tagged text.
>
>
OpenNLP and UIMA align nicely in my opinion. OpenNLP just implements engines for various NLP tasks without any further support.
UIMA on the other side can do a lot of these additional things you need to run OpenNLP in a production system e.g. scaling the engines to many machines, providing workflow support, resource loading and management, etc.
So there is not really an overlap between the two.
UIMA has some NLP related addons in their sandbox, some of them duplicate functionality which is also provided by OpenNLP e.g. pos tagging, or the dictionary annotator, but that does not seem to be that much.
Lucene contains a lot of NLP code for stemming and word segmentation in different languages. Thats probably the biggest NLP related code base next to OpenNLP at Apache.
Jörn
Re: Apache "Text Analysis" top-level project?
Posted by Jörn Kottmann <ko...@gmail.com>.
On 07/09/2012 05:56 AM, Lance Norskog wrote:
> Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
> top-level "Text Analysis" project? There are already cross-project
> connections between UIMA and OpenNLP. ORP seems dormant. It also seems
> a more natural place than OpenNLP for a database of tagged text.
>
>
OpenNLP and UIMA align nicely in my opinion. OpenNLP just implements
engines for various NLP tasks without any further support.
UIMA on the other side can do a lot of these additional things you need to
run OpenNLP in a production system e.g. scaling the engines
to many machines, providing workflow support, resource loading and
management, etc.
So there is not really an overlap between the two.
UIMA has some NLP related addons in their sandbox, some of them
duplicate functionality
which is also provided by OpenNLP e.g. pos tagging, or the dictionary
annotator, but that
does not seem to be that much.
Lucene contains a lot of NLP code for stemming and word segmentation in
different
languages. Thats probably the biggest NLP related code base next to
OpenNLP at Apache.
Jörn
Re: Apache "Text Analysis" top-level project?
Posted by Thilo Goetz <tw...@gmx.de>.
On 09/07/12 05:56, Lance Norskog wrote:
> Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
> top-level "Text Analysis" project? There are already cross-project
> connections between UIMA and OpenNLP. ORP seems dormant. It also seems
> a more natural place than OpenNLP for a database of tagged text.
>
What advantage would you see in having a joint project,
as opposed to the individual ones? As you say, there
already is pretty good interaction between OpenNLP and
UIMA. At the same time, there are lots of people
just interested in one or the other. Would you see
it more of an advantage to the developers, or the users?
--Thilo
Re: Apache "Text Analysis" top-level project?
Posted by Chris Fournier <ch...@gmail.com>.
Seconded.
As a software developer in a commercial company I use OpenNLP because of
it's simplicity, and as a researcher I want a place to submit
production-quality code to someday where it will then be used by industry.
All I want is a JAR and models to be able to use bits and pieces of
OpenNLP for very common tasks such as tokenization, sentence splitting,
chunking, etc. Do common NLP tasks, do them well, and do them and only
them. All else (web services, fancy storage formats, UI, highly
specialized pipelines) are features that I wouldn't use.
Chris
On Mon, Jul 9, 2012 at 12:05 AM, Jeyendran Balakrishnan <jp@personaltube.com
> wrote:
> My vote would be to keep OpenNLP independent.
> It is easy to use and there is good interest and activity.
> UIMA suffers from a high degree of unnecessary complexity.
> And ORP indeed seems to be inactive.
>
>
> -----Original Message-----
> From: Lance Norskog [mailto:goksron@gmail.com]
> Sent: Sunday, July 08, 2012 8:56 PM
> To: users@opennlp.apache.org
> Subject: Apache "Text Analysis" top-level project?
>
> Would it make sense to join OpenNLP, UIMA, and Open Relevance into one
> top-level "Text Analysis" project? There are already cross-project
> connections between UIMA and OpenNLP. ORP seems dormant. It also seems a
> more natural place than OpenNLP for a database of tagged text.
>
>
> --
> Lance Norskog
> goksron@gmail.com
>
>
RE: Apache "Text Analysis" top-level project?
Posted by Jeyendran Balakrishnan <jp...@personaltube.com>.
My vote would be to keep OpenNLP independent.
It is easy to use and there is good interest and activity.
UIMA suffers from a high degree of unnecessary complexity.
And ORP indeed seems to be inactive.
-----Original Message-----
From: Lance Norskog [mailto:goksron@gmail.com]
Sent: Sunday, July 08, 2012 8:56 PM
To: users@opennlp.apache.org
Subject: Apache "Text Analysis" top-level project?
Would it make sense to join OpenNLP, UIMA, and Open Relevance into one top-level "Text Analysis" project? There are already cross-project connections between UIMA and OpenNLP. ORP seems dormant. It also seems a more natural place than OpenNLP for a database of tagged text.
--
Lance Norskog
goksron@gmail.com