Portuguese

[Jason Baldridge <ja...@gmail.com>]

One thing that I think might be nice moving forward is to develop a robust
set of models and test sets that involve at least two languages. I'm
thinking Portuguese would be a good one in addition to English since:

   - several of us speak it (I'm a non-native speaker who lived in Brazil
   for a couple of years -- who else?)
   - there are truly free annotated resources for it:
   http://www.linguateca.pt/
   - it's pretty darn widely spoken in the world, both as first and second
   language

Doing something like this would help push the annotation effort forward as
well. E.g. we commit to providing support for a language means we need to
get at least some annotations going for each level of analysis we want to
support, and that will in turn spur development on the tool that Jorn has
been putting together.

Jason

-- 
Jason Baldridge
Associate Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge

Re: Portuguese

[Jörn Kottmann <ko...@gmail.com>]

On 12/6/11 11:35 AM, Alec Taylor wrote:
> Instead, how about a complete Latin implementation, then we slowly go
> about extending it to support all languages which have extended from
> this.

We mostly have this, to add a new language like Swedish or Italian we likely
wouldn't need to change a line of code when it comes to the core components.
Eventually you want to add support for a new data set to the formats 
package.

Jörn

Re: Portuguese

[Alec Taylor <al...@gmail.com>]

Instead, how about a complete Latin implementation, then we slowly go
about extending it to support all languages which have extended from
this.

I believe there are three languages: Chinese, Latin and Greek; which
we'd need to build off to get a pretty complete foundation of all
languages.

Then we slowly simulate the syntax and language evolution across time,
and voilà!

On Tue, Dec 6, 2011 at 8:33 AM, Jason Baldridge
<ja...@gmail.com> wrote:
> One thing that I think might be nice moving forward is to develop a robust
> set of models and test sets that involve at least two languages. I'm
> thinking Portuguese would be a good one in addition to English since:
>
>   - several of us speak it (I'm a non-native speaker who lived in Brazil
>   for a couple of years -- who else?)
>   - there are truly free annotated resources for it:
>   http://www.linguateca.pt/
>   - it's pretty darn widely spoken in the world, both as first and second
>   language
>
> Doing something like this would help push the annotation effort forward as
> well. E.g. we commit to providing support for a language means we need to
> get at least some annotations going for each level of analysis we want to
> support, and that will in turn spur development on the tool that Jorn has
> been putting together.
>
> Jason
>
> --
> Jason Baldridge
> Associate Professor, Department of Linguistics
> The University of Texas at Austin
> http://www.jasonbaldridge.com
> http://twitter.com/jasonbaldridge

Re: Portuguese

[Jason Baldridge <ja...@gmail.com>]

Yes, this is very much in line with what I'm thinking. I was suggesting
starting with two languages that I knew the team has specific first-hand
knowledge of, and for which we have decent resources to start the process.
And, if others want to chime in with other languages, so much the better!

On Tue, Dec 6, 2011 at 3:34 AM, Jörn Kottmann <ko...@gmail.com> wrote:

> On 12/6/11 3:23 AM, Jason Baldridge wrote:
>
>> What I'm thinking of here is in part process so that we know the steps to
>> create the data for adding new languages such that others who want to add
>> them can do so much more easily, basically following a recipe and putting
>> in the effort.
>>
>> If others want to spearhead efforts to add other languages, that's also
>> great. The more the merrier, as long as we use a standardized, replicable
>> process.
>>
>
> I think we should stick to the idea to make a web based annotation tool
> which can be used by our community to label data.
>
> To get started we can use the existing tooling and create a test set.
> The test is in my opinion the first step anyway, we need some data to
> determine how and what is labeled, we need some sample data to teach
> new annotators and we need a test set to evaluate models trained
> on community labeled data.
>
> Jörn
>
>
>


-- 
Jason Baldridge
Associate Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge

Re: Portuguese

[Jörn Kottmann <ko...@gmail.com>]

On 12/6/11 3:23 AM, Jason Baldridge wrote:
> What I'm thinking of here is in part process so that we know the steps to
> create the data for adding new languages such that others who want to add
> them can do so much more easily, basically following a recipe and putting
> in the effort.
>
> If others want to spearhead efforts to add other languages, that's also
> great. The more the merrier, as long as we use a standardized, replicable
> process.

I think we should stick to the idea to make a web based annotation tool
which can be used by our community to label data.

To get started we can use the existing tooling and create a test set.
The test is in my opinion the first step anyway, we need some data to
determine how and what is labeled, we need some sample data to teach
new annotators and we need a test set to evaluate models trained
on community labeled data.

Jörn

Re: Portuguese

[Jason Baldridge <ja...@gmail.com>]

Great!

What I'm thinking of here is in part process so that we know the steps to
create the data for adding new languages such that others who want to add
them can do so much more easily, basically following a recipe and putting
in the effort.

If others want to spearhead efforts to add other languages, that's also
great. The more the merrier, as long as we use a standardized, replicable
process.

Jason

On Mon, Dec 5, 2011 at 6:26 PM, william.colen@gmail.com <
william.colen@gmail.com> wrote:

> I am a Portuguese native speaker.
> I contributed with parsers for some of the Linguateca formats and we can
> train models for most of the OpenNLP tools now. It is missing the
> Coreference and Parser, but I will have time to work on that next year. (I
> still have to work with the paper and data you sent, about the Portuguese
> parser, but I had to change my priorities).
>
> And yes, the tools Jörn is working on are great. I hope I can start
> using/working with it as soon as I finish my thesis, in a couple of months.
> I am thinking of organizing an Apache OpenNLP event here with students from
> the Linguistics and CS departments to bootstrap the Portuguese annotation
> project, maybe we will have a few new contributors!
>
> On Mon, Dec 5, 2011 at 7:33 PM, Jason Baldridge <jasonbaldridge@gmail.com
> >wrote:
>
> > One thing that I think might be nice moving forward is to develop a
> robust
> > set of models and test sets that involve at least two languages. I'm
> > thinking Portuguese would be a good one in addition to English since:
> >
> >   - several of us speak it (I'm a non-native speaker who lived in Brazil
> >   for a couple of years -- who else?)
> >   - there are truly free annotated resources for it:
> >   http://www.linguateca.pt/
> >   - it's pretty darn widely spoken in the world, both as first and second
> >   language
> >
> > Doing something like this would help push the annotation effort forward
> as
> > well. E.g. we commit to providing support for a language means we need to
> > get at least some annotations going for each level of analysis we want to
> > support, and that will in turn spur development on the tool that Jorn has
> > been putting together.
> >
> > Jason
> >
> > --
> > Jason Baldridge
> > Associate Professor, Department of Linguistics
> > The University of Texas at Austin
> > http://www.jasonbaldridge.com
> > http://twitter.com/jasonbaldridge
> >
>



-- 
Jason Baldridge
Associate Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge

Re: Portuguese

["william.colen@gmail.com" <wi...@gmail.com>]

I am a Portuguese native speaker.
I contributed with parsers for some of the Linguateca formats and we can
train models for most of the OpenNLP tools now. It is missing the
Coreference and Parser, but I will have time to work on that next year. (I
still have to work with the paper and data you sent, about the Portuguese
parser, but I had to change my priorities).

And yes, the tools Jörn is working on are great. I hope I can start
using/working with it as soon as I finish my thesis, in a couple of months.
I am thinking of organizing an Apache OpenNLP event here with students from
the Linguistics and CS departments to bootstrap the Portuguese annotation
project, maybe we will have a few new contributors!

On Mon, Dec 5, 2011 at 7:33 PM, Jason Baldridge <ja...@gmail.com>wrote:

> One thing that I think might be nice moving forward is to develop a robust
> set of models and test sets that involve at least two languages. I'm
> thinking Portuguese would be a good one in addition to English since:
>
>   - several of us speak it (I'm a non-native speaker who lived in Brazil
>   for a couple of years -- who else?)
>   - there are truly free annotated resources for it:
>   http://www.linguateca.pt/
>   - it's pretty darn widely spoken in the world, both as first and second
>   language
>
> Doing something like this would help push the annotation effort forward as
> well. E.g. we commit to providing support for a language means we need to
> get at least some annotations going for each level of analysis we want to
> support, and that will in turn spur development on the tool that Jorn has
> been putting together.
>
> Jason
>
> --
> Jason Baldridge
> Associate Professor, Department of Linguistics
> The University of Texas at Austin
> http://www.jasonbaldridge.com
> http://twitter.com/jasonbaldridge
>

RE: Portuguese

[Boris Galitsky <bg...@hotmail.com>]

I can help with Russian at some pointBoris


> Date: Tue, 6 Dec 2011 00:35:24 +0100
> From: kottmann@gmail.com
> To: opennlp-dev@incubator.apache.org
> Subject: Re: Portuguese
> 
> On 12/6/11 12:14 AM, Aliaksandr Autayeu wrote:
> > What about more diverse languages? Chinese, Arabic, Russian might be good
> > examples. In a sense, they will provide wider test coverage. And any of the
> > above has quite a large audience. Annotated resources and speakers in the
> > team might be a problem, though.
> 
> One of the project goals is to add support for more languages. So if 
> someone wants to
> work on support for a new language this is always welcome, doesn't matter
> which language.
> 
> If you would like to add support for a new language, please go ahead.
> 
> I personally don't have a need for Arabic or Chinese currently, and that 
> is why
> I cannot do it myself, but I am happy to help out, answer question, and just
> do whortat I can to supp it.
> 
> Jörn
 		 	   		  

Re: Portuguese

[Jörn Kottmann <ko...@gmail.com>]

On 12/6/11 12:14 AM, Aliaksandr Autayeu wrote:
> What about more diverse languages? Chinese, Arabic, Russian might be good
> examples. In a sense, they will provide wider test coverage. And any of the
> above has quite a large audience. Annotated resources and speakers in the
> team might be a problem, though.

One of the project goals is to add support for more languages. So if 
someone wants to
work on support for a new language this is always welcome, doesn't matter
which language.

If you would like to add support for a new language, please go ahead.

I personally don't have a need for Arabic or Chinese currently, and that 
is why
I cannot do it myself, but I am happy to help out, answer question, and just
do whortat I can to supp it.

Jörn

Re: Portuguese

[Aliaksandr Autayeu <al...@autayeu.com>]

What about more diverse languages? Chinese, Arabic, Russian might be good
examples. In a sense, they will provide wider test coverage. And any of the
above has quite a large audience. Annotated resources and speakers in the
team might be a problem, though.

Aliaksandr

On Mon, Dec 5, 2011 at 10:33 PM, Jason Baldridge
<ja...@gmail.com>wrote:

> One thing that I think might be nice moving forward is to develop a robust
> set of models and test sets that involve at least two languages. I'm
> thinking Portuguese would be a good one in addition to English since:
>
>   - several of us speak it (I'm a non-native speaker who lived in Brazil
>   for a couple of years -- who else?)
>   - there are truly free annotated resources for it:
>   http://www.linguateca.pt/
>   - it's pretty darn widely spoken in the world, both as first and second
>   language
>
> Doing something like this would help push the annotation effort forward as
> well. E.g. we commit to providing support for a language means we need to
> get at least some annotations going for each level of analysis we want to
> support, and that will in turn spur development on the tool that Jorn has
> been putting together.
>
> Jason
>
> --
> Jason Baldridge
> Associate Professor, Department of Linguistics
> The University of Texas at Austin
> http://www.jasonbaldridge.com
> http://twitter.com/jasonbaldridge
>