You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uima.apache.org by Pablo Duboue <pa...@gmail.com> on 2023/09/05 19:59:07 UTC

UIMA CPP Report

Hi,

Just some update that might be useful for the upcoming board report.

The work on UIMA CPP is ongoing and with good progress.

A suitable delivery mechanism for releases for the project has been
identified (Docker image) and the hope is to have access to a UIMA CPP
Docker Hub slot whenever the chair files for an INFRA issue for it.

Issues regarding Python multi-interpreter support have been identified and
two possible releases of the Pythonator are available, one fully embedded
and the other with multi-interpreter support. Sadly, NumPy, a popular
numeric processing library (and a requirement to most machine learning
packages in Python) does not support multi-interpreter.

Using the fully embedded library, a proof-of-concept running an annotator
with the popular spaCy NLP library was successfully executed.

Work is still ongoing packaging UIMA CPP to build as a system library
inside the Docker image rather than using custom paths.

I'm currently looking for a good application of UIMA CPP to process large
amounts of text related to LLMs (the topic du jour), to attract more
interest in the new release.

Once this base release is done, the objective is to move to support
aggregates natively in UIMA CPP and to bring back the JNI code that would
allow to run UIMAJ annotators in the pipeline.

Questions? Comments? Would love to hear them.

P

UIMA CPP use cases (was Re: UIMA CPP Report)

Posted by Pablo Duboue <pa...@gmail.com>.
Hello,

I'm back at working on UIMACPP.

On Wed, Sep 6, 2023 at 1:36 PM Eddie Epstein <ea...@gmail.com> wrote:

> Hi Pablo!
>
> Sounds like really good progress.
>
> How about using spaCy with some rule-based processing to filter the output
> of a generative LLM?
>

Yes, and I'm also working with my company in a plagiarism detection use
case that might work well.

Any other ideas are of course welcomed.

P

Re: UIMA CPP Report

Posted by Eddie Epstein <ea...@gmail.com>.
Hi Pablo!

Sounds like really good progress.

How about using spaCy with some rule-based processing to filter the output
of a generative LLM?

Best,  Eddie

On Tue, Sep 5, 2023 at 3:59 PM Pablo Duboue <pa...@gmail.com> wrote:

> Hi,
>
> Just some update that might be useful for the upcoming board report.
>
> The work on UIMA CPP is ongoing and with good progress.
>
> A suitable delivery mechanism for releases for the project has been
> identified (Docker image) and the hope is to have access to a UIMA CPP
> Docker Hub slot whenever the chair files for an INFRA issue for it.
>
> Issues regarding Python multi-interpreter support have been identified and
> two possible releases of the Pythonator are available, one fully embedded
> and the other with multi-interpreter support. Sadly, NumPy, a popular
> numeric processing library (and a requirement to most machine learning
> packages in Python) does not support multi-interpreter.
>
> Using the fully embedded library, a proof-of-concept running an annotator
> with the popular spaCy NLP library was successfully executed.
>
> Work is still ongoing packaging UIMA CPP to build as a system library
> inside the Docker image rather than using custom paths.
>
> I'm currently looking for a good application of UIMA CPP to process large
> amounts of text related to LLMs (the topic du jour), to attract more
> interest in the new release.
>
> Once this base release is done, the objective is to move to support
> aggregates natively in UIMA CPP and to bring back the JNI code that would
> allow to run UIMAJ annotators in the pipeline.
>
> Questions? Comments? Would love to hear them.
>
> P
>

Re: UIMA CPP Tickets (was Re: UIMA CPP Report)

Posted by Pablo Duboue <pa...@gmail.com>.
The UIMACPP issues are in.

On Thu, Feb 8, 2024 at 4:10 AM Pablo Duboue <pa...@gmail.com> wrote:

>
>
> On Tue, Feb 6, 2024 at 7:27 AM Richard Eckart de Castilho <re...@apache.org>
> wrote:
>
>> Hi Pablo - welcome back :)
>
>
> Thanks! And thanks for the pointers. Very useful! I'll take your advice
> and also started looking in detail to the PRs being made in the other UIMA
> GitHub projects.
>
> P
>

Re: UIMA CPP Tickets (was Re: UIMA CPP Report)

Posted by Pablo Duboue <pa...@gmail.com>.
On Tue, Feb 6, 2024 at 7:27 AM Richard Eckart de Castilho <re...@apache.org>
wrote:

> Hi Pablo - welcome back :)


Thanks! And thanks for the pointers. Very useful! I'll take your advice and
also started looking in detail to the PRs being made in the other UIMA
GitHub projects.

P

Re: UIMA CPP Tickets (was Re: UIMA CPP Report)

Posted by Richard Eckart de Castilho <re...@apache.org>.
Hi Pablo - welcome back :)

> On 6. Feb 2024, at 12:21, Pablo Duboue <pa...@gmail.com> wrote:
> 
> I will be issuing tickets for the different improvements I'm tackling:
> 
> * Update documentation
> * Separate Python annotator from Python extension
> * Respect multi threading flag for Python
> * Install as system level Python package using Debian
> * Install UIMA as system level using Debian
> * Write GH actions to produce docker image
> * Publish docker image to GitHub registry
> 
> I'm putting it here first for discussion in case anybody has any feedback.
> I'm particularly interested in whether the granularity makes sense.

Issue granularity is almost a religious topic. So here is my practical recommendation:

Create issues at the granularity at which some topic comes into your mind. So if you are currently thinking at the level of the list above, create issues at that level of granularity.

When you think more about a particular issue, you might find that there are particular smaller things you want to do to complete an issue. You can add them as a checklist to your issue (GitHub markdown has nice support for checklists).

When you actually start working on something, you might care to turn the items from that checklist into proper issues (GitHub supports that with a single click when you go with the mouse to a checklist item), then do a PR for that issue and try to get it done and integrated as quickly as possible. Try avoiding having loooong PRs.

Avoid applying that process recursively - i.e. don't try to split issues recursively into smaller issues. Better expand the checklist on the original planning issue if new steps come up. If you have more than two levels of such "nested" issues, maintaining an overview can quickly become tricky.

You can also consider using GitHub "Projects" which give you nice boards and cards that you can use to organize your work. However, in contrast to an issue, a project does not have an associated discussion thread.

I hope that helps.

Cheers,

- Richard

UIMA CPP Tickets (was Re: UIMA CPP Report)

Posted by Pablo Duboue <pa...@gmail.com>.
On Wed, Sep 6, 2023 at 1:55 PM Richard Eckart de Castilho <re...@apache.org>
wrote:

>
> There would also be an alternative to using DockerHub, namely using GitHub:
>
>
> https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry
>
> I recently moved some of my (non-Apache) projects there when DockerHub
> threatened
> to change their open source hosting model ... which they rolled back again
> after
> a few days since the community didn't like it at all ... at that point I
> was already gone.
>
> Do you have an opinion regarding DockerHub vs GitHub container registry?
>

Over the Christmas break I learned to use GitHub Actions and it really
simplifies workflows. I'm now much more inclined to use the GitHub
container registry.

My only concern is whether it has some traffic limits but that's most
probably different for Apache Foundation accounts.



> I can also see in the GitHub web UI that there have been commits comming
> into the UIMA-CPP repo - I do watch it for all activity, but for some
> reason
> I cannot remember having seen any issue mails. Are you committing
> all your changes directly to main?
>

Well, originally I was working on my own fork. I was a little confused when
you suggested I moved the work to the main repo. Now I understand what you
had in mind.


> It would be good to open issues and corresponding pull requests for those
> issues - those generate useful mails. Cf.
> https://github.com/apache/uima-uimaj
> There area also useful templates here:
>
>   https://github.com/apache/uima-uimaj/tree/main/.github
>

Lovely!

I will be issuing tickets for the different improvements I'm tackling:

* Update documentation
* Separate Python annotator from Python extension
* Respect multi threading flag for Python
* Install as system level Python package using Debian
* Install UIMA as system level using Debian
* Write GH actions to produce docker image
* Publish docker image to GitHub registry

I'm putting it here first for discussion in case anybody has any feedback.
I'm particularly interested in whether the granularity makes sense.

Best regards,

P










>
> Cheers,
>
> -- Richard
>
>

Re: UIMA CPP Report

Posted by Richard Eckart de Castilho <re...@apache.org>.
Hi Pablo

thanks for the update!

> On 5. Sep 2023, at 21:59, Pablo Duboue <pa...@gmail.com> wrote:
> 
> A suitable delivery mechanism for releases for the project has been
> identified (Docker image) and the hope is to have access to a UIMA CPP
> Docker Hub slot whenever the chair files for an INFRA issue for it.

Hm, it fell completely off my radar that I may have a todo here. Sorry!

For what it's worth, would it be ok for you to file that Docker issue with INFRA?
If INFRA needs my thumbs up, I'll be happy to give it to them as a comment.

There would also be an alternative to using DockerHub, namely using GitHub:

https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry

I recently moved some of my (non-Apache) projects there when DockerHub threatened
to change their open source hosting model ... which they rolled back again after
a few days since the community didn't like it at all ... at that point I was already gone.

Do you have an opinion regarding DockerHub vs GitHub container registry?


I can also see in the GitHub web UI that there have been commits comming
into the UIMA-CPP repo - I do watch it for all activity, but for some reason
I cannot remember having seen any issue mails. Are you committing
all your changes directly to main? 

It would be good to open issues and corresponding pull requests for those
issues - those generate useful mails. Cf. https://github.com/apache/uima-uimaj
There area also useful templates here:

  https://github.com/apache/uima-uimaj/tree/main/.github


Cheers,

-- Richard


Re: UIMA CPP Report

Posted by Roberto Carlos Toledano Gómez <rc...@gmail.com>.
Amazing job Pablo. Thanks  you.

On Tue, Sep 5, 2023, 4:00 PM Pablo Duboue <pa...@gmail.com> wrote:

> Hi,
>
> Just some update that might be useful for the upcoming board report.
>
> The work on UIMA CPP is ongoing and with good progress.
>
> A suitable delivery mechanism for releases for the project has been
> identified (Docker image) and the hope is to have access to a UIMA CPP
> Docker Hub slot whenever the chair files for an INFRA issue for it.
>
> Issues regarding Python multi-interpreter support have been identified and
> two possible releases of the Pythonator are available, one fully embedded
> and the other with multi-interpreter support. Sadly, NumPy, a popular
> numeric processing library (and a requirement to most machine learning
> packages in Python) does not support multi-interpreter.
>
> Using the fully embedded library, a proof-of-concept running an annotator
> with the popular spaCy NLP library was successfully executed.
>
> Work is still ongoing packaging UIMA CPP to build as a system library
> inside the Docker image rather than using custom paths.
>
> I'm currently looking for a good application of UIMA CPP to process large
> amounts of text related to LLMs (the topic du jour), to attract more
> interest in the new release.
>
> Once this base release is done, the objective is to move to support
> aggregates natively in UIMA CPP and to bring back the JNI code that would
> allow to run UIMAJ annotators in the pipeline.
>
> Questions? Comments? Would love to hear them.
>
> P
>