Consistently loosing CASes

[Daniel Cosio <dc...@gmail.com>]

Here is the scenario

I have a client using Kubernetes and we run a single pipeline in a pod.. We scale up/down bases on queue pressure..

I’m observing on a consistent basis that the 
1, first time the pipelines start(one pipeline), the pipeline will pull 2 documents off of the input queue.. 
2. The pipeline is shutdown with SIGTERM.. It completes the first document and returns the CAS. It also returns the second CAS it had pulled.
3. A new pipeline is started..
4. It pulls 2 documents off the input queue.
5. The pipeline is shutdown with a SIGTERM. It completes the document and returns the CAS.. BUT this time it DOES NOT return the second CAS.


This happens consistently. So, I’m loosing documents.

I cannot for the life of life of me figure out why the new pipeline is not returning the CAS..

This is related to another email I sent last week. But I assumed it was the (what appears to be) extra consumer.. We have 3 consumers, 1 for meta, 1 for messages and apparently there is another one that I don’t know what it’s purpose is but is not related to the consumer pulling multiple records off the queue.


So, is there a way I can get the consumer to only pull 1 document off the queue? If not, is the second CAS not being returned a bug?


Im running 2.10.3, uimaj-as.. We use ddspring to start things up.

I even set the pipeline numberOfCases to 1 thinking it may only pull a single doc off the input queue, but it still pulled 2

Daniel Cosio
dccosio@gmail.com

Re: Consistently loosing CASes

[Dan Cosio <dc...@gmail.com>]

Thanks I’ll check them out

Dan Cosio Sent from Gmail Mobile


On Thu, Jun 20, 2024 at 5:33 AM Pablo Duboue <pa...@gmail.com> wrote:

> Looking at some old emails, I found some pointers from Richard regarding
> Spark+UIMA:
>
> On Tue, Jun 4, 2024 at 8:01 AM Daniel Cosio <dc...@gmail.com> wrote:
>
> Would you know what the suggested replacement for Uima-AS is?
>>
>
> Take a look:
> https://lists.apache.org/thread/1nbgyrfgxlx77pp5cy91nsn08oyxkj7r
>
> P
>
>

Re: Consistently loosing CASes

[Pablo Duboue <pa...@gmail.com>]

Looking at some old emails, I found some pointers from Richard regarding
Spark+UIMA:

On Tue, Jun 4, 2024 at 8:01 AM Daniel Cosio <dc...@gmail.com> wrote:

Would you know what the suggested replacement for Uima-AS is?
>

Take a look:
https://lists.apache.org/thread/1nbgyrfgxlx77pp5cy91nsn08oyxkj7r

P

Re: Consistently loosing CASes

[Richard Eckart de Castilho <re...@apache.org>]

Hi Daniel,

> On 4. Jun 2024, at 14:01, Daniel Cosio <dc...@gmail.com> wrote:
> 
> Would you know what the suggested replacement for Uima-AS is?

I believe there is currently no replacement for UIMA-AS. It has been retired because
currently there is nobody maintaining it, not because it has been superseded.

The sub-project can easily be brought back if there is somebody who would like to take
over maintainership.

Best,

-- Richard

Re: Consistently loosing CASes

[Daniel Cosio <dc...@gmail.com>]

Sorry for not replying to the other email. The links did not address my issue. The one related to SIGTERM was for Lima 3.x and Im using 2.10. The other explaining the consumers explained the consumers but did not address why
2 documents are being pulled from the broker input queue which is the main issue.


I’m strapped for time and Im implementing a db solution to tract the documents so I can determine which ones are being lost so I can resubmit. 


Would you know what the suggested replacement for Uima-AS is?

Thanks for responding.


Daniel Cosio
dccosio@gmail.com



> On Jun 3, 2024, at 11:10 PM, Pablo Duboue <pa...@gmail.com> wrote:
> 
> Hi Daniel,
> 
> On Monday, June 3, 2024, Daniel Cosio <dccosio@gmail.com <ma...@gmail.com>> wrote:
> > Here is the scenario
> >
> > This is related to another email I sent last week. But I assumed it was the (what appears to be) extra consumer.. We have 3 consumers, 1 for meta, 1 for messages and apparently there is another one that I don’t know what it’s purpose is but is not related to the consumer pulling multiple records off the queue.
> 
> I know. I sent you an email with a link from the mailing list archives discussing the consumers and mentioning they were unlikely to be related to your problem. You didn't replied to that information. Nor the other link in my email about the shutdown hook. It's hard to help you if we talk past each other. (Are you subscribed to the mailing list? I cc: you above just to make sure.)
> 
> The project no longer has people with UIMA-AS expertise and it has thus been retired. UIMAv3 does not officially support UIMA-AS as far as I understand it.
> 
> At this stage I suggest you open an issue for this and post a minimal code setup to reproduce the bug. Then document the steps you have taken so far to fix it. Over that stream of work we can pool brains and try to help.
> 
> P
> 

Re: Consistently loosing CASes

[Pablo Duboue <pa...@gmail.com>]

Hi Daniel,

On Monday, June 3, 2024, Daniel Cosio <dc...@gmail.com> wrote:
> Here is the scenario
>
> This is related to another email I sent last week. But I assumed it was
the (what appears to be) extra consumer.. We have 3 consumers, 1 for meta,
1 for messages and apparently there is another one that I don’t know what
it’s purpose is but is not related to the consumer pulling multiple records
off the queue.

I know. I sent you an email with a link from the mailing list archives
discussing the consumers and mentioning they were unlikely to be related to
your problem. You didn't replied to that information. Nor the other link in
my email about the shutdown hook. It's hard to help you if we talk past
each other. (Are you subscribed to the mailing list? I cc: you above just
to make sure.)

The project no longer has people with UIMA-AS expertise and it has thus
been retired. UIMAv3 does not officially support UIMA-AS as far as I
understand it.

At this stage I suggest you open an issue for this and post a minimal code
setup to reproduce the bug. Then document the steps you have taken so far
to fix it. Over that stream of work we can pool brains and try to help.

P