You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Louisa Frison (Jira)" <ji...@apache.org> on 2023/02/25 20:49:00 UTC
[jira] [Comment Edited] (CAMEL-19010) file component: confirm not called on idempotentRepository
[ https://issues.apache.org/jira/browse/CAMEL-19010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17693528#comment-17693528 ]
Louisa Frison edited comment on CAMEL-19010 at 2/25/23 8:48 PM:
----------------------------------------------------------------
Hi : )
I tried to debug this and it seems as if in the class GenericFileOnCompletion only the key is added to the idempotent repository, .confirm() is not called. [https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileOnCompletion.java#L126]
At first I thought, .confirm() should simply be called there, but then again, maybe the idempotentRepository of the GenericFileEndpoint is simply always lazy and only adds the key to the repo after the file has been completely processed. In this case there would be no need for a .confirm() ( ? ) and it also looks like .confirm() is usually not called:
[https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/core/camel-api/src/main/java/org/apache/camel/spi/IdempotentRepository.java#L31]
{quote}_* <li>eager: calls <tt>add</tt> and <tt>confirm</tt> if complete, or <tt>remove</tt> if failed</li>_
{quote}
{quote}_* <li>non-eager: calls <tt>contains</tt> and <tt>add</tt> if complete, or <tt>remove</tt> if failed</li>_
{quote}
In the doc I see no possibility to configure whether the idempotent repository should be eager or lazy (which is possible on the idempotent consumer - EIP [https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/core/camel-core-processor/src/main/java/org/apache/camel/processor/idempotent/IdempotentConsumer.java#L132|https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/core/camel-core-processor/src/main/java/org/apache/camel/processor/idempotent/IdempotentConsumer.java#L132):]
[https://camel.apache.org/components/3.20.x/file-component.html#_endpoint_query_option_idempotentRepository]
I'm not sure whether this is correct, but if it is, maybe one could
* try to make eager vs. lazy configurable for the file-idempotentRepository ( ? ) and then call .confirm() depending on which it is
* explicitly state in the doc that it is lazy-only
Otherwise I think calling .confirm() in GenericFileOnCompletion would do the job and I could implement it ^^
was (Author: JIRAUSER299020):
Hi : )
I tried to debug this and it seems as if in the class GenericFileOnCompletion only the key is added to the idempotent repository, .confirm() is not called. [https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/components/camel-file/src/main/java/org/apache/camel/component/file/GenericFileOnCompletion.java#L126]
At first I thought, .confirm() should simply be called there, but then again, maybe the idempotentRepository of the GenericFileEndpoint is simply always lazy and only adds the key to the repo after the file has been completely processed. In this case there would be no need for a .confirm() ( ? ) and it also looks like .confirm() is usually not called:
[https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/core/camel-api/src/main/java/org/apache/camel/spi/IdempotentRepository.java#L31]
{quote}_* <li>eager: calls <tt>add</tt> and <tt>confirm</tt> if complete, or <tt>remove</tt> if failed</li>_
{quote}
{quote}_* <li>non-eager: calls <tt>contains</tt> and <tt>add</tt> if complete, or <tt>remove</tt> if failed</li>_
{quote}
In the doc I see no possibility to configure whether the idempotent repository should be eager or lazy (which is possible on the idempotent consumer - EIP [https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/core/camel-core-processor/src/main/java/org/apache/camel/processor/idempotent/IdempotentConsumer.java#L132|https://github.com/apache/camel/blob/7e66aab7b6f1d2baeb15b432dde09f8a7f545486/core/camel-core-processor/src/main/java/org/apache/camel/processor/idempotent/IdempotentConsumer.java#L132):]
[https://camel.apache.org/components/3.20.x/file-component.html#_endpoint_query_option_idempotentRepository]
I'm not sure whether this is correct, but if it is, maybe one could
* try to make eager vs. lazy configurable for the file-idempotentRepository ( ? ) and then call .confirm() depending on which it is
* explicitly state in the doc that it is lazy-only
Otherwise I think calling .confirm() in GenericFileOnCompletion would do the job and I could implement it ^^
> file component: confirm not called on idempotentRepository
> ----------------------------------------------------------
>
> Key: CAMEL-19010
> URL: https://issues.apache.org/jira/browse/CAMEL-19010
> Project: Camel
> Issue Type: New Feature
> Components: camel-file
> Affects Versions: 3.20.1
> Reporter: Marco Bungart
> Priority: Major
>
> When we bind an idempotent repository to a file component:
> {{from(}}
> {{ file("in")}}
> {{ .noop(true)}}
> {{ .idempotent(true)}}
> {{ .idempotentRepository(idempotentRepository)}}
> {{ .idempotentKey("${[file:name|file:///name]}"))}}
> {{ .}}
> {{ .}}
> {{ .}}
> then method {{confirm(...)}} should be called after the exchange has been processed.
> —
> Expected behaviour: method {{confirm(...)}} should be called after the exchange has been processed
> —
> Actual behaviour: method {{confirm(...)}} is not called.
> —
> Reproducer:
> * Working scenario: bind the idempotency repository "after" the file producer:
> ** checkout [this {{github.com}} repo|https://github.com/turing85/camel-quarkus-idempotent-consumer]
> ** start the database: {{cd local-deployment && docker compose up -d && cd ..}}
> ** start the quarkus application. {{./mvnw quarkus:dev}}
> ** in a separate terminal, move some file in folder {{in}}
> ** go to pgadmin4: [http://localhost:8091|http://localhost:8091/] (username: {{{}pgadmin4@pgadmin.org{}}}, password: {{{}pgadmin4{}}})
> ** on the left side, click on {{{}docker{}}}, then on {{{}postgres{}}}. When prompted for a password for databaes {{{}postgres{}}}, enter {{postgres}}
> ** click {{{}Databases -> camel -> Schemas -> public -> tables{}}}, right-click {{{}idempotency{}}}, select "show data"
> ** observe that the entry has the column {{done}} set to {{true}}
> ** stop the quarkus application
> ** clear database table {{idempotency}}
> ** delete folders {{in}} and {{{}out{}}}: {{rm -rf in && rm -rf out}}
> * Bug scenario:
> ** switch to branch {{{}repository-on-file-producer{}}}: {{git fetch && git switch repository-on-file-producer}}
> ** start quarkus application and repeat the test
> ** go to pgadmin4, observe that a row was created, but {{done}} is not set to {{true}}
> Going further, we can also set a break point in or on [method {{confirm(...)}}|https://github.com/turing85/camel-quarkus-idempotent-consumer/blob/repository-on-file-producer/src/main/java/de/turing85/CustomJdbcMessageIdRepository.java#L103], and it will not be triggered.
> —
> Remark: this holds also true when a repository is set as {{inProgressRepository(...)}}
> —
> FTR: [Relevant zulip chat|https://camel.zulipchat.com/#narrow/stream/257298-camel/topic/camel.20file.20idempotent.20repository.2C.20when.20is.20a.20key.20committed.3F]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)