[jira] [Work started] (CAMEL-25670) camel-jbang - camel run: --data-dir for data folders that dev mode does not watch

["Claus Ibsen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on CAMEL-25670 started by Claus Ibsen.
-------------------------------------------
> camel-jbang - camel run: --data-dir for data folders that dev mode does not watch
> ---------------------------------------------------------------------------------
>
>                 Key: CAMEL-25670
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25670
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-jbang
>            Reporter: Claus Ibsen
>            Assignee: Claus Ibsen
>            Priority: Major
>             Fix For: 4.24.0
>
>
> With {{camel run --source-dir}}, dev mode watches every file under the source directory, recursively ({{routesReloadPattern=*}} and {{routesReloadDirectoryRecursive=true}}, set in {{Run.java}}). When the routes read and write data files in a folder under it, their own file activity counts as a change and reloads the routes, again and again.
> An example from a local-model benchmark:
> {noformat}
> - route:
>     id: courier
>     from:
>       uri: file
>       parameters:
>         directoryName: samples
>         noop: true
>       steps:
>         - to:
>             uri: file
>             parameters:
>               directoryName: inbox
> - route:
>     id: invoices
>     from:
>       uri: file
>       parameters:
>         directoryName: inbox
>       steps:
>         - log:
>             message: "Received ${file:name}"
> {noformat}
> Every reload restarts {{courier}} with an empty in-memory idempotent repository, so it copies {{samples/*}} into {{inbox/}} again. That is a change under the source directory, so the routes reload again. With the default 2 s scan, the routes reloaded every 2.0 s for as long as the integration ran (63 reloads in 2 minutes), and every reload processed the same files again. To a beginner it looks like duplicate processing, not like dev mode.
> h3. Proposal
> # {{--data-dir}} for {{camel run}}: one or more folders, comma-separated and relative to the source directory, that dev mode does not watch, for example {{camel run --source-dir . --data-dir=samples,inbox,archive}}. Also as a property, so it can be set in application.properties. Other tools can use it too: {{camel export}} can leave the data out, and the AI file tools can tell data from source.
> # A warning when reloads repeat: if the routes reload, say, 5 times within 15 seconds, log one WARN that names the folder of the files that triggered them, for example "Dev mode reloaded 5 times in 10 s, each time for files under inbox/. If that folder holds data, start with --data-dir=inbox." The reload strategy already knows which files changed, so this needs no look at the endpoints.
> # The built-in examples (the {{camel run --example}} and the example browser ones, {{dsl/camel-jbang/camel-jbang-core/src/main/resources/examples}}): check the ones that use files, and give those that write into their own folder a data dir in their application.properties. Of the 11 that use the file component, three write into their own folder:
> ** {{connect/file-processing}}: to {{inbox}} and {{archive}}, and its {{inbox}} consumer moves files to {{done}} and {{failed}};
> ** {{fail-well/error-handling}}: to {{orders}} and {{parked}};
> ** {{transform/csv-to-json}}: to {{outbox}}.
> The other eight only read their files ({{noop=true}}): no change, so no loop.
> A shorter scan interval (CAMEL-25661) makes the loop faster: at 250 ms the same example reloaded 178 times.
> _Claude Code on behalf of davsclaus_



--
This message was sent by Atlassian Jira
(v8.20.10#820010)