[jira] [Work started] (CAMEL-25906) camel-ftp / sftp / smb / azure-files - checksumFileAlgorithm writes no checksum file since 4.18, or the checksum of an earlier file producer's payload

["shashank (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on CAMEL-25906 started by shashank.
----------------------------------------
> camel-ftp / sftp / smb / azure-files - checksumFileAlgorithm writes no checksum file since 4.18, or the checksum of an earlier file producer's payload
> ------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25906
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25906
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-file, camel-ftp
>            Reporter: shashank
>            Assignee: shashank
>            Priority: Major
>
> {{checksumFileAlgorithm}} ("calculate a checksum from the file that has been written ... and store the result in the CamelFileChecksum header"; {{checksumWriteFile}} writes it as {{<file>.<algorithm>}}) is declared on {{GenericFileEndpoint}}, so it is an option of every file component: file, ftp, ftps, sftp, mina-sftp, scp, smb and azure-files. CAMEL-20172 (4.3) added it for "camel-file/ftp": {{GenericFileProducer.writeChecksumFile}} calculated the checksum from the body for every producer.
> CAMEL-22846 (4.18) moved the calculation into {{FileOperations.storeFile}} (a digest while it writes the local file), and {{GenericFileProducer.writeChecksumFile}} (main 74ee085f7d64, line 290) now only writes the header that the operations set:
> {code:java}
> String hash = exchange.getMessage().getHeader(FileConstants.FILE_CHECKSUM, String.class);
> if (hash != null) {
>     ... operations.storeFileDirectly(checksumFileName, hash) ...
> }
> {code}
> Only {{FileOperations}} sets {{CamelFileChecksum}}. So since 4.18:
> * an ftp, ftps, sftp, mina-sftp, scp, smb or azure-files producer with {{checksumFileAlgorithm}} uploads the file and writes no checksum file and no header, without an error or a log line. A receiver that waits for {{<file>.sha256}} (or verifies it) never gets it; an upgrade from 4.14 LTS to 4.18 LTS loses the checksum files silently;
> * when an earlier file producer of the route set the header, e.g. {{.to("file:archive?checksumFileAlgorithm=sha256").to("sftp:...?checksumFileAlgorithm=sha256")}}, the remote producer writes that checksum: the checksum of the archive copy, which is not the uploaded file as soon as the route changed the body in between, or the remote producer encodes it with its {{charset}}.
> *Fix* (in {{GenericFileProducer}}, one place for all components): before the write, a {{CamelFileChecksum}} header set earlier is removed; for operations other than {{FileOperations}} the checksum is calculated of the bytes the remote operations upload. All of them read the body as an {{InputStream}} ({{getMandatoryBody(InputStream.class)}}), or through {{GenericFileHelper.toInputStream}} when the endpoint has a {{charset}}. A stream body that can be read once is wrapped in a {{DigestInputStream}} that the upload reads, so its checksum takes no second pass and no memory; with a {{charset}} such a stream is kept as a stream cache first (memory, or a spool file when spooling is enabled). Other bodies (a file, bytes, text, a stream cache) are read once more for the checksum: for a large file that is a second read from disk, the price of keeping the fix in one place instead of a digest in the upload code of each of the seven operations classes. With {{fileExist=Ignore}} and an existing file nothing is uploaded and no checksum is written, as with the file component. {{FileOperations}} is unchanged. Upgrade note in the 4.24 guide.
> *Tests*: new {{FtpProducerChecksumFileAlgorithmIT}} (5 tests, camel-ftp, the embedded FTP server of camel-test-infra-ftp, no Docker): the default ({{checksumWriteFile=true}}) with a String body, a one-shot stream body, a String and a one-shot stream with {{charset=ISO-8859-1}}, {{checksumWriteFile=false}} (header only), and a file producer earlier in the route. On main all 5 fail: {{Checksum file should exist ==> expected: <true> but was: <false>}} (3), {{expected: <a591a6d4...> but was: <null>}} (header), and {{expected: <01285a4c...> but was: <a591a6d4...>}} (the checksum of the archived {{Hello World}} written for an uploaded {{Bye World}}). With the fix: camel-ftp unit tests 81 run, 0 failures (9 skipped); camel-ftp producer ITs ({{*Producer*IT}}: 42 FTP and 4 SFTP classes with the embedded servers) 86 run, 0 failures (3 skipped); camel-core {{org.apache.camel.component.file}} tests 474 run, 0 failures (12 skipped); camel-file 28 run.
> *How it was found*: following up the producer paths of {{checksumFileAlgorithm}} while fixing its charset path in camel-file (separate ticket): the option is declared on the shared endpoint, the calculation lives in one subclass of {{GenericFileOperations}}.
> *Affected*: main, camel-4.23.x, camel-4.22.x and camel-4.18.x (read, not run: {{writeChecksumFile}} reads only the header, and no remote operations set it). camel-4.14.x calculated the checksum for every producer.
> _Filed with Claude Code on behalf of allthingssecurity._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)