[jira] [Updated] (CAMEL-25896) camel-ftp - Let sendNoop=false also skip the consumer's pre-poll noop (SFTP sends keepalive@jcraft.com on every poll)

["Claus Ibsen (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CAMEL-25896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Claus Ibsen updated CAMEL-25896:
--------------------------------
    Fix Version/s: 4.24.0

> camel-ftp - Let sendNoop=false also skip the consumer's pre-poll noop (SFTP sends keepalive@jcraft.com on every poll)
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-25896
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25896
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-ftp, camel-mina-sftp, camel-sftp
>    Affects Versions: 4.18.5, 4.23.0
>            Reporter: Johan Almqvist
>            Priority: Minor
>             Fix For: 4.24.0
>
>
> h3. Problem
> Before every poll, {{RemoteFileConsumer.connectIfNecessary()}} calls {{getOperations().sendNoop()}}. For the JSch-based sftp component, {{SftpOperations.sendNoop()}} calls {{Session.sendKeepAliveMsg()}}, which sends a {{keepalive@jcraft.com}} global request. The {{sendNoop}} option (CAMEL-8807) is read only by {{RemoteFileProducer}}, so a consumer cannot turn this off. The code is the same in 4.18.5, in 4.23.0 and on main, where {{RemoteFileConsumer}} now lives in camel-ftp-common.
> ProFTPD's mod_sftp does not support that request. It replies with a failure and writes one SFTPLog line for each one:
> {noformat}
> <timestamp> mod_sftp/<version>[<pid>]: client sent GLOBAL_REQUEST for 'keepalive@jcraft.com', denying
> {noformat}
> With the default {{delay=500}}, that is about two lines a second, roughly 170,000 a day, for each route polling the server. The volume grows with every route, and it buries authentication failures and other real errors in the same log. mod_sftp writes the line for every global request it does not handle (the same code in 1.3.7 to 1.3.9, 1.3.10rc4 and master), so the server cannot silence it without turning SFTPLog off entirely.
> h3. For the JSch-based sftp component the keepalive does not change the outcome
> * {{Session.sendKeepAliveMsg()}} writes the request and returns without waiting for the reply, so it cannot tell a live server from a dead one.
> * {{SftpOperations.sendNoop()}} only sends it when {{isConnected()}} is true. Otherwise it returns false without sending anything.
> * If the write throws, {{sendNoop()}} returns false and {{connectIfNecessary()}} calls {{connect()}}. That returns at once while {{isConnected()}} is still true, so nothing reconnects at that point. If JSch has already marked the session disconnected, {{isConnected()}} alone triggers the same reconnect.
> * When the server closes the socket (the CAMEL-6591 case), JSch's reader thread marks the session disconnected, so {{isConnected()}} already returns false and {{sendNoop()}} sends nothing.
> * A session that dies without the client noticing is recovered as it is today: the poll fails, {{RemoteFilePollingConsumerPollStrategy.rollback()}} force-disconnects, and the next poll reconnects.
> So for an sftp consumer the result is the same as calling {{isConnected()}}, plus one packet per poll.
> h3. Proposal
> Let {{sendNoop}} also apply to the consumer's pre-poll check. With {{sendNoop=false}}, the consumer relies on {{isConnected()}}:
> {code:java}
> protected void connectIfNecessary() {
>     boolean isConnected = false;
>     try {
>         if (getEndpoint().getConfiguration().isSendNoop()) {
>             isConnected = getOperations().sendNoop();
>         } else {
>             isConnected = getOperations().isConnected();
>         }
>     } catch (Exception ex) {
>         // unchanged
>     }
>     // unchanged from here
> }
> {code}
> * The default stays {{true}}. The only users affected are those who already set {{sendNoop=false}} on a consumer URI, where it is ignored today (for example a URI shared between {{from()}} and {{to()}}). That is worth a line in the upgrade guide.
> * {{RemoteFileConsumer}} is shared, so this also applies to ftp, ftps and mina-sftp. There {{sendNoop()}} is a real round trip (FTP {{NOOP}}, MINA {{stat("/")}}). With {{sendNoop=false}}, a connection the server has closed is found only when the poll fails, and the poll strategy then reconnects on the next poll. That is the same trade-off {{sendNoop=false}} already makes for producers.
> * The {{@UriParam}} label changes from {{producer,advanced}} to {{common,advanced}} (as {{fastExistsCheck}} uses). The description and the {{setSendNoop()}} Javadoc cover the pre-poll check as well as the pre-write check. Regenerating updates the component JSON, the catalog and the endpoint DSL.
> * Tests: a consumer counterpart of the existing producer {{sendNoop=false}} test for ftp and sftp, and a unit test that with {{sendNoop=false}} the consumer calls {{isConnected()}} instead of {{sendNoop()}} and still reconnects when it returns false.
> h3. Alternatives considered
> * Going back to {{sendIgnore()}}: CAMEL-10149 moved away from it because ProFTPD closed the connection on {{SSH_MSG_IGNORE}}.
> * {{disconnect=true}}: no keepalive, but a full SSH handshake and login on every poll.
> * camel-mina-sftp: its {{sendNoop()}} is an SFTP {{stat("/")}} with no global request, but switching components is not always an option.
> * Changing {{SftpOperations.sendNoop()}} for everyone, either to return {{isConnected()}} or to make a real round trip. That would change the default for every SFTP user, so this issue proposes an opt-in that matches CAMEL-8807, but either would also solve it.
> h3. Workaround today
> The only way to stop the keepalive today is to subclass {{SftpOperations}}, {{SftpEndpoint}} and {{SftpComponent}} and register the subclass as {{sftp}}.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)