You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@nifi.apache.org by "Richard Zuidhof (Jira)" <ji...@apache.org> on 2020/11/11 12:08:00 UTC
[jira] [Updated] (NIFI-7997) ListFTP does not use HTTP proxy
[ https://issues.apache.org/jira/browse/NIFI-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Richard Zuidhof updated NIFI-7997:
----------------------------------
Description:
After configuring a HTTP type proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy.
You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy:
2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go
When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to Connection timed out (Connection timed out): java.net.ConnectException: Connection timed out (Connection timed out) java.net.ConnectException: Connection timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
was:
After configuring a proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy.
You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy:
2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go
When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to Connection timed out (Connection timed out): java.net.ConnectException: Connection timed out (Connection timed out) java.net.ConnectException: Connection timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
> ListFTP does not use HTTP proxy
> -------------------------------
>
> Key: NIFI-7997
> URL: https://issues.apache.org/jira/browse/NIFI-7997
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.11.4
> Reporter: Richard Zuidhof
> Priority: Major
>
> After configuring a HTTP type proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy.
> You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy:
> 2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go
> When using an unreachable IP (like 10.123.1.1) you will see this: 2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to Connection timed out (Connection timed out): java.net.ConnectException: Connection timed out (Connection timed out) java.net.ConnectException: Connection timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)