You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Alexander Trushev (Jira)" <ji...@apache.org> on 2022/09/08 07:07:00 UTC
[jira] [Updated] (HUDI-4523) Sequential submitting of flink jobs leads to java.net.ConnectException
[ https://issues.apache.org/jira/browse/HUDI-4523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alexander Trushev updated HUDI-4523:
------------------------------------
Status: Open (was: In Progress)
> Sequential submitting of flink jobs leads to java.net.ConnectException
> ----------------------------------------------------------------------
>
> Key: HUDI-4523
> URL: https://issues.apache.org/jira/browse/HUDI-4523
> Project: Apache Hudi
> Issue Type: Bug
> Components: flink-sql
> Reporter: Alexander Trushev
> Assignee: Alexander Trushev
> Priority: Major
> Labels: pull-request-available
>
> h3. How to reproduce
> Flink job:
> {code:java}
> write.batch.size: 0.00000000001
> metadata.enabled: true
> hoodie.filesystem.remote.backup.view.enable: false
> tEnv.executeSql("insert into t1 values (0)").await();
> tEnv.executeSql("insert into t1 values (1), (2)").await(); // it is important that there is more than one value
> {code}
> Note: this is a concurrent bug. To reproduce you need to add
> {code:java}
> public class EmbeddedTimelineService {
> ...
> public void stop() {
> if (null != server) {
> LOG.info("Closing Timeline server");
> this.server.close();
> Thread.sleep(1000); <------------------------------
> this.server = null;
> this.viewManager = null;
> LOG.info("Closed Timeline server");
> }
> }
> }
> {code}
> Result:
> {code:java}
> Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 192.168.0.5:58517 [/192.168.0.5] failed: Connection refused (Connection refused)
> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
> at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
> at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
> at org.apache.http.client.fluent.Request.execute(Request.java:151)
> at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:172)
> at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestBaseFilesFromParams(RemoteHoodieTableFileSystemView.java:237)
> ... 22 more
> Caused by: java.net.ConnectException: Connection refused (Connection refused)
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:613)
> at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
> at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
> ... 35 more
> {code}
> When hoodie.filesystem.remote.backup.view.enable is true it is an annoying error in log while switching the option to false leads to job failure with exception
> h3. Why do we need to fix it
> # When metadata.enabled: false the code snippet works fine
> # Even though hoodie.filesystem.remote.backup.view.enable: false is debug option the error java.net.ConnectException is confusing in log
--
This message was sent by Atlassian Jira
(v8.20.10#820010)