[jira] [Created] (HUDI-4523) Sequential submitting of flink jobs leads to java.net.ConnectException

["Alexander Trushev (Jira)" <ji...@apache.org>]

Alexander Trushev created HUDI-4523:
---------------------------------------

             Summary: Sequential submitting of flink jobs leads to java.net.ConnectException
                 Key: HUDI-4523
                 URL: https://issues.apache.org/jira/browse/HUDI-4523
             Project: Apache Hudi
          Issue Type: Bug
            Reporter: Alexander Trushev
            Assignee: Alexander Trushev


h3. How to reproduce

{code:java}
metadata.enabled: true
hoodie.filesystem.remote.backup.view.enable: false

tEnv.executeSql("insert into t1 values (0)").await();
tEnv.executeSql("insert into t1 values (1), (2)").await(); // it is important that there is more than one value
{code}

Note: this is a conccurent bug. To reproduce you need to add

{code:java}
public class EmbeddedTimelineService {

...

  public void stop() {  
    if (null != server) {
      LOG.info("Closing Timeline server");
      this.server.close();
      Thread.sleep(1000); <------------------------------
      this.server = null;
      this.viewManager = null;
      LOG.info("Closed Timeline server");
    }
  }
}
{code}

Result:

{code}
Caused by: org.apache.http.conn.HttpHostConnectException: Connect to 192.168.0.5:58517 [/192.168.0.5] failed: Connection refused (Connection refused)
  at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:151)
  at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
  at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
  at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
  at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
  at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
  at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
  at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
  at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
  at org.apache.http.client.fluent.Request.execute(Request.java:151)
  at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.executeRequest(RemoteHoodieTableFileSystemView.java:172)
  at org.apache.hudi.common.table.view.RemoteHoodieTableFileSystemView.getLatestBaseFilesFromParams(RemoteHoodieTableFileSystemView.java:237)
  ... 22 more
Caused by: java.net.ConnectException: Connection refused (Connection refused)
  at java.net.PlainSocketImpl.socketConnect(Native Method)
  at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
  at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
  at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
  at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
  at java.net.Socket.connect(Socket.java:613)
  at org.apache.http.conn.socket.PlainConnectionSocketFactory.connectSocket(PlainConnectionSocketFactory.java:74)
  at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
  ... 35 more
{code}


When hoodie.filesystem.remote.backup.view.enable is true it is an annoying error in log while switching the option to false leads to job failure with exception

h3. Why do we need to fix it

# When metadata.enabled: false the code snippet works fine
# Even though hoodie.filesystem.remote.backup.view.enable: false is debug option the error java.net.ConnectException is confusing in log






--
This message was sent by Atlassian Jira
(v8.20.10#820010)