You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2024/03/08 02:19:38 UTC
(kyuubi) branch master updated: [KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 182f4349a [KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
182f4349a is described below
commit 182f4349a744480c189bdb1646a3be8ffed0ede1
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Fri Mar 8 10:19:30 2024 +0800
[KYUUBI #6117] Bump Kyuubi Shaded 0.3.0
# :mag: Description
## Issue References ๐
This PR upgrades Kyuubi Shaded to 0.3.0, the major changes are:
- upgrade Thrift from 0.9.3-1 to 0.16.0
- upgrade JLine used by Zookeeper CLI from 0.9.94 to 2.14.6
- split kyuubi-relocated-thrift from kyuubi-relocated-hive-service-rpc
- introduce a light kyuubi-relocated-hive-metastore-client for getting DelegationToken
Due to THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength, the following Kyuubi configurations are removed too.
- kyuubi.frontend.login.timeout
- kyuubi.frontend.thrift.login.timeout
- kyuubi.frontend.backoff.slot.length
- kyuubi.frontend.thrift.backoff.slot.length
Previously, Zookeeper 3.4 depends on JLine 0.9.94 while Zookeeper 3.6 and Hive Beeline depends on JLine 2.14.6, we pulls different versions of JLine into the binary distribution. Now we align them to a single JLine 2.14.6.
## Describe Your Solution ๐ง
Upgrade dependencies and change code to adapt to the breaking changes, also upgrade migration guide to mention the removed configurations.
## Types of changes :bookmark:
- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
## Test Plan ๐งช
Pass GA.
---
# Checklist ๐
- [x] This patch was not authored or co-authored using [Generative Tooling](https://www.apache.org/legal/generative-tooling.html)
**Be nice. Be informative.**
Closes #6117 from pan3793/shaded-0.3.
Closes #6117
5639c9fd8 [Cheng Pan] nit
d281cdb4b [Cheng Pan] fix
184e1b95b [Cheng Pan] fix
1628337c8 [Cheng Pan] remove unused conf
24db2d5ad [Cheng Pan] try fix
1e995bb34 [Cheng Pan] nit
1d8e9bce9 [Cheng Pan] dep list
7f33624e8 [Cheng Pan] Bump Kyuubi Shaded 0.3.0
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Cheng Pan <ch...@apache.org>
---
dev/dependencyList | 2 +-
docs/configuration/settings.md | 4 ---
docs/deployment/migration-guide.md | 1 +
kyuubi-common/pom.xml | 5 +++
.../org/apache/kyuubi/config/KyuubiConf.scala | 36 ----------------------
.../kyuubi/service/TBinaryFrontendService.scala | 6 +---
.../apache/kyuubi/service/TFrontendService.scala | 4 +++
.../HadoopThriftAuthBridgeServer.scala | 2 +-
.../authentication/TSetIpAddressProcessor.scala | 2 +-
kyuubi-hive-beeline/pom.xml | 7 -----
kyuubi-hive-jdbc/pom.xml | 5 +++
.../kyuubi/jdbc/hive/auth/KerberosSaslHelper.java | 3 +-
.../kyuubi/jdbc/hive/auth/PlainSaslHelper.java | 4 ++-
.../kyuubi/jdbc/hive/auth/TFilterTransport.java | 16 ++++++++++
.../apache/kyuubi/jdbc/hive/auth/ThriftUtils.java | 5 +--
.../kyuubi/client/KyuubiSyncThriftClient.scala | 3 +-
.../KyuubiOperationPerConnectionSuite.scala | 9 ++++--
pom.xml | 10 ++++--
18 files changed, 58 insertions(+), 66 deletions(-)
diff --git a/dev/dependencyList b/dev/dependencyList
index 88e3e3c9f..4143b511a 100644
--- a/dev/dependencyList
+++ b/dev/dependencyList
@@ -105,7 +105,7 @@ jetty-server/9.4.52.v20230823//jetty-server-9.4.52.v20230823.jar
jetty-servlet/9.4.52.v20230823//jetty-servlet-9.4.52.v20230823.jar
jetty-util-ajax/9.4.52.v20230823//jetty-util-ajax-9.4.52.v20230823.jar
jetty-util/9.4.52.v20230823//jetty-util-9.4.52.v20230823.jar
-jline/0.9.94//jline-0.9.94.jar
+jline/2.14.6//jline-2.14.6.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
kafka-clients/3.5.1//kafka-clients-3.5.1.jar
kubernetes-client-api/6.8.1//kubernetes-client-api-6.8.1.jar
diff --git a/docs/configuration/settings.md b/docs/configuration/settings.md
index a5af4e180..66b551166 100644
--- a/docs/configuration/settings.md
+++ b/docs/configuration/settings.md
@@ -230,11 +230,9 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| Key | Default | Meaning [...]
|--------------------------------------------------------|--------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
| kyuubi.frontend.advertised.host | <undefined> | Hostname or IP of the Kyuubi server's frontend services to publish to external systems such as the service discovery ensemble and metadata store. Use it when you want to advertise a different hostname or IP than the bind host. [...]
-| kyuubi.frontend.backoff.slot.length | PT0.1S | (deprecated) Time to back off during login to the thrift frontend service. [...]
| kyuubi.frontend.bind.host | <undefined> | Hostname or IP of the machine on which to run the frontend services. [...]
| kyuubi.frontend.bind.port | 10009 | (deprecated) Port of the machine on which to run the thrift frontend service via the binary protocol. [...]
| kyuubi.frontend.connection.url.use.hostname | true | When true, frontend services prefer hostname, otherwise, ip address. Note that, the default value is set to `false` when engine running on Kubernetes to prevent potential network issues. [...]
-| kyuubi.frontend.login.timeout | PT20S | (deprecated) Timeout for Thrift clients during login to the thrift frontend service. [...]
| kyuubi.frontend.max.message.size | 104857600 | (deprecated) Maximum message size in bytes a Kyuubi server will accept. [...]
| kyuubi.frontend.max.worker.threads | 999 | (deprecated) Maximum number of threads in the frontend worker thread pool for the thrift frontend service [...]
| kyuubi.frontend.min.worker.threads | 9 | (deprecated) Minimum number of threads in the frontend worker thread pool for the thrift frontend service [...]
@@ -254,7 +252,6 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.frontend.ssl.keystore.password | <undefined> | SSL certificate keystore password. [...]
| kyuubi.frontend.ssl.keystore.path | <undefined> | SSL certificate keystore location. [...]
| kyuubi.frontend.ssl.keystore.type | <undefined> | SSL certificate keystore type. [...]
-| kyuubi.frontend.thrift.backoff.slot.length | PT0.1S | Time to back off during login to the thrift frontend service. [...]
| kyuubi.frontend.thrift.binary.bind.host | <undefined> | Hostname or IP of the machine on which to run the thrift frontend service via the binary protocol. [...]
| kyuubi.frontend.thrift.binary.bind.port | 10009 | Port of the machine on which to run the thrift frontend service via the binary protocol. [...]
| kyuubi.frontend.thrift.binary.ssl.disallowed.protocols | SSLv2,SSLv3 | SSL versions to disallow for Kyuubi thrift binary frontend. [...]
@@ -278,7 +275,6 @@ You can configure the Kyuubi properties in `$KYUUBI_HOME/conf/kyuubi-defaults.co
| kyuubi.frontend.thrift.http.ssl.protocol.blacklist | SSLv2,SSLv3 | SSL Versions to disable when using HTTP transport mode. [...]
| kyuubi.frontend.thrift.http.use.SSL | false | Set this to true for using SSL encryption in http mode. [...]
| kyuubi.frontend.thrift.http.xsrf.filter.enabled | false | If enabled, Kyuubi will block any requests made to it over HTTP if an X-XSRF-HEADER header is not present [...]
-| kyuubi.frontend.thrift.login.timeout | PT20S | Timeout for Thrift clients during login to the thrift frontend service. [...]
| kyuubi.frontend.thrift.max.message.size | 104857600 | Maximum message size in bytes a Kyuubi server will accept. [...]
| kyuubi.frontend.thrift.max.worker.threads | 999 | Maximum number of threads in the frontend worker thread pool for the thrift frontend service [...]
| kyuubi.frontend.thrift.min.worker.threads | 9 | Minimum number of threads in the frontend worker thread pool for the thrift frontend service [...]
diff --git a/docs/deployment/migration-guide.md b/docs/deployment/migration-guide.md
index 3b3541b7e..cbd74ee44 100644
--- a/docs/deployment/migration-guide.md
+++ b/docs/deployment/migration-guide.md
@@ -23,6 +23,7 @@
* Since Kyuubi 1.9.0, the support of Derby is removal for Kyuubi metastore.
* Since Kyuubi 1.9.0, the support of Spark SQL engine for Spark 3.1 is deprecated, and will be removed in the future.
* Since Kyuubi 1.9.0, the support of Spark extensions for Spark 3.1 is removed, please use Spark 3.2 or higher versions.
+* Since Kyuubi 1.9.0, `kyuubi.frontend.login.timeout`, `kyuubi.frontend.thrift.login.timeout`, `kyuubi.frontend.backoff.slot.length`, `kyuubi.frontend.thrift.backoff.slot.length` are removed.
## Upgrading from Kyuubi 1.8.0 to 1.8.1
diff --git a/kyuubi-common/pom.xml b/kyuubi-common/pom.xml
index 63ac03ed7..c36eb2f46 100644
--- a/kyuubi-common/pom.xml
+++ b/kyuubi-common/pom.xml
@@ -98,6 +98,11 @@
<artifactId>commons-lang3</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-relocated-thrift</artifactId>
+ </dependency>
+
<dependency>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
index 5346d19a2..6baca63e5 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/config/KyuubiConf.scala
@@ -605,34 +605,6 @@ object KyuubiConf {
.version("1.4.0")
.fallbackConf(FRONTEND_MAX_MESSAGE_SIZE)
- @deprecated("using kyuubi.frontend.thrift.login.timeout instead", "1.4.0")
- val FRONTEND_LOGIN_TIMEOUT: ConfigEntry[Long] =
- buildConf("kyuubi.frontend.login.timeout")
- .doc("(deprecated) Timeout for Thrift clients during login to the thrift frontend service.")
- .version("1.0.0")
- .timeConf
- .createWithDefault(Duration.ofSeconds(20).toMillis)
-
- val FRONTEND_THRIFT_LOGIN_TIMEOUT: ConfigEntry[Long] =
- buildConf("kyuubi.frontend.thrift.login.timeout")
- .doc("Timeout for Thrift clients during login to the thrift frontend service.")
- .version("1.4.0")
- .fallbackConf(FRONTEND_LOGIN_TIMEOUT)
-
- @deprecated("using kyuubi.frontend.thrift.backoff.slot.length instead", "1.4.0")
- val FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH: ConfigEntry[Long] =
- buildConf("kyuubi.frontend.backoff.slot.length")
- .doc("(deprecated) Time to back off during login to the thrift frontend service.")
- .version("1.0.0")
- .timeConf
- .createWithDefault(Duration.ofMillis(100).toMillis)
-
- val FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH: ConfigEntry[Long] =
- buildConf("kyuubi.frontend.thrift.backoff.slot.length")
- .doc("Time to back off during login to the thrift frontend service.")
- .version("1.4.0")
- .fallbackConf(FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH)
-
val FRONTEND_THRIFT_HTTP_REQUEST_HEADER_SIZE: ConfigEntry[Int] =
buildConf("kyuubi.frontend.thrift.http.request.header.size")
.doc("Request header size in bytes, when using HTTP transport mode. Jetty defaults used.")
@@ -3144,14 +3116,6 @@ object KyuubiConf {
FRONTEND_MAX_MESSAGE_SIZE.key,
"1.4.0",
s"Use ${FRONTEND_THRIFT_MAX_MESSAGE_SIZE.key} instead"),
- DeprecatedConfig(
- FRONTEND_LOGIN_TIMEOUT.key,
- "1.4.0",
- s"Use ${FRONTEND_THRIFT_LOGIN_TIMEOUT.key} instead"),
- DeprecatedConfig(
- FRONTEND_LOGIN_BACKOFF_SLOT_LENGTH.key,
- "1.4.0",
- s"Use ${FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH.key} instead"),
DeprecatedConfig(
SESSION_TIMEOUT.key,
"1.2.0",
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TBinaryFrontendService.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TBinaryFrontendService.scala
index 19e2e31ea..92b3a8a81 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TBinaryFrontendService.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TBinaryFrontendService.scala
@@ -101,17 +101,13 @@ abstract class TBinaryFrontendService(name: String)
}
_actualPort = tServerSocket.getServerSocket.getLocalPort
val maxMessageSize = conf.get(FRONTEND_THRIFT_MAX_MESSAGE_SIZE)
- val requestTimeout = conf.get(FRONTEND_THRIFT_LOGIN_TIMEOUT).toInt
- val beBackoffSlotLength = conf.get(FRONTEND_THRIFT_LOGIN_BACKOFF_SLOT_LENGTH).toInt
val args = new TThreadPoolServer.Args(tServerSocket)
.processorFactory(tProcFactory)
.transportFactory(transFactory)
.protocolFactory(new TBinaryProtocol.Factory)
.inputProtocolFactory(
new TBinaryProtocol.Factory(true, true, maxMessageSize, maxMessageSize))
- .requestTimeout(requestTimeout).requestTimeoutUnit(TimeUnit.MILLISECONDS)
- .beBackoffSlotLength(beBackoffSlotLength)
- .beBackoffSlotLengthUnit(TimeUnit.MILLISECONDS)
+ // THRIFT-5297 (fixed in 0.14.0) removes requestTimeout and beBackoffSlotLength
.executorService(executor)
// TCP Server
server = Some(new TThreadPoolServer(args))
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
index 9aefe63c8..32e004505 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/TFrontendService.scala
@@ -649,5 +649,9 @@ private[kyuubi] object TFrontendService {
}
def getSessionHandle: SessionHandle = sessionHandle
+
+ override def unwrap[T](aClass: Class[T]): T = null.asInstanceOf[T]
+
+ override def isWrapperFor(aClass: Class[_]): Boolean = false
}
}
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/HadoopThriftAuthBridgeServer.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/HadoopThriftAuthBridgeServer.scala
index 6c1dfa5da..845767c88 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/HadoopThriftAuthBridgeServer.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/HadoopThriftAuthBridgeServer.scala
@@ -131,7 +131,7 @@ object HadoopThriftAuthBridgeServer {
class TUGIAssumingProcessor(
wrapped: TProcessor,
secretMgr: KyuubiDelegationTokenManager) extends TProcessor with Logging {
- override def process(in: TProtocol, out: TProtocol): Boolean = {
+ override def process(in: TProtocol, out: TProtocol): Unit = {
val transport = in.getTransport
transport match {
case saslTrans: TSaslServerTransport =>
diff --git a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/TSetIpAddressProcessor.scala b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/TSetIpAddressProcessor.scala
index 6a8905936..3c632b8bf 100644
--- a/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/TSetIpAddressProcessor.scala
+++ b/kyuubi-common/src/main/scala/org/apache/kyuubi/service/authentication/TSetIpAddressProcessor.scala
@@ -28,7 +28,7 @@ class TSetIpAddressProcessor[I <: Iface](
import TSetIpAddressProcessor._
@throws[TException]
- override def process(in: TProtocol, out: TProtocol): Boolean = {
+ override def process(in: TProtocol, out: TProtocol): Unit = {
setIpAddress(in)
setUserName(in)
try {
diff --git a/kyuubi-hive-beeline/pom.xml b/kyuubi-hive-beeline/pom.xml
index 1068a81ce..77337adcd 100644
--- a/kyuubi-hive-beeline/pom.xml
+++ b/kyuubi-hive-beeline/pom.xml
@@ -135,13 +135,6 @@
<dependency>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
- <version>${hive.client.jline.version}</version>
- <exclusions>
- <exclusion>
- <groupId>*</groupId>
- <artifactId>*</artifactId>
- </exclusion>
- </exclusions>
</dependency>
<dependency>
diff --git a/kyuubi-hive-jdbc/pom.xml b/kyuubi-hive-jdbc/pom.xml
index 1ec5e597f..2bc7f690b 100644
--- a/kyuubi-hive-jdbc/pom.xml
+++ b/kyuubi-hive-jdbc/pom.xml
@@ -97,6 +97,11 @@
<artifactId>httpclient</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-relocated-thrift</artifactId>
+ </dependency>
+
<dependency>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>
diff --git a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/KerberosSaslHelper.java b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/KerberosSaslHelper.java
index e3fb67293..bd0dfcdf8 100644
--- a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/KerberosSaslHelper.java
+++ b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/KerberosSaslHelper.java
@@ -22,6 +22,7 @@ import javax.security.auth.Subject;
import javax.security.sasl.SaslException;
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
+import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -34,7 +35,7 @@ public final class KerberosSaslHelper {
String host,
TTransport underlyingTransport,
Map<String, String> saslProps)
- throws SaslException {
+ throws SaslException, TTransportException {
String resolvedPrincipal = KerberosUtils.canonicalPrincipal(serverPrincipal, host);
String[] names = KerberosUtils.splitPrincipal(resolvedPrincipal);
TTransport saslTransport =
diff --git a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/PlainSaslHelper.java b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/PlainSaslHelper.java
index 43272e48d..95aae29fd 100644
--- a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/PlainSaslHelper.java
+++ b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/PlainSaslHelper.java
@@ -22,11 +22,13 @@ import javax.security.auth.callback.*;
import javax.security.sasl.SaslException;
import org.apache.kyuubi.shaded.thrift.transport.TSaslClientTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
+import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
public final class PlainSaslHelper {
public static TTransport getPlainTransport(
- String username, String password, TTransport underlyingTransport) throws SaslException {
+ String username, String password, TTransport underlyingTransport)
+ throws SaslException, TTransportException {
return new TSaslClientTransport(
"PLAIN",
null,
diff --git a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/TFilterTransport.java b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/TFilterTransport.java
index 6d462717b..f3691e8cf 100644
--- a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/TFilterTransport.java
+++ b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/TFilterTransport.java
@@ -17,6 +17,7 @@
package org.apache.kyuubi.jdbc.hive.auth;
+import org.apache.kyuubi.shaded.thrift.TConfiguration;
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
@@ -95,4 +96,19 @@ public class TFilterTransport extends TTransport {
public void consumeBuffer(int len) {
wrapped.consumeBuffer(len);
}
+
+ @Override
+ public TConfiguration getConfiguration() {
+ return wrapped.getConfiguration();
+ }
+
+ @Override
+ public void updateKnownMessageSize(long l) throws TTransportException {
+ wrapped.updateKnownMessageSize(l);
+ }
+
+ @Override
+ public void checkReadBytesAvailable(long l) throws TTransportException {
+ wrapped.checkReadBytesAvailable(l);
+ }
}
diff --git a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/ThriftUtils.java b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/ThriftUtils.java
index b76401d50..7f0099b29 100644
--- a/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/ThriftUtils.java
+++ b/kyuubi-hive-jdbc/src/main/java/org/apache/kyuubi/jdbc/hive/auth/ThriftUtils.java
@@ -19,6 +19,7 @@ package org.apache.kyuubi.jdbc.hive.auth;
import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket;
+import org.apache.kyuubi.shaded.thrift.TConfiguration;
import org.apache.kyuubi.shaded.thrift.transport.TSSLTransportFactory;
import org.apache.kyuubi.shaded.thrift.transport.TSocket;
import org.apache.kyuubi.shaded.thrift.transport.TTransport;
@@ -30,8 +31,8 @@ import org.apache.kyuubi.shaded.thrift.transport.TTransportException;
*/
public class ThriftUtils {
public static TTransport getSocketTransport(
- String host, int port, int connectTimeout, int socketTimeout) {
- return new TSocket(host, port, socketTimeout, connectTimeout);
+ String host, int port, int connectTimeout, int socketTimeout) throws TTransportException {
+ return new TSocket(TConfiguration.DEFAULT, host, port, socketTimeout, connectTimeout);
}
public static TTransport getSSLSocket(
diff --git a/kyuubi-server/src/main/scala/org/apache/kyuubi/client/KyuubiSyncThriftClient.scala b/kyuubi-server/src/main/scala/org/apache/kyuubi/client/KyuubiSyncThriftClient.scala
index d24387341..755e55d6d 100644
--- a/kyuubi-server/src/main/scala/org/apache/kyuubi/client/KyuubiSyncThriftClient.scala
+++ b/kyuubi-server/src/main/scala/org/apache/kyuubi/client/KyuubiSyncThriftClient.scala
@@ -36,6 +36,7 @@ import org.apache.kyuubi.operation.FetchOrientation.FetchOrientation
import org.apache.kyuubi.service.authentication.PlainSASLHelper
import org.apache.kyuubi.session.SessionHandle
import org.apache.kyuubi.shaded.hive.service.rpc.thrift._
+import org.apache.kyuubi.shaded.thrift.TConfiguration
import org.apache.kyuubi.shaded.thrift.protocol.{TBinaryProtocol, TProtocol}
import org.apache.kyuubi.shaded.thrift.transport.TSocket
import org.apache.kyuubi.util.{ThreadUtils, ThriftUtils}
@@ -457,7 +458,7 @@ private[kyuubi] object KyuubiSyncThriftClient extends Logging {
port: Int,
socketTimeout: Int,
connectionTimeout: Int): TProtocol = {
- val tSocket = new TSocket(host, port, socketTimeout, connectionTimeout)
+ val tSocket = new TSocket(TConfiguration.DEFAULT, host, port, socketTimeout, connectionTimeout)
val tTransport = PlainSASLHelper.getPlainTransport(user, passwd, tSocket)
tTransport.open()
new TBinaryProtocol(tTransport)
diff --git a/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiOperationPerConnectionSuite.scala b/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiOperationPerConnectionSuite.scala
index 1324c70d7..859ccac98 100644
--- a/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiOperationPerConnectionSuite.scala
+++ b/kyuubi-server/src/test/scala/org/apache/kyuubi/operation/KyuubiOperationPerConnectionSuite.scala
@@ -80,7 +80,9 @@ class KyuubiOperationPerConnectionSuite extends WithKyuubiServer with HiveJDBCTe
assert(executeStmtResp.getOperationHandle === null)
val errMsg = executeStmtResp.getStatus.getErrorMessage
assert(errMsg.contains("Caused by: java.net.SocketException: Connection reset") ||
- errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed"))
+ errMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
+ errMsg.contains("Socket is closed by peer") ||
+ errMsg.contains("SparkContext was shut down"))
}
}
@@ -332,9 +334,10 @@ class KyuubiOperationPerConnectionSuite extends WithKyuubiServer with HiveJDBCTe
assert(executeStmtResp.getStatus.getStatusCode === TStatusCode.ERROR_STATUS)
val errorMsg = executeStmtResp.getStatus.getErrorMessage
assert(errorMsg.contains("java.net.SocketException") ||
- errorMsg.contains("org.apache.thrift.transport.TTransportException") ||
+ errorMsg.contains("org.apache.kyuubi.shaded.thrift.transport.TTransportException") ||
errorMsg.contains("connection does not exist") ||
- errorMsg.contains(s"Socket for ${SessionHandle(handle)} is closed"))
+ errorMsg.contains(s"Socket for ${SessionHandle(handle)} is closed") ||
+ errorMsg.contains("Error submitting query in background, query rejected"))
val elapsedTime = System.currentTimeMillis() - startTime
assert(elapsedTime < 20 * 1000)
eventually(timeout(3.seconds)) {
diff --git a/pom.xml b/pom.xml
index b70d5310a..8f3c0baaf 100644
--- a/pom.xml
+++ b/pom.xml
@@ -169,11 +169,11 @@
<jersey.version>2.39.1</jersey.version>
<jetcd.version>0.7.7</jetcd.version>
<jetty.version>9.4.52.v20230823</jetty.version>
- <jline.version>0.9.94</jline.version>
+ <jline.version>2.14.6</jline.version>
<junit.version>4.13.2</junit.version>
<kafka.version>3.5.1</kafka.version>
<kubernetes-client.version>6.8.1</kubernetes-client.version>
- <kyuubi-relocated.version>0.2.0</kyuubi-relocated.version>
+ <kyuubi-relocated.version>0.3.0</kyuubi-relocated.version>
<kyuubi-relocated-zookeeper.artifacts>kyuubi-relocated-zookeeper-34</kyuubi-relocated-zookeeper.artifacts>
<ldapsdk.version>6.0.5</ldapsdk.version>
<log4j.version>2.20.0</log4j.version>
@@ -223,7 +223,6 @@
<pnpm.version>v8.6.1</pnpm.version>
<!-- apply to kyuubi-hive-jdbc/kyuubi-hive-beeline module -->
- <hive.client.jline.version>2.12</hive.client.jline.version>
<hive.client.supercsv.version>2.2.0</hive.client.supercsv.version>
<hive.jdbc.artifact>kyuubi-hive-jdbc</hive.jdbc.artifact>
@@ -279,6 +278,11 @@
<dependencyManagement>
<dependencies>
+ <dependency>
+ <groupId>org.apache.kyuubi</groupId>
+ <artifactId>kyuubi-relocated-thrift</artifactId>
+ <version>${kyuubi-relocated.version}</version>
+ </dependency>
<dependency>
<groupId>org.apache.kyuubi</groupId>
<artifactId>kyuubi-relocated-hive-service-rpc</artifactId>