You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2022/11/11 02:05:38 UTC
[incubator-kyuubi] branch master updated: [KYUUBI #3776] [BUILD] Revise Kyuubi Spark engine shaded
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 72c1f53dd [KYUUBI #3776] [BUILD] Revise Kyuubi Spark engine shaded
72c1f53dd is described below
commit 72c1f53dd0c4a8eafb9ee2b13c77240b711cb58d
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Fri Nov 11 10:05:23 2022 +0800
[KYUUBI #3776] [BUILD] Revise Kyuubi Spark engine shaded
### _Why are the changes needed?_
This PR revises the shaded rule of spark engine module, especially to make sure that netty native libs is shaded properly.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```
(kyuubi) ➜ apache-kyuubi git:(spark-engine-shade) tree externals/kyuubi-spark-sql-engine/target/unpacked -L 5 | grep -v class
externals/kyuubi-spark-sql-engine/target/unpacked
├── META-INF
│ ├── LICENSE
│ ├── MANIFEST.MF
│ ├── NOTICE
│ ├── io.netty.versions.properties
│ ├── native
│ │ ├── liborg_apache_kyuubi_shade_netty_transport_native_epoll_aarch_64.so
│ │ └── liborg_apache_kyuubi_shade_netty_transport_native_epoll_x86_64.so
│ ├── services
│ │ ├── org.apache.kyuubi.shade.io.grpc.LoadBalancerProvider
│ │ ├── org.apache.kyuubi.shade.io.grpc.ManagedChannelProvider
│ │ ├── org.apache.kyuubi.shade.io.grpc.NameResolverProvider
│ │ ├── org.apache.kyuubi.shade.io.grpc.ServerProvider
│ │ ├── org.apache.kyuubi.shade.io.vertx.core.spi.launcher.CommandFactory
│ │ ├── org.apache.spark.status.AppHistoryServerPlugin
│ │ └── reactor.blockhound.integration.BlockHoundIntegration
│ ├── versions
│ │ └── 11
│ │ └── io
│ │ └── vertx
│ └── vertx
│ └── vertx-version.txt
├── kyuubi-version-info.properties
├── log4j2-defaults.xml
├── org
│ └── apache
│ ├── kyuubi
│ │ ├── cli
│ │ ├── config
│ │ │ └── internal
│ │ ├── engine
│ │ │ └── spark
│ │ ├── events
│ │ │ └── handler
│ │ ├── ha
│ │ │ └── client
│ │ ├── operation
│ │ │ ├── log
│ │ │ └── meta
│ │ ├── reflection
│ │ ├── service
│ │ │ └── authentication
│ │ ├── session
│ │ ├── shade
│ │ │ ├── android
│ │ │ ├── com
│ │ │ ├── io
│ │ │ ├── net
│ │ │ └── org
│ │ └── util
│ └── spark
│ ├── api
│ │ └── python
│ ├── kyuubi
│ └── ui
└── python
├── execute_python.py
└── kyuubi_util.py
40 directories, 211 files
```
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes #3776 from pan3793/spark-engine-shade.
Closes #3776
6a6e2a64 [Cheng Pan] nit
e247923a [Cheng Pan] 1
a53b7c06 [Cheng Pan] nit
19382ef3 [Cheng Pan] [BUILD] Revisit Kyuubi Spark engine shaded
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Cheng Pan <ch...@apache.org>
---
LICENSE-binary | 1 -
externals/kyuubi-spark-sql-engine/pom.xml | 141 ++++++++++-
.../src/main/resources/META-INF/LICENSE | 43 +++-
.../src/main/resources/META-INF/NOTICE | 268 +--------------------
.../org.apache.spark.status.AppHistoryServerPlugin | 19 +-
kyuubi-ha/pom.xml | 12 +
pom.xml | 25 +-
7 files changed, 215 insertions(+), 294 deletions(-)
diff --git a/LICENSE-binary b/LICENSE-binary
index bfaee55ba..03c301011 100644
--- a/LICENSE-binary
+++ b/LICENSE-binary
@@ -296,7 +296,6 @@ io.netty:netty-handler
io.netty:netty-resolver-dns
io.netty:netty-resolver
io.netty:netty-transport-classes-epoll
-io.netty:netty-transport-classes-kqueue
io.netty:netty-transport-native-epoll
io.netty:netty-transport-native-unix-common
io.netty:netty-transport
diff --git a/externals/kyuubi-spark-sql-engine/pom.xml b/externals/kyuubi-spark-sql-engine/pom.xml
index 90034933d..55f0e80e0 100644
--- a/externals/kyuubi-spark-sql-engine/pom.xml
+++ b/externals/kyuubi-spark-sql-engine/pom.xml
@@ -208,7 +208,12 @@
<shadedArtifactAttached>false</shadedArtifactAttached>
<artifactSet>
<includes>
+ <include>com.google.android:annotations</include>
+ <include>com.google.api.grpc:proto-google-common-protos</include>
+ <include>com.google.code.gson:gson</include>
+ <include>com.google.errorprone:error_prone_annotations</include>
<include>com.google.guava:*</include>
+ <include>com.google.j2objc:j2objc-annotations</include>
<include>com.google.protobuf:*</include>
<include>io.etcd:*</include>
<include>io.grpc:*</include>
@@ -216,21 +221,24 @@
<include>io.perfmark:perfmark-api</include>
<include>io.vertx:*</include>
<include>net.jodah:failsafe</include>
- <include>org.apache.kyuubi:kyuubi-common_${scala.binary.version}</include>
- <include>org.apache.kyuubi:kyuubi-events_${scala.binary.version}</include>
- <include>org.apache.kyuubi:kyuubi-ha_${scala.binary.version}</include>
<include>org.apache.curator:curator-client</include>
<include>org.apache.curator:curator-framework</include>
<include>org.apache.curator:curator-recipes</include>
- <include>org.apache.zookeeper:zookeeper</include>
<include>org.apache.hive:hive-service-rpc</include>
+ <include>org.apache.kyuubi:kyuubi-common_${scala.binary.version}</include>
+ <include>org.apache.kyuubi:kyuubi-events_${scala.binary.version}</include>
+ <include>org.apache.kyuubi:kyuubi-ha_${scala.binary.version}</include>
<include>org.apache.thrift:*</include>
+ <include>org.apache.zookeeper:zookeeper</include>
+ <include>org.checkerframework:checker-qual</include>
+ <include>org.codehaus.mojo:animal-sniffer-annotations</include>
</includes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
+ <exclude>**/*.proto</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
@@ -289,12 +297,16 @@
</includes>
</relocation>
<relocation>
- <pattern>com.google</pattern>
- <shadedPattern>${kyuubi.shade.packageName}.com.google</shadedPattern>
+ <pattern>io.etcd</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.io.etcd</shadedPattern>
<includes>
- <include>com.google.**</include>
+ <include>io.etcd.**</include>
</includes>
</relocation>
+ <relocation>
+ <pattern>io.grpc</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.io.grpc</shadedPattern>
+ </relocation>
<relocation>
<pattern>io.netty</pattern>
<shadedPattern>${kyuubi.shade.packageName}.io.netty</shadedPattern>
@@ -302,6 +314,10 @@
<include>io.netty.**</include>
</includes>
</relocation>
+ <relocation>
+ <pattern>io.perfmark</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.io.perfmark</shadedPattern>
+ </relocation>
<relocation>
<pattern>io.vertx</pattern>
<shadedPattern>${kyuubi.shade.packageName}.io.vertx</shadedPattern>
@@ -317,12 +333,83 @@
</includes>
</relocation>
<relocation>
- <pattern>io.perfmark</pattern>
- <shadedPattern>${kyuubi.shade.packageName}.io.perfmark</shadedPattern>
+ <pattern>android.annotation</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.android.annotation</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.common</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.common</shadedPattern>
+ <includes>
+ <include>com.google.common.**</include>
+ </includes>
+ </relocation>
+ <relocation>
+ <pattern>com.google.thirdparty</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.thirdparty</shadedPattern>
<includes>
- <include>io.perfmark.**</include>
+ <include>com.google.thirdparty.**</include>
</includes>
</relocation>
+ <relocation>
+ <pattern>com.google.protobuf</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.protobuf</shadedPattern>
+ <includes>
+ <include>com.google.protobuf.**</include>
+ </includes>
+ </relocation>
+ <relocation>
+ <pattern>org.codehaus.mojo.animal_sniffer</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.org.codehaus.mojo.animal_sniffer</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.j2objc.annotations</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.j2objc.annotations</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.errorprone.annotations</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.errorprone.annotations</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>org.checkerframework</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.org.checkerframework</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.gson</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.gson</shadedPattern>
+ </relocation>
+ <!--
+ For `com.google.api.grpc:proto-google-common-protos`, do not directly define pattern
+ as `common.google`, otherwise, otherwise, the relocation result may be uncertain due
+ to the change of rule order.
+ -->
+ <relocation>
+ <pattern>com.google.api</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.api</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.cloud</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.cloud</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.geo</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.geo</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.logging</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.logging</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.longrunning</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.longrunning</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.rpc</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.rpc</shadedPattern>
+ </relocation>
+ <relocation>
+ <pattern>com.google.type</pattern>
+ <shadedPattern>${kyuubi.shade.packageName}.com.google.type</shadedPattern>
+ </relocation>
</relocations>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
@@ -338,6 +425,40 @@
</executions>
</plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>rename-native-library</id>
+ <phase>package</phase>
+ <configuration>
+ <target>
+ <echo message="unpacking netty jar"/>
+ <unzip src="${project.build.directory}/${project.artifactId}-${project.version}.jar"
+ dest="${project.build.directory}/unpacked/"/>
+ <echo message="renaming netty native libraries"/>
+ <move includeemptydirs="false"
+ todir="${project.build.directory}/unpacked/META-INF/native">
+ <fileset dir="${project.build.directory}/unpacked/META-INF/native"/>
+ <mapper type="regexp"
+ from="libnetty_transport_native_epoll_(aarch_64.so|x86_64.so)"
+ to="liborg_apache_kyuubi_shade_netty_transport_native_epoll_\1"/>
+ </move>
+ <echo message="deleting META-INF/native-image folder"/>
+ <delete dir="${project.build.directory}/unpacked/META-INF/native-image"/>
+ <echo message="repackaging netty jar"/>
+ <jar destfile="${project.build.directory}/${project.artifactId}-${project.version}.jar"
+ basedir="${project.build.directory}/unpacked"/>
+ </target>
+ </configuration>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
diff --git a/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/LICENSE b/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/LICENSE
index a37b034dc..fc5e57e6b 100644
--- a/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/LICENSE
+++ b/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/LICENSE
@@ -210,31 +210,54 @@ Apache License Version 2.0
org.apache.curator:curator-client
org.apache.curator:curator-framework
org.apache.curator:curator-recipes
-net.jodah:failsafe
-io.grpc:grpc-core
-io.grpc:grpc-protobuf
+org.apache.hive:hive-service-rpc
+org.apache.thrift:libfb303
+org.apache.thrift:libthrift
+org.apache.zookeeper:zookeeper
+com.google.api.grpc:proto-google-common-protos
+com.google.android:annotations
+com.google.code.gson:gson
+com.google.errorprone:error_prone_annotations
+com.google.guava:guava
+com.google.guava:failureaccess
+com.google.j2objc:j2objc-annotations
+io.etcd:jetcd-api
+io.etcd:jetcd-common
+io.etcd:jetcd-core
+io.etcd:jetcd-grpc
io.grpc:grpc-api
io.grpc:grpc-context
+io.grpc:grpc-core
io.grpc:grpc-grpclb
io.grpc:grpc-netty
+io.grpc:grpc-protobuf
io.grpc:grpc-protobuf-lite
io.grpc:grpc-stub
-com.google.guava:guava
-io.etcd:jetcd-api
-io.etcd:jetcd-common
-io.etcd:jetcd-core
-io.etcd:jetcd-grpc
+io.netty:netty-buffer
+io.netty:netty-codec
io.netty:netty-codec-dns
io.netty:netty-codec-http
io.netty:netty-codec-http2
io.netty:netty-codec-socks
+io.netty:netty-common
+io.netty:netty-handler
io.netty:netty-handler-proxy
+io.netty:netty-resolver
io.netty:netty-resolver-dns
+io.netty:netty-transport
+io.netty:netty-transport-classes-epoll
+io.netty:netty-transport-native-epoll
+io.netty:netty-transport-native-unix-common
io.perfmark:perfmark-api
-io.vertx:vertx-core
io.vertx:vertx-grpc
+io.vertx:vertx-core
+net.jodah:failsafe
BSD License
--------------------------
-com.google.protobuf:protobuf-java-util
com.google.protobuf:protobuf-java
+com.google.protobuf:protobuf-java-util
+
+MIT license
+--------------------------
+org.codehaus.mojo:animal-sniffer-annotations
diff --git a/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/NOTICE b/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/NOTICE
index b80c1b166..da91fb4fb 100644
--- a/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/NOTICE
+++ b/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/NOTICE
@@ -23,269 +23,5 @@ Copyright 2019 The Apache Software Foundation
Apache Thrift
Copyright 2006-2010 The Apache Software Foundation.
-Google Guava Version 18.0
-* Copyright (C) 2009 The Guava Authors
-
- The Netty Project
- =================
-
-Please visit the Netty web site for more information:
-
- * https://netty.io/
-
-Copyright 2014 The Netty Project
-
-The Netty Project licenses this file to you under the Apache License,
-version 2.0 (the "License"); you may not use this file except in compliance
-with the License. You may obtain a copy of the License at:
-
- https://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-License for the specific language governing permissions and limitations
-under the License.
-
-Also, please refer to each LICENSE.<component>.txt file, which is located in
-the 'license' directory of the distribution file, for the license terms of the
-components that this product depends on.
-
--------------------------------------------------------------------------------
-This product contains the extensions to Java Collections Framework which has
-been derived from the works by JSR-166 EG, Doug Lea, and Jason T. Greene:
-
- * LICENSE:
- * license/LICENSE.jsr166y.txt (Public Domain)
- * HOMEPAGE:
- * http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/
- * http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/
-
-This product contains a modified version of Robert Harder's Public Domain
-Base64 Encoder and Decoder, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.base64.txt (Public Domain)
- * HOMEPAGE:
- * http://iharder.sourceforge.net/current/java/base64/
-
-This product contains a modified portion of 'Webbit', an event based
-WebSocket and HTTP server, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.webbit.txt (BSD License)
- * HOMEPAGE:
- * https://github.com/joewalnes/webbit
-
-This product contains a modified portion of 'SLF4J', a simple logging
-facade for Java, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.slf4j.txt (MIT License)
- * HOMEPAGE:
- * https://www.slf4j.org/
-
-This product contains a modified portion of 'Apache Harmony', an open source
-Java SE, which can be obtained at:
-
- * NOTICE:
- * license/NOTICE.harmony.txt
- * LICENSE:
- * license/LICENSE.harmony.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://archive.apache.org/dist/harmony/
-
-This product contains a modified portion of 'jbzip2', a Java bzip2 compression
-and decompression library written by Matthew J. Francis. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.jbzip2.txt (MIT License)
- * HOMEPAGE:
- * https://code.google.com/p/jbzip2/
-
-This product contains a modified portion of 'libdivsufsort', a C API library to construct
-the suffix array and the Burrows-Wheeler transformed string for any input string of
-a constant-size alphabet written by Yuta Mori. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.libdivsufsort.txt (MIT License)
- * HOMEPAGE:
- * https://github.com/y-256/libdivsufsort
-
-This product contains a modified portion of Nitsan Wakart's 'JCTools', Java Concurrency Tools for the JVM,
- which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.jctools.txt (ASL2 License)
- * HOMEPAGE:
- * https://github.com/JCTools/JCTools
-
-This product optionally depends on 'JZlib', a re-implementation of zlib in
-pure Java, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.jzlib.txt (BSD style License)
- * HOMEPAGE:
- * http://www.jcraft.com/jzlib/
-
-This product optionally depends on 'Compress-LZF', a Java library for encoding and
-decoding data in LZF format, written by Tatu Saloranta. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.compress-lzf.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/ning/compress
-
-This product optionally depends on 'lz4', a LZ4 Java compression
-and decompression library written by Adrien Grand. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.lz4.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/jpountz/lz4-java
-
-This product optionally depends on 'lzma-java', a LZMA Java compression
-and decompression library, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.lzma-java.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/jponge/lzma-java
-
-This product optionally depends on 'zstd-jni', a zstd-jni Java compression
-and decompression library, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.zstd-jni.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/luben/zstd-jni
-
-This product contains a modified portion of 'jfastlz', a Java port of FastLZ compression
-and decompression library written by William Kinney. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.jfastlz.txt (MIT License)
- * HOMEPAGE:
- * https://code.google.com/p/jfastlz/
-
-This product contains a modified portion of and optionally depends on 'Protocol Buffers', Google's data
-interchange format, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.protobuf.txt (New BSD License)
- * HOMEPAGE:
- * https://github.com/google/protobuf
-
-This product optionally depends on 'Bouncy Castle Crypto APIs' to generate
-a temporary self-signed X.509 certificate when the JVM does not provide the
-equivalent functionality. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.bouncycastle.txt (MIT License)
- * HOMEPAGE:
- * https://www.bouncycastle.org/
-
-This product optionally depends on 'Snappy', a compression library produced
-by Google Inc, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.snappy.txt (New BSD License)
- * HOMEPAGE:
- * https://github.com/google/snappy
-
-This product optionally depends on 'JBoss Marshalling', an alternative Java
-serialization API, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.jboss-marshalling.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/jboss-remoting/jboss-marshalling
-
-This product optionally depends on 'Caliper', Google's micro-
-benchmarking framework, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.caliper.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/google/caliper
-
-This product optionally depends on 'Apache Commons Logging', a logging
-framework, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.commons-logging.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://commons.apache.org/logging/
-
-This product optionally depends on 'Apache Log4J', a logging framework, which
-can be obtained at:
-
- * LICENSE:
- * license/LICENSE.log4j.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://logging.apache.org/log4j/
-
-This product optionally depends on 'Aalto XML', an ultra-high performance
-non-blocking XML processor, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.aalto-xml.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://wiki.fasterxml.com/AaltoHome
-
-This product contains a modified version of 'HPACK', a Java implementation of
-the HTTP/2 HPACK algorithm written by Twitter. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.hpack.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/twitter/hpack
-
-This product contains a modified version of 'HPACK', a Java implementation of
-the HTTP/2 HPACK algorithm written by Cory Benfield. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.hyper-hpack.txt (MIT License)
- * HOMEPAGE:
- * https://github.com/python-hyper/hpack/
-
-This product contains a modified version of 'HPACK', a Java implementation of
-the HTTP/2 HPACK algorithm written by Tatsuhiro Tsujikawa. It can be obtained at:
-
- * LICENSE:
- * license/LICENSE.nghttp2-hpack.txt (MIT License)
- * HOMEPAGE:
- * https://github.com/nghttp2/nghttp2/
-
-This product contains a modified portion of 'Apache Commons Lang', a Java library
-provides utilities for the java.lang API, which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.commons-lang.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://commons.apache.org/proper/commons-lang/
-
-
-This product contains the Maven wrapper scripts from 'Maven Wrapper', that provides an easy way to ensure a user has everything necessary to run the Maven build.
-
- * LICENSE:
- * license/LICENSE.mvn-wrapper.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/takari/maven-wrapper
-
-This product contains the dnsinfo.h header file, that provides a way to retrieve the system DNS configuration on MacOS.
-This private header is also used by Apple's open source
- mDNSResponder (https://opensource.apple.com/tarballs/mDNSResponder/).
-
- * LICENSE:
- * license/LICENSE.dnsinfo.txt (Apple Public Source License 2.0)
- * HOMEPAGE:
- * https://www.opensource.apple.com/source/configd/configd-453.19/dnsinfo/dnsinfo.h
-
-This product optionally depends on 'Brotli4j', Brotli compression and
-decompression for Java., which can be obtained at:
-
- * LICENSE:
- * license/LICENSE.brotli4j.txt (Apache License 2.0)
- * HOMEPAGE:
- * https://github.com/hyperxpro/Brotli4j
+Netty
+Copyright 2016 The Netty Project
diff --git a/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/services/org.apache.spark.status.AppHistoryServerPlugin b/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/services/org.apache.spark.status.AppHistoryServerPlugin
index 2be4ed7e4..2450f7ba1 100644
--- a/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/services/org.apache.spark.status.AppHistoryServerPlugin
+++ b/externals/kyuubi-spark-sql-engine/src/main/resources/META-INF/services/org.apache.spark.status.AppHistoryServerPlugin
@@ -1 +1,18 @@
-org.apache.spark.ui.KyuubiHistoryServerPlugin
\ No newline at end of file
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+org.apache.spark.ui.KyuubiHistoryServerPlugin
diff --git a/kyuubi-ha/pom.xml b/kyuubi-ha/pom.xml
index 3bee82803..942fedaf6 100644
--- a/kyuubi-ha/pom.xml
+++ b/kyuubi-ha/pom.xml
@@ -79,6 +79,18 @@
<artifactId>grpc-netty</artifactId>
</dependency>
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-epoll</artifactId>
+ <classifier>linux-aarch_64</classifier>
+ </dependency>
+
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-transport-native-epoll</artifactId>
+ <classifier>linux-x86_64</classifier>
+ </dependency>
+
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
diff --git a/pom.xml b/pom.xml
index 428f15fed..984d016d2 100644
--- a/pom.xml
+++ b/pom.xml
@@ -159,6 +159,7 @@
<netty.version>4.1.84.Final</netty.version>
<parquet.version>1.10.1</parquet.version>
<prometheus.version>0.16.0</prometheus.version>
+ <protobuf.version>3.21.1</protobuf.version>
<py4j.version>0.10.7</py4j.version>
<ranger.version>2.3.0</ranger.version>
<scalacheck.version>3.2.9.0</scalacheck.version>
@@ -1013,33 +1014,33 @@
<dependency>
<groupId>io.netty</groupId>
- <artifactId>netty-transport-classes-epoll</artifactId>
+ <artifactId>netty-transport</artifactId>
<version>${netty.version}</version>
</dependency>
<dependency>
<groupId>io.netty</groupId>
- <artifactId>netty-transport-native-epoll</artifactId>
+ <artifactId>netty-transport-classes-epoll</artifactId>
<version>${netty.version}</version>
- <classifier>linux-aarch_64</classifier>
</dependency>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-transport-native-epoll</artifactId>
<version>${netty.version}</version>
- <classifier>linux-x86_64</classifier>
+ <classifier>linux-aarch_64</classifier>
</dependency>
<dependency>
<groupId>io.netty</groupId>
- <artifactId>netty-transport-native-unix-common</artifactId>
+ <artifactId>netty-transport-native-epoll</artifactId>
<version>${netty.version}</version>
+ <classifier>linux-x86_64</classifier>
</dependency>
<dependency>
<groupId>io.netty</groupId>
- <artifactId>netty-transport</artifactId>
+ <artifactId>netty-transport-native-unix-common</artifactId>
<version>${netty.version}</version>
</dependency>
@@ -1195,6 +1196,18 @@
<version>${grpc.version}</version>
</dependency>
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java</artifactId>
+ <version>${protobuf.version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>com.google.protobuf</groupId>
+ <artifactId>protobuf-java-util</artifactId>
+ <version>${protobuf.version}</version>
+ </dependency>
+
<dependency>
<groupId>net.jodah</groupId>
<artifactId>failsafe</artifactId>