You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ch...@apache.org on 2022/09/13 03:55:08 UTC
[incubator-kyuubi] branch branch-1.6 updated: [KYUUBI #3459] [BUILD] Share jars to reduce binary release tarball size
This is an automated email from the ASF dual-hosted git repository.
chengpan pushed a commit to branch branch-1.6
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
The following commit(s) were added to refs/heads/branch-1.6 by this push:
new c1c430121 [KYUUBI #3459] [BUILD] Share jars to reduce binary release tarball size
c1c430121 is described below
commit c1c4301216efa66891ebfa58556e8276b6eca6f6
Author: Cheng Pan <ch...@apache.org>
AuthorDate: Tue Sep 13 11:54:46 2022 +0800
[KYUUBI #3459] [BUILD] Share jars to reduce binary release tarball size
### _Why are the changes needed?_
The 1.6.0 binary tarball is nearly 300MB, we can reduce the size by sharing jars between engine and server, as beeline does.
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [x] Add screenshots for manual tests if appropriate
```
-rw-r--r-- 1 chengpan staff 155M Sep 9 14:21 apache-kyuubi-1.7.0-SNAPSHOT-bin.tgz
```
- [ ] [Run test](https://kyuubi.apache.org/docs/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes #3459 from pan3793/dist.
Closes #3459
3a995e77 [Cheng Pan] fix
b86d1a0b [Cheng Pan] [BUILD] Share jars to reduce binary release tarball size
Authored-by: Cheng Pan <ch...@apache.org>
Signed-off-by: Cheng Pan <ch...@apache.org>
(cherry picked from commit b0dd4ead37c83591a2d2f9c376c904c60dc73e8a)
Signed-off-by: Cheng Pan <ch...@apache.org>
---
build/dist | 21 ++++++++++++++++-----
1 file changed, 16 insertions(+), 5 deletions(-)
diff --git a/build/dist b/build/dist
index 6ea731128..9cd93cbf1 100755
--- a/build/dist
+++ b/build/dist
@@ -234,15 +234,12 @@ cp -r "$KYUUBI_HOME/kyuubi-assembly/target/scala-$SCALA_VERSION/jars/" "$DISTDIR
# Copy kyuubi beeline jars
cp "$KYUUBI_HOME"/kyuubi-hive-beeline/target/*.jar "$DISTDIR/beeline-jars/"
-# Share the jars between server and beeline to reduce binary size
-cd $DISTDIR/beeline-jars
+# Share the jars w/ server to reduce binary size
for jar in $(ls "$DISTDIR/jars/"); do
if [[ -f "$DISTDIR/beeline-jars/$jar" ]]; then
- rm "$DISTDIR/beeline-jars/$jar"
- ln -sn "../jars/$jar" "$DISTDIR/beeline-jars/$jar"
+ (cd $DISTDIR/beeline-jars; ln -snf "../jars/$jar" "$DISTDIR/beeline-jars/$jar")
fi
done
-cd -
# Copy flink engines
cp "$KYUUBI_HOME/externals/kyuubi-flink-sql-engine/target/kyuubi-flink-sql-engine_${SCALA_VERSION}-${VERSION}.jar" "$DISTDIR/externals/engines/flink"
@@ -254,6 +251,13 @@ cp "$KYUUBI_HOME/externals/kyuubi-spark-sql-engine/target/kyuubi-spark-sql-engin
cp "$KYUUBI_HOME/externals/kyuubi-trino-engine/target/kyuubi-trino-engine_${SCALA_VERSION}-${VERSION}.jar" "$DISTDIR/externals/engines/trino"
cp -r "$KYUUBI_HOME/externals/kyuubi-trino-engine/target/scala-$SCALA_VERSION/jars/" "$DISTDIR/externals/engines/trino"
+# Share the jars w/ server to reduce binary size
+for jar in $(ls "$DISTDIR/jars/"); do
+ if [[ -f "$DISTDIR/externals/engines/trino/$jar" ]]; then
+ (cd $DISTDIR/externals/engines/trino; ln -snf "../../../jars/$jar" "$DISTDIR/externals/engines/trino/$jar")
+ fi
+done
+
# Copy hive engines
cp "$KYUUBI_HOME/externals/kyuubi-hive-sql-engine/target/kyuubi-hive-sql-engine_${SCALA_VERSION}-${VERSION}.jar" "$DISTDIR/externals/engines/hive"
@@ -261,6 +265,13 @@ cp "$KYUUBI_HOME/externals/kyuubi-hive-sql-engine/target/kyuubi-hive-sql-engine_
cp "$KYUUBI_HOME/externals/kyuubi-jdbc-engine/target/kyuubi-jdbc-engine_${SCALA_VERSION}-${VERSION}.jar" "$DISTDIR/externals/engines/jdbc"
cp -r "$KYUUBI_HOME/externals/kyuubi-jdbc-engine/target/scala-$SCALA_VERSION/jars/" "$DISTDIR/externals/engines/jdbc"
+# Share the jars w/ server to reduce binary size
+for jar in $(ls "$DISTDIR/jars/"); do
+ if [[ -f "$DISTDIR/externals/engines/jdbc/$jar" ]]; then
+ (cd $DISTDIR/externals/engines/jdbc; ln -snf "../../../jars/$jar" "$DISTDIR/externals/engines/jdbc/$jar")
+ fi
+done
+
# Copy kyuubi tools
if [[ -f "$KYUUBI_HOME/tools/spark-block-cleaner/target/spark-block-cleaner_${SCALA_VERSION}-${VERSION}.jar" ]]; then
mkdir -p "$DISTDIR/tools/spark-block-cleaner/kubernetes"