You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@kyuubi.apache.org by ul...@apache.org on 2021/10/16 11:24:01 UTC
[incubator-kyuubi] branch master updated: [KYUUBI #1241] Build
Spark extension fat jar with common module
This is an automated email from the ASF dual-hosted git repository.
ulyssesyou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new 7961b32 [KYUUBI #1241] Build Spark extension fat jar with common module
7961b32 is described below
commit 7961b321666faea371fa0de27596f2aea499869e
Author: ulysses-you <ul...@gmail.com>
AuthorDate: Sat Oct 16 19:23:45 2021 +0800
[KYUUBI #1241] Build Spark extension fat jar with common module
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
see https://github.com/apache/incubator-kyuubi/issues/1241
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [x] [Run test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests) locally before make a pull request
Closes #1242 from ulysses-you/KYUUBI-1241.
Closes #1241
a8e25c37 [ulysses-you] address comment
d25a3664 [ulysses-you] version
3c982938 [ulysses-you] Build Spark extension fat jar with common module
Authored-by: ulysses-you <ul...@gmail.com>
Signed-off-by: ulysses-you <ul...@apache.org>
---
build/dist | 2 +-
dev/kyuubi-extension-spark-3-1/pom.xml | 23 +++++++++++++++++++++++
dev/kyuubi-extension-spark-3-2/pom.xml | 30 ++++++++++++++++++------------
3 files changed, 42 insertions(+), 13 deletions(-)
diff --git a/build/dist b/build/dist
index fc3562d..27310d9 100755
--- a/build/dist
+++ b/build/dist
@@ -201,7 +201,7 @@ if [[ -f "$KYUUBI_HOME/tools/spark-block-cleaner/target/spark-block-cleaner_${SC
fi
# Copy Kyuubi extension
-SPARK_EXTENSION_VERSIONS=('3-1')
+SPARK_EXTENSION_VERSIONS=('3-1' '3-2')
for SPARK_EXTENSION_VERSION in ${SPARK_EXTENSION_VERSIONS[@]}; do
if [[ -f $"$KYUUBI_HOME/dev/kyuubi-extension-spark-$SPARK_EXTENSION_VERSION/target/kyuubi-extension-spark-${SPARK_EXTENSION_VERSION}_${SCALA_VERSION}-${VERSION}.jar" ]]; then
mkdir -p "$DISTDIR/extension"
diff --git a/dev/kyuubi-extension-spark-3-1/pom.xml b/dev/kyuubi-extension-spark-3-1/pom.xml
index 652a62c..1a7d517 100644
--- a/dev/kyuubi-extension-spark-3-1/pom.xml
+++ b/dev/kyuubi-extension-spark-3-1/pom.xml
@@ -131,6 +131,29 @@
<build>
<outputDirectory>target/scala-${scala.binary.version}/classes</outputDirectory>
<testOutputDirectory>target/scala-${scala.binary.version}/test-classes</testOutputDirectory>
+
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <shadedArtifactAttached>false</shadedArtifactAttached>
+ <artifactSet>
+ <includes>
+ <include>org.apache.kyuubi:kyuubi-extension-spark-common_${scala.binary.version}</include>
+ </includes>
+ </artifactSet>
+ </configuration>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
</build>
<profiles>
diff --git a/dev/kyuubi-extension-spark-3-2/pom.xml b/dev/kyuubi-extension-spark-3-2/pom.xml
index 8e61e78..92d8064 100644
--- a/dev/kyuubi-extension-spark-3-2/pom.xml
+++ b/dev/kyuubi-extension-spark-3-2/pom.xml
@@ -139,18 +139,24 @@
<plugins>
<plugin>
- <groupId>net.alchim31.maven</groupId>
- <artifactId>scala-maven-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>org.scalastyle</groupId>
- <artifactId>scalastyle-maven-plugin</artifactId>
- </plugin>
-
- <plugin>
- <groupId>org.scalatest</groupId>
- <artifactId>scalatest-maven-plugin</artifactId>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <configuration>
+ <shadedArtifactAttached>false</shadedArtifactAttached>
+ <artifactSet>
+ <includes>
+ <include>org.apache.kyuubi:kyuubi-extension-spark-common_${scala.binary.version}</include>
+ </includes>
+ </artifactSet>
+ </configuration>
+ <executions>
+ <execution>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ </execution>
+ </executions>
</plugin>
</plugins>
</build>