You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ratis.apache.org by ji...@apache.org on 2017/03/29 02:40:11 UTC
incubator-ratis git commit: RATIS-49. Add a clean profile to remove
shaded source. Contributed by Tsz Wo Nicholas Sze.
Repository: incubator-ratis
Updated Branches:
refs/heads/master 768d7caa4 -> 930f4c019
RATIS-49. Add a clean profile to remove shaded source. Contributed by Tsz Wo Nicholas Sze.
Project: http://git-wip-us.apache.org/repos/asf/incubator-ratis/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-ratis/commit/930f4c01
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ratis/tree/930f4c01
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ratis/diff/930f4c01
Branch: refs/heads/master
Commit: 930f4c019e3d8acc81376182c941ef887cc8a865
Parents: 768d7ca
Author: Jing Zhao <ji...@apache.org>
Authored: Tue Mar 28 19:40:01 2017 -0700
Committer: Jing Zhao <ji...@apache.org>
Committed: Tue Mar 28 19:40:01 2017 -0700
----------------------------------------------------------------------
BUILDING.md | 55 +++++++++++++++++++++++++---------------
pom.xml | 2 +-
ratis-hadoop-shaded/pom.xml | 42 +++++++++++++++++++++++-------
ratis-proto-shaded/pom.xml | 54 +++++++++++++++++++++------------------
4 files changed, 99 insertions(+), 54 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/BUILDING.md
----------------------------------------------------------------------
diff --git a/BUILDING.md b/BUILDING.md
index 4c634a4..42a6f08 100644
--- a/BUILDING.md
+++ b/BUILDING.md
@@ -1,38 +1,54 @@
# Building
-Ratis uses Apache Maven for the builds. A 3.0+ version of Maven is required as well as
+Apache Ratis uses Apache Maven for the builds. A 3.2+ version of Maven is required as well as
at least Java-1.8.
-When Ratis is build for the first time, shaded protobuf files needs to be generated first
-which happens only in the mvn package phase. That is why you should run:
+When building Ratis the first time, shaded files need to be generated by the following command:
```
$ mvn package -DskipTests
```
-the first time the sources are checked out first. After doing that mvn compile or mvn test
-can be used as normal.
+After that, `mvn compile` or `mvn test` can be used as normal.
+For example, we may run the basic tests by
+```
+$ mvn test -Dtest=TestRaftWith\*
+```
-# Raft Proto Shaded
+# Shading
-We shade protos, protobuf and other libraries such as Netty, gRPC and Hadoop so that
-applications using Raft may use protobuf and other libraries with versions different
-from the versions used here.
+We shade protos, protobuf and other libraries such as Netty, gRPC, Guava and Hadoop
+so that applications using Ratis may use protobuf and other libraries with versions
+different from the versions used here.
-The library requires the shaded sources for compilation. The generated sources are stored in
-`ratis-proto-shaded/src/main/java/`. They are not checked-in git though.
+Ratis requires the shaded sources for compilation.
+The generated sources are stored in the following directories.
+```
+ratis-proto-shaded/src/main/java/
+ratis-hadoop-shaded/src/main/java/
+```
+They are not checked-in to git though.
-If you want to force-compile the proto files (for example after changing them), you should
-run with
+If you want to force compiling the proto files (for example after changing them),
+you should run
```
$ mvn package -DskipTests -Dcompile-protobuf
```
+For removing the shaded sources, run
+```
+$ mvn clean -Pclean-shade
+```
+
+Note that `mvn clean` alone does not remove the shaded sources.
+
## What are shaded?
-| Original packages | Shaded packages |
-| ----------------------------------|----------------------------------------------------------|
-| `com.google.protobuf` | `org.apache.ratis.shaded.com.google.protobuf` |
-| `io.grpc` | `org.apache.ratis.shaded.io.grpc` |
-| `io.netty.handler.codec.protobuf` | `org.apache.ratis.shaded.io.netty.handler.codec.protobuf` |
-| `org.apache.hadoop.ipc.protobuf` | `org.apache.ratis.shaded.org.apache.hadoop.ipc.protobuf` |
+| Original packages | Shaded packages |
+| ------------------------------------|--------------------------------------------------------------|
+| `com.google.common` | `org.apache.ratis.shaded.com.google.common` |
+| `com.google.protobuf` | `org.apache.ratis.shaded.com.google.protobuf` |
+| `com.google.thirdparty.publicsuffix`| `org.apache.ratis.shaded.com.google.thirdparty.publicsuffix` |
+| `io.grpc` | `org.apache.ratis.shaded.io.grpc` |
+| `io.netty` | `org.apache.ratis.shaded.io.netty` |
+| `org.apache.hadoop.ipc.protobuf` | `org.apache.ratis.shaded.org.apache.hadoop.ipc.protobuf` |
The protos defined in this project are stored in the `org.apache.ratis.shaded.proto` package.
@@ -53,7 +69,6 @@ To publish, use the following settings.xml file ( placed in ~/.m2/settings.xml )
<username>ratis_committer</username>
<password>********</password>
</server>
-
</servers>
</settings>
```
http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/pom.xml
----------------------------------------------------------------------
diff --git a/pom.xml b/pom.xml
index 7e0869f..ea4f72b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -134,7 +134,7 @@
<maven-assembly-plugin.version>2.4</maven-assembly-plugin.version>
<maven-bundle-plugin.version>2.5.3</maven-bundle-plugin.version>
<maven-checkstyle-plugin.version>2.15</maven-checkstyle-plugin.version>
- <maven-clean-plugin.version>2.5</maven-clean-plugin.version>
+ <maven-clean-plugin.version>3.0.0</maven-clean-plugin.version>
<maven-clover2-plugin.version>3.3.0</maven-clover2-plugin.version>
<maven-compiler-plugin.version>3.2</maven-compiler-plugin.version>
<maven-dependency-plugin.version>2.10</maven-dependency-plugin.version>
http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/ratis-hadoop-shaded/pom.xml
----------------------------------------------------------------------
diff --git a/ratis-hadoop-shaded/pom.xml b/ratis-hadoop-shaded/pom.xml
index cb74fa2..0321a22 100644
--- a/ratis-hadoop-shaded/pom.xml
+++ b/ratis-hadoop-shaded/pom.xml
@@ -32,7 +32,7 @@
<!--The Default target dir-->
<classes.dir>${project.build.directory}/classes</classes.dir>
<!--The Default location for sources-->
- <sources.dir>src/main/java</sources.dir>
+ <shaded.sources.dir>src/main/java</shaded.sources.dir>
<!--Version of Hadoop to be shaded -->
<shaded.hadoop.version>${hadoop.version}</shaded.hadoop.version>
@@ -41,7 +41,7 @@
<build>
<!--I want to override these in profile so define them
with variables up here-->
- <sourceDirectory>${sources.dir}</sourceDirectory>
+ <sourceDirectory>${shaded.sources.dir}</sourceDirectory>
<outputDirectory>${classes.dir}</outputDirectory>
<plugins>
<!-- Make a jar and put the sources in the jar -->
@@ -72,17 +72,20 @@
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>${shaded.protobuf.version}</version>
+ <optional>true</optional>
</dependency>
<dependency>
<groupId>com.google.protobuf.nano</groupId>
<artifactId>protobuf-javanano</artifactId>
<version>${shaded.protobuf.version}</version>
+ <optional>true</optional>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>${shaded.hadoop.version}</version>
+ <optional>true</optional>
</dependency>
</dependencies>
@@ -101,19 +104,41 @@
</profile>
<profile>
- <id>skipShade</id>
+ <id>skipCompileProto</id>
<activation>
<file>
- <exists>${sources.dir}</exists>
+ <exists>${shaded.sources.dir}</exists>
</file>
<property>
- <name>!shade-hadoop</name>
+ <name>!compile-protobuf</name>
</property>
</activation>
</profile>
<profile>
- <id>shade-hadoop</id>
+ <id>clean-shade</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-clean-plugin</artifactId>
+ <configuration>
+ <filesets>
+ <fileset>
+ <directory>${shaded.sources.dir}</directory>
+ <followSymlinks>false</followSymlinks>
+ </fileset>
+ </filesets>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <profile>
+ <id>compile-protobuf</id>
<!--
Shade and drop the generated java files under src/main/java.
-->
@@ -121,7 +146,7 @@
<activeByDefault>true</activeByDefault>
</activation>
<properties>
- <profile.id>shade-hadoop</profile.id>
+ <profile.id>compile-protobuf</profile.id>
<!--When the compile for this profile runs, make sure it makes jars that
can be related back to this shading profile. Give them a shading prefix.
-->
@@ -273,8 +298,7 @@
<classifier>sources</classifier>
<type>jar</type>
<overWrite>true</overWrite>
- <outputDirectory>${basedir}/src/main/java
- </outputDirectory>
+ <outputDirectory>${basedir}/${shaded.sources.dir}</outputDirectory>
<includes>**/*.java</includes>
</artifactItem>
</artifactItems>
http://git-wip-us.apache.org/repos/asf/incubator-ratis/blob/930f4c01/ratis-proto-shaded/pom.xml
----------------------------------------------------------------------
diff --git a/ratis-proto-shaded/pom.xml b/ratis-proto-shaded/pom.xml
index 76148c4..dc32403 100644
--- a/ratis-proto-shaded/pom.xml
+++ b/ratis-proto-shaded/pom.xml
@@ -32,8 +32,7 @@
<!--The Default target dir-->
<classes.dir>${project.build.directory}/classes</classes.dir>
<!--The Default location for sources-->
- <sources.dir>src/main/java</sources.dir>
-
+ <shaded.sources.dir>src/main/java</shaded.sources.dir>
<!--Version of grpc to be shaded -->
<shaded.grpc.version>1.0.1</shaded.grpc.version>
@@ -44,7 +43,7 @@
<build>
<!--I want to override these in profile so define them
with variables up here-->
- <sourceDirectory>${sources.dir}</sourceDirectory>
+ <sourceDirectory>${shaded.sources.dir}</sourceDirectory>
<outputDirectory>${classes.dir}</outputDirectory>
<plugins>
<!-- Make a jar and put the sources in the jar -->
@@ -372,10 +371,10 @@
<profiles>
<!-- Skip the tests in this module -->
<profile>
- <id>skip-proto-shaded-tests</id>
+ <id>skip-shaded-tests</id>
<activation>
<property>
- <name>skip-proto-shaded-tests</name>
+ <name>skip-shaded-tests</name>
</property>
</activation>
<properties>
@@ -387,7 +386,7 @@
<id>skipCompileProto</id>
<activation>
<file>
- <exists>${sources.dir}</exists>
+ <exists>${shaded.sources.dir}</exists>
</file>
<property>
<name>!compile-protobuf</name>
@@ -396,25 +395,33 @@
</profile>
<profile>
+ <id>clean-shade</id>
+ <activation>
+ <activeByDefault>false</activeByDefault>
+ </activation>
+ <build>
+ <plugins>
+ <plugin>
+ <artifactId>maven-clean-plugin</artifactId>
+ <configuration>
+ <filesets>
+ <fileset>
+ <directory>${shaded.sources.dir}</directory>
+ <followSymlinks>false</followSymlinks>
+ </fileset>
+ </filesets>
+ </configuration>
+ </plugin>
+ </plugins>
+ </build>
+ </profile>
+
+ <profile>
<id>compile-protobuf</id>
<!--
- Generate and shade proto files. Drops generated java files
- under src/main/java. Check in the generated files so available
- at build time. Run this profile/step everytime you change proto
+ Shade and drop the generated java files under src/main/java.
+ Run this profile/step everytime you change proto
files or update the protobuf version.
-
- The below does a bunch of ugly stuff. It purges current content
- of the generated and shaded com.google.protobuf java files first.
- It does this because later we apply patches later and patches
- fail they've already been applied. We remove too because we
- overlay the shaded protobuf and if files have been removed or
- added, it'll be more plain if we have first done this delete.
-
- Next up we generate proto, build a jar, shade it (which
- includes the referenced protobuf), undo it over the src/main/java
- directory, and then apply patches.
-
- The result needs to be checked in.
-->
<activation>
<activeByDefault>true</activeByDefault>
@@ -594,8 +601,7 @@
<classifier>sources</classifier>
<type>jar</type>
<overWrite>true</overWrite>
- <outputDirectory>${basedir}/src/main/java
- </outputDirectory>
+ <outputDirectory>${basedir}/${shaded.sources.dir}</outputDirectory>
<includes>**/*.java</includes>
</artifactItem>
</artifactItems>