You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by ch...@apache.org on 2019/05/15 07:00:21 UTC
[flink-shaded] branch master updated (651ecb5 -> 916cab3)
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/flink-shaded.git.
from 651ecb5 [hotfix][build] Add Java 9 support
new 4875bdc [FLINK-11580][netty] add statically linked netty-tcnative
new 4769596 [FLINK-11580][netty] add dynamically linked netty-tcnative
new fa96419 fixup! [FLINK-9816][netty] add statically linked netty-tcnative
new 916cab3 fixup! [FLINK-9816][netty] add dynamically linked netty-tcnative
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
flink-shaded-netty-tcnative-dynamic/pom.xml | 186 +++++++++++++++++++++
.../src/main/resources/META-INF/NOTICE | 6 +-
.../pom.xml | 19 ++-
.../src/main/resources/META-INF/NOTICE | 14 ++
pom.xml | 17 ++
5 files changed, 230 insertions(+), 12 deletions(-)
create mode 100644 flink-shaded-netty-tcnative-dynamic/pom.xml
copy {flink-shaded-netty-4 => flink-shaded-netty-tcnative-dynamic}/src/main/resources/META-INF/NOTICE (65%)
copy {flink-shaded-netty-4 => flink-shaded-netty-tcnative-static}/pom.xml (84%)
create mode 100644 flink-shaded-netty-tcnative-static/src/main/resources/META-INF/NOTICE
[flink-shaded] 03/04: fixup! [FLINK-9816][netty] add statically
linked netty-tcnative
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-shaded.git
commit fa96419b0226d0bc76084cec6fd84d3cce3a19ed
Author: Nico Kruber <ni...@gmail.com>
AuthorDate: Sat May 11 09:45:18 2019 +0200
fixup! [FLINK-9816][netty] add statically linked netty-tcnative
---
pom.xml | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/pom.xml b/pom.xml
index 843ad64..d668fb2 100644
--- a/pom.xml
+++ b/pom.xml
@@ -63,6 +63,10 @@ under the License.
</modules>
<profiles>
+ <!--
+ This module is excluded by default since it is not compliant with
+ the apache license. https://issues.apache.org/jira/browse/LEGAL-393
+ -->
<profile>
<id>include-netty-tcnative-static</id>
<activation>
[flink-shaded] 04/04: fixup! [FLINK-9816][netty] add dynamically
linked netty-tcnative
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-shaded.git
commit 916cab38bae47121901d9180500dd35def56e30d
Author: Nico Kruber <ni...@gmail.com>
AuthorDate: Sat May 11 09:45:25 2019 +0200
fixup! [FLINK-9816][netty] add dynamically linked netty-tcnative
---
flink-shaded-netty-tcnative-dynamic/pom.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/flink-shaded-netty-tcnative-dynamic/pom.xml b/flink-shaded-netty-tcnative-dynamic/pom.xml
index 6badaaa..c9450f5 100644
--- a/flink-shaded-netty-tcnative-dynamic/pom.xml
+++ b/flink-shaded-netty-tcnative-dynamic/pom.xml
@@ -91,7 +91,7 @@ under the License.
</executions>
<configuration>
<artifactItems>
- <!-- native wrapper libraries linked dynamically to OpenSSL - we will extract and these below-->
+ <!-- native wrapper libraries linked dynamically to OpenSSL - we will extract, rename, and include these below-->
<artifactItem>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative</artifactId>
[flink-shaded] 02/04: [FLINK-11580][netty] add dynamically linked
netty-tcnative
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-shaded.git
commit 4769596c0c328a5c4a54d17d6599a63dd379c828
Author: Nico Kruber <ni...@gmail.com>
AuthorDate: Wed May 8 11:07:57 2019 +0200
[FLINK-11580][netty] add dynamically linked netty-tcnative
With one of the artifacts of netty-tcnative [1], this enables support for using
the OpenSSL engine instead of the one from the JDK. [2] claims significant
improvements up to 3x over the JDK version and we may also save some resources.
From the available artifacts, this commits selects the ones which are
dynamically linked against OpenSSL. Please refer to [1] and [3] for further
details.
Compared to the statically-linked version, this does not bundle any components
under the openSSL license and should thus be safe to use under the Apache 2
license.
[1] http://netty.io/wiki/forked-tomcat-native.html
[2] https://netty.io/wiki/requirements-for-4.x.html#wiki-h4-4
[3] https://netty.io/wiki/forked-tomcat-native.html#prerequisites-for-dynamically-linked-netty-tcnative
---
flink-shaded-netty-tcnative-dynamic/pom.xml | 186 +++++++++++++++++++++
.../src/main/resources/META-INF/NOTICE | 9 +
pom.xml | 1 +
3 files changed, 196 insertions(+)
diff --git a/flink-shaded-netty-tcnative-dynamic/pom.xml b/flink-shaded-netty-tcnative-dynamic/pom.xml
new file mode 100644
index 0000000..6badaaa
--- /dev/null
+++ b/flink-shaded-netty-tcnative-dynamic/pom.xml
@@ -0,0 +1,186 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-shaded</artifactId>
+ <version>7.0</version>
+ <relativePath>..</relativePath>
+ </parent>
+
+ <artifactId>flink-shaded-netty-tcnative-dynamic</artifactId>
+ <name>flink-shaded-netty-tcnative-dynamic</name>
+ <version>${netty.tcnative.version}-7.0</version>
+
+ <properties>
+ <netty.tcnative.version>2.0.25.Final</netty.tcnative.version>
+ </properties>
+
+ <dependencies>
+ <!-- Netty's native openSSL support -->
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-tcnative</artifactId>
+ <version>${netty.tcnative.version}</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>shade-flink</id>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <createDependencyReducedPom>true</createDependencyReducedPom>
+ <dependencyReducedPomLocation>${project.basedir}/target/dependency-reduced-pom.xml</dependencyReducedPomLocation>
+ <artifactSet>
+ <includes>
+ <include>io.netty:netty-tcnative</include>
+ </includes>
+ </artifactSet>
+ <relocations>
+ <relocation>
+ <pattern>io.netty</pattern>
+ <shadedPattern>org.apache.flink.shaded.netty4.io.netty</shadedPattern>
+ </relocation>
+ </relocations>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <version>3.1.1</version>
+ <executions>
+ <execution>
+ <id>copy</id>
+ <phase>package</phase>
+ <goals>
+ <goal>copy</goal>
+ </goals>
+ </execution>
+ </executions>
+ <configuration>
+ <artifactItems>
+ <!-- native wrapper libraries linked dynamically to OpenSSL - we will extract and these below-->
+ <artifactItem>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-tcnative</artifactId>
+ <version>${netty.tcnative.version}</version>
+ <classifier>linux-x86_64</classifier>
+ <type>jar</type>
+ <overWrite>false</overWrite>
+ <outputDirectory>${project.build.directory}/native_libs</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-tcnative</artifactId>
+ <version>${netty.tcnative.version}</version>
+ <classifier>linux-x86_64-fedora</classifier>
+ <type>jar</type>
+ <overWrite>false</overWrite>
+ <outputDirectory>${project.build.directory}/native_libs</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-tcnative</artifactId>
+ <version>${netty.tcnative.version}</version>
+ <classifier>osx-x86_64</classifier>
+ <type>jar</type>
+ <overWrite>false</overWrite>
+ <outputDirectory>${project.build.directory}/native_libs</outputDirectory>
+ </artifactItem>
+ <artifactItem>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-tcnative</artifactId>
+ <version>${netty.tcnative.version}</version>
+ <classifier>windows-x86_64</classifier>
+ <type>jar</type>
+ <overWrite>false</overWrite>
+ <outputDirectory>${project.build.directory}/native_libs</outputDirectory>
+ </artifactItem>
+ </artifactItems>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <version>1.7</version>
+ <executions>
+ <execution>
+ <id>rename-native-library</id>
+ <phase>package</phase>
+ <configuration>
+ <target>
+ <echo message="unpacking netty jar" />
+ <unzip src="${project.build.directory}/${artifactId}-${version}.jar" dest="${project.build.directory}/unpacked/" />
+ <echo message="extracting netty_tcnative dynamically linked wrapper libraries" />
+ <!-- Fix the dynamically linked native libraries in netty-tcnative -->
+ <unzip src="${project.build.directory}/native_libs/netty-tcnative-${netty.tcnative.version}-linux-x86_64.jar" dest="${project.build.directory}/native_libs/unpacked/" />
+ <move todir="${project.build.directory}/unpacked/META-INF/native" includeemptydirs="false">
+ <fileset dir="${project.build.directory}/native_libs/unpacked/META-INF/native"/>
+ <mapper type="glob" from="libnetty_tcnative.so" to="liborg_apache_flink_shaded_netty4_netty_tcnative_linux_x86_64.so"/>
+ </move>
+ <delete dir="${project.build.directory}/native_libs/unpacked/" />
+ <unzip src="${project.build.directory}/native_libs/netty-tcnative-${netty.tcnative.version}-linux-x86_64-fedora.jar" dest="${project.build.directory}/native_libs/unpacked/" />
+ <move todir="${project.build.directory}/unpacked/META-INF/native" includeemptydirs="false">
+ <fileset dir="${project.build.directory}/native_libs/unpacked/META-INF/native"/>
+ <mapper type="glob" from="libnetty_tcnative.so" to="liborg_apache_flink_shaded_netty4_netty_tcnative_linux_x86_64_fedora.so"/>
+ </move>
+ <delete dir="${project.build.directory}/native_libs/unpacked/" />
+ <unzip src="${project.build.directory}/native_libs/netty-tcnative-${netty.tcnative.version}-osx-x86_64.jar" dest="${project.build.directory}/native_libs/unpacked/" />
+ <move todir="${project.build.directory}/unpacked/META-INF/native" includeemptydirs="false">
+ <fileset dir="${project.build.directory}/native_libs/unpacked/META-INF/native"/>
+ <mapper type="glob" from="libnetty_tcnative.jnilib" to="liborg_apache_flink_shaded_netty4_netty_tcnative_osx_x86_64.jnilib"/>
+ </move>
+ <delete dir="${project.build.directory}/native_libs/unpacked/" />
+ <unzip src="${project.build.directory}/native_libs/netty-tcnative-${netty.tcnative.version}-windows-x86_64.jar" dest="${project.build.directory}/native_libs/unpacked/" />
+ <move todir="${project.build.directory}/unpacked/META-INF/native" includeemptydirs="false">
+ <fileset dir="${project.build.directory}/native_libs/unpacked/META-INF/native"/>
+ <mapper type="glob" from="netty_tcnative.dll" to="org_apache_flink_shaded_netty4_netty_tcnative_windows_x86_64.dll"/>
+ </move>
+ <delete dir="${project.build.directory}/native_libs/unpacked/" />
+ <echo message="repackaging netty jar" />
+ <jar destfile="${project.build.directory}/${artifactId}-${version}.jar" basedir="${project.build.directory}/unpacked" />
+ </target>
+ </configuration>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+
+</project>
diff --git a/flink-shaded-netty-tcnative-dynamic/src/main/resources/META-INF/NOTICE b/flink-shaded-netty-tcnative-dynamic/src/main/resources/META-INF/NOTICE
new file mode 100644
index 0000000..f95de42
--- /dev/null
+++ b/flink-shaded-netty-tcnative-dynamic/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,9 @@
+flink-shaded-netty-openssl-static
+Copyright 2014-2019 The Apache Software Foundation
+
+This project includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+This project bundles the following dependencies under the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+- io.netty:netty-tcnative:2.0.25.Final
diff --git a/pom.xml b/pom.xml
index 515917b..843ad64 100644
--- a/pom.xml
+++ b/pom.xml
@@ -56,6 +56,7 @@ under the License.
<module>flink-shaded-asm-6</module>
<module>flink-shaded-guava-18</module>
<module>flink-shaded-netty-4</module>
+ <module>flink-shaded-netty-tcnative-dynamic</module>
<module>flink-shaded-jackson-parent</module>
<module>flink-shaded-hadoop-2</module>
<module>flink-shaded-hadoop-2-uber</module>
[flink-shaded] 01/04: [FLINK-11580][netty] add statically linked
netty-tcnative
Posted by ch...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
chesnay pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink-shaded.git
commit 4875bdc24d7d8a0076b65a11762fd49e536faaff
Author: Nico Kruber <ni...@data-artisans.com>
AuthorDate: Wed Jul 11 22:27:47 2018 +0200
[FLINK-11580][netty] add statically linked netty-tcnative
With one of the artifacts of netty-tcnative [1], this enables support for using
the OpenSSL engine instead of the one from the JDK. [2] claims significant
improvements (3x) over the JDK and we may just also save some resources.
From the available artifacts, this commits selects the one which is statically
linked against Google's BoringSSL. Please refer to [1] for further details.
Due to possible licensing issues with the openSSL license BoringSSL is based on,
we only build this module if the build profile
'include-netty-tcnative-static-Pinclude-netty-tcnative-static' is activated.
[1] http://netty.io/wiki/forked-tomcat-native.html
[2] https://netty.io/wiki/requirements-for-4.x.html#wiki-h4-4
---
flink-shaded-netty-tcnative-static/pom.xml | 110 +++++++++++++++++++++
.../src/main/resources/META-INF/NOTICE | 14 +++
pom.xml | 12 +++
3 files changed, 136 insertions(+)
diff --git a/flink-shaded-netty-tcnative-static/pom.xml b/flink-shaded-netty-tcnative-static/pom.xml
new file mode 100644
index 0000000..7c8fc97
--- /dev/null
+++ b/flink-shaded-netty-tcnative-static/pom.xml
@@ -0,0 +1,110 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-shaded</artifactId>
+ <version>7.0</version>
+ <relativePath>..</relativePath>
+ </parent>
+
+ <artifactId>flink-shaded-netty-tcnative-static</artifactId>
+ <name>flink-shaded-netty-tcnative-static</name>
+ <version>${netty.tcnative.version}-7.0</version>
+
+ <properties>
+ <netty.tcnative.version>2.0.25.Final</netty.tcnative.version>
+ </properties>
+
+ <dependencies>
+ <!-- Netty's native openSSL support -->
+ <dependency>
+ <groupId>io.netty</groupId>
+ <artifactId>netty-tcnative-boringssl-static</artifactId>
+ <version>${netty.tcnative.version}</version>
+ </dependency>
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>shade-flink</id>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <createDependencyReducedPom>true</createDependencyReducedPom>
+ <dependencyReducedPomLocation>${project.basedir}/target/dependency-reduced-pom.xml</dependencyReducedPomLocation>
+ <artifactSet>
+ <includes>
+ <include>io.netty:netty-tcnative-boringssl-static</include>
+ </includes>
+ </artifactSet>
+ <relocations>
+ <relocation>
+ <pattern>io.netty</pattern>
+ <shadedPattern>org.apache.flink.shaded.netty4.io.netty</shadedPattern>
+ </relocation>
+ </relocations>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-antrun-plugin</artifactId>
+ <version>1.7</version>
+ <executions>
+ <execution>
+ <id>rename-native-library</id>
+ <phase>package</phase>
+ <configuration>
+ <target>
+ <echo message="unpacking netty jar" />
+ <unzip src="${project.build.directory}/${artifactId}-${version}.jar" dest="${project.build.directory}/unpacked/" />
+ <echo message="renaming netty_tcnative library" />
+ <move todir="${project.build.directory}/unpacked/META-INF/native" includeemptydirs="false">
+ <fileset dir="${project.build.directory}/unpacked/META-INF/native"/>
+ <mapper type="regexp" from="(lib)?netty_tcnative_(linux_x86_64.so|osx_x86_64.jnilib|windows_x86_64.dll)" to="\1org_apache_flink_shaded_netty4_netty_tcnative_\2"/>
+ </move>
+ <echo message="repackaging netty jar" />
+ <jar destfile="${project.build.directory}/${artifactId}-${version}.jar" basedir="${project.build.directory}/unpacked" />
+ </target>
+ </configuration>
+ <goals>
+ <goal>run</goal>
+ </goals>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+
+
+</project>
diff --git a/flink-shaded-netty-tcnative-static/src/main/resources/META-INF/NOTICE b/flink-shaded-netty-tcnative-static/src/main/resources/META-INF/NOTICE
new file mode 100644
index 0000000..ef57f70
--- /dev/null
+++ b/flink-shaded-netty-tcnative-static/src/main/resources/META-INF/NOTICE
@@ -0,0 +1,14 @@
+flink-shaded-netty-openssl-static
+Copyright 2014-2019 The Apache Software Foundation
+
+This project includes software developed at
+The Apache Software Foundation (http://www.apache.org/).
+
+This project bundles the following dependencies under the Apache Software License 2.0 (http://www.apache.org/licenses/LICENSE-2.0.txt)
+
+- io.netty:netty-tcnative-boringssl-static:2.0.25.Final
+
+This project bundles the following dependencies under the OpenSSL license.
+See bundled license files for details.
+
+- BoringSSL (statically linked native libraries)
diff --git a/pom.xml b/pom.xml
index 4eb1837..515917b 100644
--- a/pom.xml
+++ b/pom.xml
@@ -63,6 +63,18 @@ under the License.
<profiles>
<profile>
+ <id>include-netty-tcnative-static</id>
+ <activation>
+ <property>
+ <name>include-netty-tcnative-static</name>
+ </property>
+ </activation>
+ <modules>
+ <module>flink-shaded-netty-tcnative-static</module>
+ </modules>
+ </profile>
+
+ <profile>
<id>release</id>
<build>
<plugins>