You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/11/13 10:55:39 UTC
[GitHub] [flink-connector-aws] dannycranmer opened a new pull request, #21: [FLINK-30008][Connectors/AWS] Upgrade to Flink 1.16.0 and add CI build
dannycranmer opened a new pull request, #21:
URL: https://github.com/apache/flink-connector-aws/pull/21
## What is the purpose of the change
Add support for Flink 1.16
## Brief change log
* Set default Flink version to 1.16.0
* Remove snapshot repo
* Add CI build profile for 1.16.0
## Verifying this change
Run locally
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): yes
- The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? n/a
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-aws] dannycranmer merged pull request #21: [FLINK-30008][Connectors/AWS] Upgrade to Flink 1.16.0 and add CI build
Posted by GitBox <gi...@apache.org>.
dannycranmer merged PR #21:
URL: https://github.com/apache/flink-connector-aws/pull/21
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-aws] hlteoh37 commented on a diff in pull request #21: [FLINK-30008][Connectors/AWS] Upgrade to Flink 1.16.0 and add CI build
Posted by GitBox <gi...@apache.org>.
hlteoh37 commented on code in PR #21:
URL: https://github.com/apache/flink-connector-aws/pull/21#discussion_r1021217406
##########
pom.xml:
##########
@@ -18,145 +18,239 @@ specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
-
- <modelVersion>4.0.0</modelVersion>
-
- <parent>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-connectors</artifactId>
- <version>1.15.2</version>
- <relativePath/>
- </parent>
-
- <artifactId>flink-connector-aws</artifactId>
- <version>1.0.0-SNAPSHOT</version>
-
- <name>Flink : Connectors : AWS : Parent</name>
- <packaging>pom</packaging>
- <url>https://flink.apache.org</url>
- <inceptionYear>2022</inceptionYear>
-
- <licenses>
- <license>
- <name>The Apache Software License, Version 2.0</name>
- <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
- <distribution>repo</distribution>
- </license>
- </licenses>
-
- <scm>
- <url>https://github.com/apache/flink-connector-aws</url>
- <connection>git@github.com:apache/flink-connector-aws.git</connection>
- <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/flink-connector-aws.git</developerConnection>
- </scm>
-
- <pluginRepositories>
- <pluginRepository>
- <!-- Allows exec-maven-plugin to resolve snapshot plugin dependencies -->
- <id>apache.snapshots.https</id>
- <name>${distMgmtSnapshotsName}</name>
- <url>${distMgmtSnapshotsUrl}</url>
- </pluginRepository>
- </pluginRepositories>
-
- <properties>
- <aws.sdk.version>2.17.247</aws.sdk.version>
- <flink.version>1.15.2</flink.version>
- </properties>
-
- <modules>
- <module>flink-connector-dynamodb</module>
- <module>flink-sql-connector-dynamodb</module>
- </modules>
-
- <dependencyManagement>
- <dependencies>
- <dependency>
- <groupId>software.amazon.awssdk</groupId>
- <artifactId>bom</artifactId>
- <version>${aws.sdk.version}</version>
- <type>pom</type>
- <scope>import</scope>
- </dependency>
-
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-test-utils-junit</artifactId>
- <version>${project.parent.version}</version>
- <scope>test</scope>
- </dependency>
-
- </dependencies>
- </dependencyManagement>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.codehaus.mojo</groupId>
- <artifactId>exec-maven-plugin</artifactId>
- <version>3.1.0</version>
- <inherited>false</inherited>
- <executions>
- <execution>
- <id>check-license</id>
- <!-- manually called -->
- <phase>none</phase>
- <goals>
- <goal>java</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <mainClass>org.apache.flink.tools.ci.licensecheck.LicenseChecker</mainClass>
- <includePluginDependencies>true</includePluginDependencies>
- <includeProjectDependencies>false</includeProjectDependencies>
- </configuration>
- <dependencies>
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-ci-tools</artifactId>
- <version>1.16-SNAPSHOT</version>
- </dependency>
- </dependencies>
- </plugin>
-
- <plugin>
- <!-- Override directory for parent project -->
- <groupId>org.commonjava.maven.plugins</groupId>
- <artifactId>directory-maven-plugin</artifactId>
- <executions>
- <execution>
- <id>directories</id>
- <goals>
- <goal>directory-of</goal>
- </goals>
- <configuration>
- <property>rootDir</property>
- <project>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-connector-aws</artifactId>
- </project>
- </configuration>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- <executions>
- <execution>
- <id>dependency-convergence</id>
- <goals>
- <goal>enforce</goal>
- </goals>
- <configuration>
- <skip>false</skip>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>io.github.zentol.flink</groupId>
+ <artifactId>flink-connector-parent</artifactId>
+ <version>1.0</version>
+ </parent>
+
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-connector-aws-parent</artifactId>
+ <version>1.0.0-SNAPSHOT</version>
+
+ <name>Flink : Connectors : AWS : Parent</name>
+ <packaging>pom</packaging>
+ <url>https://flink.apache.org</url>
+ <inceptionYear>2022</inceptionYear>
+
+ <licenses>
+ <license>
+ <name>The Apache Software License, Version 2.0</name>
+ <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
+ <distribution>repo</distribution>
+ </license>
+ </licenses>
+
+ <scm>
+ <url>https://github.com/apache/flink-connector-aws</url>
+ <connection>git@github.com:apache/flink-connector-aws.git</connection>
+ <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/flink-connector-aws.git</developerConnection>
+ </scm>
+
+ <properties>
+ <aws.sdk.version>2.17.247</aws.sdk.version>
+ <netty.version>4.1.77.Final</netty.version>
+ <flink.version>1.16.0</flink.version>
+ <flink.shaded.version>15.0</flink.shaded.version>
Review Comment:
Q. Why is this 15.0? We see this in the outputs
```
Warning: Could not find artifact org.apache.flink:flink-sql-connector-dynamodb:jar:1.15.0 in central (https://repo.maven.apache.org/maven2)
Warning: Could not resolve dependency with descriptor 'org.apache.flink:flink-sql-connector-dynamodb:1.15.0'.
Warning: Please provide at least one resolvable old version using one of the configuration elements <oldVersion/> or <oldVersions/>.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-aws] dannycranmer commented on a diff in pull request #21: [FLINK-30008][Connectors/AWS] Upgrade to Flink 1.16.0 and add CI build
Posted by GitBox <gi...@apache.org>.
dannycranmer commented on code in PR #21:
URL: https://github.com/apache/flink-connector-aws/pull/21#discussion_r1021638516
##########
pom.xml:
##########
@@ -18,145 +18,239 @@ specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
-
- <modelVersion>4.0.0</modelVersion>
-
- <parent>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-connectors</artifactId>
- <version>1.15.2</version>
- <relativePath/>
- </parent>
-
- <artifactId>flink-connector-aws</artifactId>
- <version>1.0.0-SNAPSHOT</version>
-
- <name>Flink : Connectors : AWS : Parent</name>
- <packaging>pom</packaging>
- <url>https://flink.apache.org</url>
- <inceptionYear>2022</inceptionYear>
-
- <licenses>
- <license>
- <name>The Apache Software License, Version 2.0</name>
- <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
- <distribution>repo</distribution>
- </license>
- </licenses>
-
- <scm>
- <url>https://github.com/apache/flink-connector-aws</url>
- <connection>git@github.com:apache/flink-connector-aws.git</connection>
- <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/flink-connector-aws.git</developerConnection>
- </scm>
-
- <pluginRepositories>
- <pluginRepository>
- <!-- Allows exec-maven-plugin to resolve snapshot plugin dependencies -->
- <id>apache.snapshots.https</id>
- <name>${distMgmtSnapshotsName}</name>
- <url>${distMgmtSnapshotsUrl}</url>
- </pluginRepository>
- </pluginRepositories>
-
- <properties>
- <aws.sdk.version>2.17.247</aws.sdk.version>
- <flink.version>1.15.2</flink.version>
- </properties>
-
- <modules>
- <module>flink-connector-dynamodb</module>
- <module>flink-sql-connector-dynamodb</module>
- </modules>
-
- <dependencyManagement>
- <dependencies>
- <dependency>
- <groupId>software.amazon.awssdk</groupId>
- <artifactId>bom</artifactId>
- <version>${aws.sdk.version}</version>
- <type>pom</type>
- <scope>import</scope>
- </dependency>
-
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-test-utils-junit</artifactId>
- <version>${project.parent.version}</version>
- <scope>test</scope>
- </dependency>
-
- </dependencies>
- </dependencyManagement>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.codehaus.mojo</groupId>
- <artifactId>exec-maven-plugin</artifactId>
- <version>3.1.0</version>
- <inherited>false</inherited>
- <executions>
- <execution>
- <id>check-license</id>
- <!-- manually called -->
- <phase>none</phase>
- <goals>
- <goal>java</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <mainClass>org.apache.flink.tools.ci.licensecheck.LicenseChecker</mainClass>
- <includePluginDependencies>true</includePluginDependencies>
- <includeProjectDependencies>false</includeProjectDependencies>
- </configuration>
- <dependencies>
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-ci-tools</artifactId>
- <version>1.16-SNAPSHOT</version>
- </dependency>
- </dependencies>
- </plugin>
-
- <plugin>
- <!-- Override directory for parent project -->
- <groupId>org.commonjava.maven.plugins</groupId>
- <artifactId>directory-maven-plugin</artifactId>
- <executions>
- <execution>
- <id>directories</id>
- <goals>
- <goal>directory-of</goal>
- </goals>
- <configuration>
- <property>rootDir</property>
- <project>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-connector-aws</artifactId>
- </project>
- </configuration>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- <executions>
- <execution>
- <id>dependency-convergence</id>
- <goals>
- <goal>enforce</goal>
- </goals>
- <configuration>
- <skip>false</skip>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>io.github.zentol.flink</groupId>
+ <artifactId>flink-connector-parent</artifactId>
+ <version>1.0</version>
+ </parent>
+
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-connector-aws-parent</artifactId>
+ <version>1.0.0-SNAPSHOT</version>
+
+ <name>Flink : Connectors : AWS : Parent</name>
+ <packaging>pom</packaging>
+ <url>https://flink.apache.org</url>
+ <inceptionYear>2022</inceptionYear>
+
+ <licenses>
+ <license>
+ <name>The Apache Software License, Version 2.0</name>
+ <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
+ <distribution>repo</distribution>
+ </license>
+ </licenses>
+
+ <scm>
+ <url>https://github.com/apache/flink-connector-aws</url>
+ <connection>git@github.com:apache/flink-connector-aws.git</connection>
+ <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/flink-connector-aws.git</developerConnection>
+ </scm>
+
+ <properties>
+ <aws.sdk.version>2.17.247</aws.sdk.version>
+ <netty.version>4.1.77.Final</netty.version>
+ <flink.version>1.16.0</flink.version>
+ <flink.shaded.version>15.0</flink.shaded.version>
Review Comment:
These things are unrelated. I have upgraded Flink shaded to 16.0. The warnings were from the `japicmp` plugin which is trying to compare binary compatibility. Given that this is the first version of the DDB connector there is no baseline to compare against. I have removed this plugin, we can add it back on the next version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [flink-connector-aws] dannycranmer commented on a diff in pull request #21: [FLINK-30008][Connectors/AWS] Upgrade to Flink 1.16.0 and add CI build
Posted by GitBox <gi...@apache.org>.
dannycranmer commented on code in PR #21:
URL: https://github.com/apache/flink-connector-aws/pull/21#discussion_r1021633172
##########
pom.xml:
##########
@@ -18,145 +18,239 @@ specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
-
- <modelVersion>4.0.0</modelVersion>
-
- <parent>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-connectors</artifactId>
- <version>1.15.2</version>
- <relativePath/>
- </parent>
-
- <artifactId>flink-connector-aws</artifactId>
- <version>1.0.0-SNAPSHOT</version>
-
- <name>Flink : Connectors : AWS : Parent</name>
- <packaging>pom</packaging>
- <url>https://flink.apache.org</url>
- <inceptionYear>2022</inceptionYear>
-
- <licenses>
- <license>
- <name>The Apache Software License, Version 2.0</name>
- <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
- <distribution>repo</distribution>
- </license>
- </licenses>
-
- <scm>
- <url>https://github.com/apache/flink-connector-aws</url>
- <connection>git@github.com:apache/flink-connector-aws.git</connection>
- <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/flink-connector-aws.git</developerConnection>
- </scm>
-
- <pluginRepositories>
- <pluginRepository>
- <!-- Allows exec-maven-plugin to resolve snapshot plugin dependencies -->
- <id>apache.snapshots.https</id>
- <name>${distMgmtSnapshotsName}</name>
- <url>${distMgmtSnapshotsUrl}</url>
- </pluginRepository>
- </pluginRepositories>
-
- <properties>
- <aws.sdk.version>2.17.247</aws.sdk.version>
- <flink.version>1.15.2</flink.version>
- </properties>
-
- <modules>
- <module>flink-connector-dynamodb</module>
- <module>flink-sql-connector-dynamodb</module>
- </modules>
-
- <dependencyManagement>
- <dependencies>
- <dependency>
- <groupId>software.amazon.awssdk</groupId>
- <artifactId>bom</artifactId>
- <version>${aws.sdk.version}</version>
- <type>pom</type>
- <scope>import</scope>
- </dependency>
-
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-test-utils-junit</artifactId>
- <version>${project.parent.version}</version>
- <scope>test</scope>
- </dependency>
-
- </dependencies>
- </dependencyManagement>
-
- <build>
- <plugins>
- <plugin>
- <groupId>org.codehaus.mojo</groupId>
- <artifactId>exec-maven-plugin</artifactId>
- <version>3.1.0</version>
- <inherited>false</inherited>
- <executions>
- <execution>
- <id>check-license</id>
- <!-- manually called -->
- <phase>none</phase>
- <goals>
- <goal>java</goal>
- </goals>
- </execution>
- </executions>
- <configuration>
- <mainClass>org.apache.flink.tools.ci.licensecheck.LicenseChecker</mainClass>
- <includePluginDependencies>true</includePluginDependencies>
- <includeProjectDependencies>false</includeProjectDependencies>
- </configuration>
- <dependencies>
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-ci-tools</artifactId>
- <version>1.16-SNAPSHOT</version>
- </dependency>
- </dependencies>
- </plugin>
-
- <plugin>
- <!-- Override directory for parent project -->
- <groupId>org.commonjava.maven.plugins</groupId>
- <artifactId>directory-maven-plugin</artifactId>
- <executions>
- <execution>
- <id>directories</id>
- <goals>
- <goal>directory-of</goal>
- </goals>
- <configuration>
- <property>rootDir</property>
- <project>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-connector-aws</artifactId>
- </project>
- </configuration>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-enforcer-plugin</artifactId>
- <executions>
- <execution>
- <id>dependency-convergence</id>
- <goals>
- <goal>enforce</goal>
- </goals>
- <configuration>
- <skip>false</skip>
- </configuration>
- </execution>
- </executions>
- </plugin>
- </plugins>
- </build>
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
+
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>io.github.zentol.flink</groupId>
+ <artifactId>flink-connector-parent</artifactId>
+ <version>1.0</version>
+ </parent>
+
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-connector-aws-parent</artifactId>
+ <version>1.0.0-SNAPSHOT</version>
+
+ <name>Flink : Connectors : AWS : Parent</name>
+ <packaging>pom</packaging>
+ <url>https://flink.apache.org</url>
+ <inceptionYear>2022</inceptionYear>
+
+ <licenses>
+ <license>
+ <name>The Apache Software License, Version 2.0</name>
+ <url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
+ <distribution>repo</distribution>
+ </license>
+ </licenses>
+
+ <scm>
+ <url>https://github.com/apache/flink-connector-aws</url>
+ <connection>git@github.com:apache/flink-connector-aws.git</connection>
+ <developerConnection>scm:git:https://gitbox.apache.org/repos/asf/flink-connector-aws.git</developerConnection>
+ </scm>
+
+ <properties>
+ <aws.sdk.version>2.17.247</aws.sdk.version>
+ <netty.version>4.1.77.Final</netty.version>
+ <flink.version>1.16.0</flink.version>
+ <flink.shaded.version>15.0</flink.shaded.version>
Review Comment:
I copied this from elasticsearch actually. But I do see a 16.0 available. https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-force-shading
Not sure on the warnings, will take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org