You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2023/05/09 10:56:05 UTC
[inlong] branch branch-1.7 updated (d7cede714 -> 229f49a80)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a change to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/inlong.git
from d7cede714 [INLONG-7968][Release] Change the tag of Docker images to 1.7.0 (#7980)
new 0028cfb81 [INLONG-7970][Sort] Fix read metadata filed failed of mongodb cdc (#7975)
new a10ac7ab1 [INLONG-7983][Sort] Remove the duplicate dependency (#7984)
new 229f49a80 [INLONG-7968][Docker] Add JDBC connector jar for test (#7988)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.md | 3 ++-
inlong-manager/manager-docker/Dockerfile | 6 ++++--
.../sort-connectors/elasticsearch-base/pom.xml | 20 +++++++-------------
.../cdc/mongodb/table/MongoDBReadableMetadata.java | 7 +++++--
4 files changed, 18 insertions(+), 18 deletions(-)
[inlong] 03/03: [INLONG-7968][Docker] Add JDBC connector jar for test (#7988)
Posted by do...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/inlong.git
commit 229f49a80e0eea63f8fe4ce54f1f501377b5da05
Author: Charles Zhang <do...@apache.org>
AuthorDate: Tue May 9 18:54:08 2023 +0800
[INLONG-7968][Docker] Add JDBC connector jar for test (#7988)
---
CHANGES.md | 3 ++-
inlong-manager/manager-docker/Dockerfile | 6 ++++--
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index 6861df5b5..908a75a46 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -196,4 +196,5 @@
| [INLONG-7745](https://github.com/apache/inlong/issues/7745) | [Improve][CVE] Spring Framework vulnerable to denial of service via specially crafted SpEL expression |
| [INLONG-7742](https://github.com/apache/inlong/issues/7742) | [Improve][Tool] Only support inlong-dev-toolkit on MacOS and Linux |
| [INLONG-7698](https://github.com/apache/inlong/issues/7698) | [Bug][Docker] docker images build faild for tubemq-build |
-| [INLONG-7697](https://github.com/apache/inlong/issues/7697) | [Feature] Reduce the memory usage of JM when split table chunks |
\ No newline at end of file
+| [INLONG-7697](https://github.com/apache/inlong/issues/7697) | [Feature] Reduce the memory usage of JM when split table chunks |
+| [INLONG-7986](https://github.com/apache/inlong/issues/7986) | [Bug] Collect file to Postgresql failed when using docker image for test |
\ No newline at end of file
diff --git a/inlong-manager/manager-docker/Dockerfile b/inlong-manager/manager-docker/Dockerfile
index 825ed00c8..240ca7583 100644
--- a/inlong-manager/manager-docker/Dockerfile
+++ b/inlong-manager/manager-docker/Dockerfile
@@ -41,9 +41,13 @@ ENV MANAGER_JVM_HEAP_OPTS="-XX:+UseContainerSupport -XX:InitialRAMPercentage=40.
WORKDIR /opt/inlong-manager
# add manager tarball
ARG VERSION
+ARG default_jdbc_connector_url=https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.27/mysql-connector-java-8.0.27.jar
ADD target/manager-web-${VERSION}-bin.tar.gz /opt/inlong-manager
+# add mysql connector
+RUN wget -P lib/ ${default_jdbc_connector_url}
# add sort resource
ADD target/sort-dist-${VERSION}.jar /opt/inlong-sort/
+RUN wget -P /opt/inlong-sort/connectors/ ${default_jdbc_connector_url}
ADD target/sort-connector-pulsar-${VERSION}.jar /opt/inlong-sort/connectors/
ADD target/sort-connector-kafka-${VERSION}.jar /opt/inlong-sort/connectors/
ADD target/sort-connector-jdbc-${VERSION}.jar /opt/inlong-sort/connectors/
@@ -53,8 +57,6 @@ ENV AUDIT_CK_URL=127.0.0.1:8123
ENV AUDIT_CK_USERNAME=default
ENV AUDIT_CK_PASSWD=default
ENV AUDIT_CK_DBNAME="apache_inlong_audit"
-# add mysql connector
-RUN wget -P lib/ https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.27/mysql-connector-java-8.0.27.jar
ADD manager-docker.sh bin/
RUN chmod +x bin/manager-docker.sh
CMD ["bin/manager-docker.sh"]
[inlong] 02/03: [INLONG-7983][Sort] Remove the duplicate dependency (#7984)
Posted by do...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/inlong.git
commit a10ac7ab175d2d809860a1c1d227b67ea6eb69d6
Author: haifxu <xh...@gmail.com>
AuthorDate: Tue May 9 18:53:57 2023 +0800
[INLONG-7983][Sort] Remove the duplicate dependency (#7984)
---
.../sort-connectors/elasticsearch-base/pom.xml | 20 +++++++-------------
1 file changed, 7 insertions(+), 13 deletions(-)
diff --git a/inlong-sort/sort-connectors/elasticsearch-base/pom.xml b/inlong-sort/sort-connectors/elasticsearch-base/pom.xml
index 0faededfd..a30ac80ca 100644
--- a/inlong-sort/sort-connectors/elasticsearch-base/pom.xml
+++ b/inlong-sort/sort-connectors/elasticsearch-base/pom.xml
@@ -35,6 +35,13 @@
<version>${project.version}</version>
</dependency>
+ <!-- Elasticsearch table sink factory testing -->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-json</artifactId>
+ <version>${flink.version}</version>
+ </dependency>
+
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java_2.12</artifactId>
@@ -108,14 +115,6 @@
<version>${project.version}</version>
</dependency>
- <!-- Elasticsearch table sink factory testing -->
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-json</artifactId>
- <version>${flink.version}</version>
- <scope>test</scope>
- </dependency>
-
<!--
Including Log4j2 dependencies for tests is required for the
embedded Elasticsearch nodes used in tests to run correctly.
@@ -131,11 +130,6 @@
<artifactId>log4j-core</artifactId>
<scope>test</scope>
</dependency>
- <dependency>
- <groupId>org.apache.flink</groupId>
- <artifactId>flink-json</artifactId>
- <scope>provided</scope>
- </dependency>
</dependencies>
<build>
[inlong] 01/03: [INLONG-7970][Sort] Fix read metadata filed failed of mongodb cdc (#7975)
Posted by do...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch branch-1.7
in repository https://gitbox.apache.org/repos/asf/inlong.git
commit 0028cfb815ac83a45aaa2eb64146b5c1450be87d
Author: emhui <11...@users.noreply.github.com>
AuthorDate: Tue May 9 14:19:01 2023 +0800
[INLONG-7970][Sort] Fix read metadata filed failed of mongodb cdc (#7975)
---
.../inlong/sort/cdc/mongodb/table/MongoDBReadableMetadata.java | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/inlong-sort/sort-connectors/mongodb-cdc/src/main/java/org/apache/inlong/sort/cdc/mongodb/table/MongoDBReadableMetadata.java b/inlong-sort/sort-connectors/mongodb-cdc/src/main/java/org/apache/inlong/sort/cdc/mongodb/table/MongoDBReadableMetadata.java
index 0fa28fe37..a2dd092eb 100644
--- a/inlong-sort/sort-connectors/mongodb-cdc/src/main/java/org/apache/inlong/sort/cdc/mongodb/table/MongoDBReadableMetadata.java
+++ b/inlong-sort/sort-connectors/mongodb-cdc/src/main/java/org/apache/inlong/sort/cdc/mongodb/table/MongoDBReadableMetadata.java
@@ -30,6 +30,7 @@ import javax.annotation.Nullable;
import org.apache.flink.table.api.DataTypes;
import org.apache.flink.table.data.RowData;
+import org.apache.flink.table.data.StringData;
import org.apache.flink.table.data.TimestampData;
import org.apache.flink.table.types.DataType;
import org.apache.inlong.sort.cdc.mongodb.debezium.table.MetadataConverter;
@@ -48,7 +49,8 @@ public enum MongoDBReadableMetadata {
@Override
public Object read(SourceRecord record) {
- return getMetaData(record, MongoDBEnvelope.NAMESPACE_COLLECTION_FIELD);
+ return StringData.fromString(
+ getMetaData(record, MongoDBEnvelope.NAMESPACE_COLLECTION_FIELD));
}
}),
@@ -61,7 +63,8 @@ public enum MongoDBReadableMetadata {
@Override
public Object read(SourceRecord record) {
- return getMetaData(record, MongoDBEnvelope.NAMESPACE_DATABASE_FIELD);
+ return StringData.fromString(
+ getMetaData(record, MongoDBEnvelope.NAMESPACE_DATABASE_FIELD));
}
}),