You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "hackergin (via GitHub)" <gi...@apache.org> on 2025/05/22 05:38:48 UTC
[PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
hackergin opened a new pull request, #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201
<!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! -->
## Purpose of the pull request
Add a new task plugin for refresh flink materialized table.
https://github.com/apache/dolphinscheduler/issues/17086
## Brief change log
## Verify this pull request
- *Added dolphinscheduler flink materialized table end to end test*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2980833804
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "github-advanced-security[bot] (via GitHub)" <gi...@apache.org>.
github-advanced-security[bot] commented on code in PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#discussion_r2103269456
##########
dolphinscheduler-task-plugin/dolphinscheduler-task-flink-materialized-table/src/main/java/org/apache/dolphinscheduler/plugin/task/flink/gateway/FlinkSqlClient.java:
##########
@@ -0,0 +1,237 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.plugin.task.flink.gateway;
+
+import org.apache.dolphinscheduler.plugin.task.flink.gateway.model.FetchResultResponseBody;
+import org.apache.dolphinscheduler.plugin.task.flink.gateway.model.JobStatus;
+import org.apache.dolphinscheduler.plugin.task.flink.gateway.model.RefreshMaterializedTableRequest;
+
+import org.apache.http.HttpEntity;
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.apache.http.util.EntityUtils;
+
+import java.io.IOException;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import jline.internal.Log;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+
+/**
+ * Client for interacting with Flink SQL Gateway.
+ */
+public class FlinkSqlClient {
+
+ private static final Logger log = LoggerFactory.getLogger(FlinkSqlClient.class);
+
+ private static final int MAX_RETRIES = 30;
+ private static final int RETRY_INTERVAL_SECONDS = 2;
+ private static final ObjectMapper objectMapper = new ObjectMapper();
+
+ private final String baseUrl;
+ private final CloseableHttpClient httpClient;
+
+ /**
+ * Constructs a new FlinkSqlClient with the specified base URL.
+ *
+ * @param baseUrl The base URL of the Flink SQL Gateway
+ */
+ public FlinkSqlClient(String baseUrl) {
+ this.baseUrl = baseUrl;
+ this.httpClient = HttpClients.createDefault();
+ }
+
+ /**
+ * Opens a new session with the Flink SQL Gateway.
+ *
+ * @param config Configuration parameters for the session
+ * @return The session handle
+ * @throws IOException if an I/O error occurs
+ */
+ public String openSession(Map<String, String> config) throws IOException {
+ String url = baseUrl + "/v3/sessions";
+ HttpPost request = new HttpPost(url);
+ request.setHeader("Content-Type", "application/json");
+ if (config != null) {
+ request.setEntity(new StringEntity(objectMapper.writeValueAsString(config)));
+ }
+
+ try (CloseableHttpResponse response = httpClient.execute(request)) {
+ HttpEntity entity = response.getEntity();
+ String responseBody = EntityUtils.toString(entity);
+ log.info("open session response: ", responseBody);
Review Comment:
## Unused format argument
This format call refers to 0 argument(s) but supplies 1 argument(s).
[Show more details](https://github.com/apache/dolphinscheduler/security/code-scanning/5178)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2903100938
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [48.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2977398474
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2927654119
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-3017457185
1. You can ignore `OWASP Dependentcy Check` for now.
> Code coverage is below threshold. Not sure if this is required since I noticed other similar tasks don't seem to have corresponding tests either.
Yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2976757615
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2976759066
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS closed pull request #17201: [DSIP-90][plugin] Introduce new task plugin for Flink materialized table
URL: https://github.com/apache/dolphinscheduler/pull/17201
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2928630023
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2902396305
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [48.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2927653649
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "hackergin (via GitHub)" <gi...@apache.org>.
hackergin commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2998472413
Two CI issues:
1. dependency check is failing with `Cannot construct instance of ModifiedCiaType, problem: SAFETY` - seems like a tool compatibility issue with new NVD data, not related to my changes.
2. Code coverage is below threshold. Not sure if this is required since I noticed other similar tasks don't seem to have corresponding tests either.
@SbloodyS Should I address the coverage issue or is it OK to proceed as-is?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2928624627
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2903099952
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [48.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2900504871
Please check the failed CI. @hackergin
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2902397412
Please retry analysis of this Pull-Request directly on SonarQube Cloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2899970788
Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "lsyldliu (via GitHub)" <gi...@apache.org>.
lsyldliu commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2900070264
Hi, @ruanwenjun, @SbloodyS can you help review this pr?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#discussion_r2108234197
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/flink-materialized-table/catalog_store/mydb/full_users_shops/schema/full_users_shops_schema.json:
##########
@@ -0,0 +1,24 @@
+{
+ "tableKind" : "MATERIALIZED_TABLE",
+ "catalogTableInfo" : {
+ "schema.0.data-type" : "BIGINT",
+ "definition-query" : "SELECT `tmp`.`user_id`, `tmp`.`ds`, SUM(`tmp`.`payment_amount_cents`) AS `payed_buy_fee_sum`, SUM(1) AS `PV`\nFROM (SELECT `json_source`.`user_id`, `json_source`.`order_created_at` AS `ds`, `json_source`.`payment_amount_cents`\nFROM `mt_cat`.`mydb`.`json_source` AS `json_source`) AS `tmp`\nGROUP BY `tmp`.`user_id`, `tmp`.`ds`",
+ "partition.fields.ds.date-formatter" : "yyyy-MM-dd",
+ "schema.2.name" : "payed_buy_fee_sum",
+ "format" : "json",
+ "refresh-status" : "ACTIVATED",
+ "refresh-handler-bytes" : "rO0ABXNyAD5vcmcuYXBhY2hlLmZsaW5rLnRhYmxlLmdhdGV3YXkud29ya2Zsb3cuRW1iZWRkZWRSZWZyZXNoSGFuZGxlcgAAAAAAAAABAgACTAANd29ya2Zsb3dHcm91cHQAEkxqYXZhL2xhbmcvU3RyaW5nO0wADHdvcmtmbG93TmFtZXEAfgABeHB0AA1kZWZhdWx0X2dyb3VwdAAtcXVhcnR6X2pvYl9gbXRfY2F0YC5gbXlkYmAuYGZ1bGxfdXNlcnNfc2hvcHNg",
+ "schema.1.name" : "ds",
+ "freshness-interval" : "1",
+ "refresh-mode" : "FULL",
+ "schema.1.data-type" : "VARCHAR(2147483647)",
+ "partition.keys.0.name" : "ds",
+ "schema.3.data-type" : "INT NOT NULL",
+ "schema.2.data-type" : "BIGINT",
+ "schema.3.name" : "PV",
+ "freshness-unit" : "MINUTE",
+ "refresh-handler-desc" : "{\n workflowName: quartz_job_`mt_cat`.`mydb`.`full_users_shops`,\n workflowGroup: default_group\n}",
+ "logical-refresh-mode" : "FULL",
+ "schema.0.name" : "user_id"
+ }
+}
Review Comment:
Please leave a blank line in the last line.
##########
dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/java/org/apache/dolphinscheduler/e2e/cases/WorkflowFlinkMaterializedTableE2ETest.java:
##########
@@ -0,0 +1,162 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.e2e.cases;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import org.apache.dolphinscheduler.e2e.core.DolphinScheduler;
+import org.apache.dolphinscheduler.e2e.pages.LoginPage;
+import org.apache.dolphinscheduler.e2e.pages.common.NavBarPage;
+import org.apache.dolphinscheduler.e2e.pages.project.ProjectDetailPage;
+import org.apache.dolphinscheduler.e2e.pages.project.ProjectPage;
+import org.apache.dolphinscheduler.e2e.pages.project.workflow.WorkflowDefinitionTab;
+import org.apache.dolphinscheduler.e2e.pages.project.workflow.WorkflowForm;
+import org.apache.dolphinscheduler.e2e.pages.project.workflow.WorkflowInstanceTab;
+import org.apache.dolphinscheduler.e2e.pages.project.workflow.task.FlinkMaterializedTableTaskForm;
+import org.apache.dolphinscheduler.e2e.pages.security.SecurityPage;
+import org.apache.dolphinscheduler.e2e.pages.security.TenantPage;
+import org.apache.dolphinscheduler.e2e.pages.security.UserPage;
+
+import java.time.Duration;
+
+import org.junit.jupiter.api.AfterAll;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Order;
+import org.junit.jupiter.api.Test;
+import org.openqa.selenium.By;
+import org.openqa.selenium.remote.RemoteWebDriver;
+import org.openqa.selenium.support.ui.ExpectedConditions;
+import org.openqa.selenium.support.ui.WebDriverWait;
+import org.testcontainers.shaded.org.awaitility.Awaitility;
+
+@DolphinScheduler(composeFiles = "docker/flink-materialized-table/docker-compose.yaml")
+class WorkflowFlinkMaterializedTableE2ETest {
+
Review Comment:
You should add this test case to `https://github.com/apache/dolphinscheduler/blob/dev/.github/workflows/e2e.yml` to activate it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2980829513
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2974149437
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [DSIP-90][plugin] Introduce new task plugin for Flink materialized table [dolphinscheduler]
Posted by "sonarqubecloud[bot] (via GitHub)" <gi...@apache.org>.
sonarqubecloud[bot] commented on PR #17201:
URL: https://github.com/apache/dolphinscheduler/pull/17201#issuecomment-2974146641
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201) **Quality Gate failed**
Failed conditions
 [53.8% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=17201&metric=new_coverage&view=list) (required ≥ 60%)
[See analysis details on SonarQube Cloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=17201)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org