You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/06/14 10:05:05 UTC
[GitHub] [dolphinscheduler] SbloodyS opened a new pull request, #10442: [Feature-10411] Add tenant api test
SbloodyS opened a new pull request, #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442
<!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
## Purpose of the pull request
Related issue: #10411
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS merged pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS merged PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897536469
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/pom.xml:
##########
@@ -0,0 +1,40 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to Apache Software Foundation (ASF) under one or more contributor
+ ~ license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright
+ ~ ownership. Apache Software Foundation (ASF) licenses this file to you under
+ ~ the Apache License, Version 2.0 (the "License"); you may
+ ~ not use this file except in compliance with the License.
+ ~ You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>dolphinscheduler-api-test</artifactId>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <version>1.0-SNAPSHOT</version>
Review Comment:
> Should we use `dev-SNAPSHOT` in this? @kezhenxu94 @caishunfeng @zhongjiajie
I don't think it necessary. `-api-test` and `-e2e` are out of the project hierarchy of the main project so we don't bother to make it consistent
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897561396
##########
dolphinscheduler-api-test/README.md:
##########
@@ -0,0 +1,44 @@
+# DolphinScheduler Backend API Test
+
+## Page Object Model
+
+DolphinScheduler API test respects
+the [Page Object Model (POM)](https://www.selenium.dev/documentation/guidelines/page_object_models/) design pattern.
+Every page of DolphinScheduler's api is abstracted into a class for better maintainability.
+
+### Example
+
+The login page's api is abstracted
+as [`LoginPage`](dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/pages/LoginPage.java), with the
+following fields,
+
+```java
+public HttpResponse login(String username, String password) {
+ Map<String, Object> params = new HashMap<>();
+
+ params.put("userName", username);
+ params.put("userPassword", password);
+
+ RequestClient requestClient = new RequestClient();
+
+ return requestClient.post("/login", null, params);
+}
+```
+
+where `userName`, `userPassword` are the main elements on UI that we are interested in.
+
+## Test Environment Setup
+
+DolphinScheduler End-to-End test uses [testcontainers](https://www.testcontainers.org) to set up the testing
Review Comment:
Fixed.
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/utils/RequestClient.java:
##########
@@ -0,0 +1,171 @@
+/*
+ * Licensed to Apache Software Foundation (ASF) under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Apache Software Foundation (ASF) licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.utils;
+
+import lombok.SneakyThrows;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.dolphinscheduler.api.test.core.Constants;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponse;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponseBody;
+import org.testcontainers.shaded.okhttp3.FormBody;
+import org.testcontainers.shaded.okhttp3.Headers;
+import org.testcontainers.shaded.okhttp3.MediaType;
+import org.testcontainers.shaded.okhttp3.OkHttpClient;
+import org.testcontainers.shaded.okhttp3.Request;
+import org.testcontainers.shaded.okhttp3.RequestBody;
+import org.testcontainers.shaded.okhttp3.Response;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+@Slf4j
+public class RequestClient {
+
+ private OkHttpClient httpClient = null;
+
+ public RequestClient() {
+ this.httpClient = new OkHttpClient();
+ }
+
+ @SneakyThrows
+ public HttpResponse get(String url, Map<String, String> headers, Map<String, Object> params) {
+ String requestUrl = String.format("%s%s%s", Constants.DOLPHINSCHEDULER_API_URL, url, getParams(params));
+
+ Headers headersBuilder = new Headers.Builder().build();
+ if (headers != null) {
+ headersBuilder = Headers.of(headers);
+ }
+
+ LOGGER.info("GET request to {}, Headers: {}", requestUrl, headersBuilder);
+ Request request = new Request.Builder()
+ .url(requestUrl)
+ .headers(headersBuilder)
+ .get()
+ .build();
+
+ Response response = this.httpClient.newCall(request).execute();
+
+ HttpResponseBody responseData = null;
+ int responseCode = response.code();
+ if (response.body() != null) {
+ responseData = JSONUtils.parseObject(response.body().string(), HttpResponseBody.class);
+ }
+ response.close();
+
+ HttpResponse httpResponse = new HttpResponse(responseCode, responseData);
+
+ LOGGER.info("GET response: {}", httpResponse);
+
+ return httpResponse;
+ }
+
+ public static String getParams(Map<String, Object> params) {
+ StringBuilder sb = new StringBuilder("?");
+ if (params.size() > 0) {
+ for (Map.Entry<String, Object> item : params.entrySet()) {
+ Object value = item.getValue();
+ if (Objects.nonNull(value)) {
+ sb.append("&");
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897484969
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/utils/JSONUtils.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.utils;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import static com.fasterxml.jackson.databind.DeserializationFeature.ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT;
+import static com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES;
+import static com.fasterxml.jackson.databind.DeserializationFeature.READ_UNKNOWN_ENUM_VALUES_AS_NULL;
+import static com.fasterxml.jackson.databind.MapperFeature.REQUIRE_SETTERS_FOR_GETTERS;
+
+import java.io.IOException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.TimeZone;
+
+import org.apache.dolphinscheduler.api.test.core.Constants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.DeserializationContext;
+import com.fasterxml.jackson.databind.JsonDeserializer;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.JsonSerializer;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.ObjectWriter;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.SerializerProvider;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.fasterxml.jackson.databind.node.TextNode;
+import com.fasterxml.jackson.databind.type.CollectionType;
+import org.testcontainers.shaded.org.apache.commons.lang.StringUtils;
+
+/**
+ * json utils
+ */
+public class JSONUtils {
Review Comment:
I think test modules should not rely on actual production code. So as to avoid test exceptions caused by bugs in the production code. At the same time, reducing the dependence on the production code can reduce the compilation time and the size of the compressed tar package.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897561502
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/utils/RequestClient.java:
##########
@@ -0,0 +1,171 @@
+/*
+ * Licensed to Apache Software Foundation (ASF) under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Apache Software Foundation (ASF) licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.utils;
+
+import lombok.SneakyThrows;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.dolphinscheduler.api.test.core.Constants;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponse;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponseBody;
+import org.testcontainers.shaded.okhttp3.FormBody;
+import org.testcontainers.shaded.okhttp3.Headers;
+import org.testcontainers.shaded.okhttp3.MediaType;
+import org.testcontainers.shaded.okhttp3.OkHttpClient;
+import org.testcontainers.shaded.okhttp3.Request;
+import org.testcontainers.shaded.okhttp3.RequestBody;
+import org.testcontainers.shaded.okhttp3.Response;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+@Slf4j
+public class RequestClient {
+
+ private OkHttpClient httpClient = null;
+
+ public RequestClient() {
+ this.httpClient = new OkHttpClient();
+ }
+
+ @SneakyThrows
+ public HttpResponse get(String url, Map<String, String> headers, Map<String, Object> params) {
+ String requestUrl = String.format("%s%s%s", Constants.DOLPHINSCHEDULER_API_URL, url, getParams(params));
+
+ Headers headersBuilder = new Headers.Builder().build();
+ if (headers != null) {
+ headersBuilder = Headers.of(headers);
+ }
+
+ LOGGER.info("GET request to {}, Headers: {}", requestUrl, headersBuilder);
+ Request request = new Request.Builder()
+ .url(requestUrl)
+ .headers(headersBuilder)
+ .get()
+ .build();
+
+ Response response = this.httpClient.newCall(request).execute();
+
+ HttpResponseBody responseData = null;
+ int responseCode = response.code();
+ if (response.body() != null) {
+ responseData = JSONUtils.parseObject(response.body().string(), HttpResponseBody.class);
+ }
+ response.close();
+
+ HttpResponse httpResponse = new HttpResponse(responseCode, responseData);
+
+ LOGGER.info("GET response: {}", httpResponse);
+
+ return httpResponse;
+ }
+
+ public static String getParams(Map<String, Object> params) {
+ StringBuilder sb = new StringBuilder("?");
+ if (params.size() > 0) {
+ for (Map.Entry<String, Object> item : params.entrySet()) {
+ Object value = item.getValue();
+ if (Objects.nonNull(value)) {
+ sb.append("&");
+ sb.append(item.getKey());
+ sb.append("=");
Review Comment:
Fixed.
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-core/src/main/java/org/apache/dolphinscheduler/api/test/core/DolphinSchedulerExtension.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to Apache Software Foundation (ASF) under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Apache Software Foundation (ASF) licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.core;
+
+import java.io.File;
+import java.net.URL;
+import java.time.Duration;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import org.junit.jupiter.api.extension.AfterAllCallback;
+import org.junit.jupiter.api.extension.BeforeAllCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import org.testcontainers.containers.DockerComposeContainer;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+final class DolphinSchedulerExtension implements BeforeAllCallback, AfterAllCallback {
+ private final boolean LOCAL_MODE = Objects.equals(System.getProperty("local"), "true");
+
+ private final String SERVICE_NAME = "dolphinscheduler_1";
+
+ private DockerComposeContainer<?> compose;
+
+ @Override
+ public void beforeAll(ExtensionContext context) {
+ if (!LOCAL_MODE) {
+ compose = createDockerCompose(context);
+ compose.start();
+ }
+ }
+
+ @Override
+ public void afterAll(ExtensionContext context) {
+ if (compose != null) {
+ compose.stop();
+ }
+ }
+
+ private DockerComposeContainer<?> createDockerCompose(ExtensionContext context) {
+ final Class<?> clazz = context.getRequiredTestClass();
+ final DolphinScheduler annotation = clazz.getAnnotation(DolphinScheduler.class);
+ final List<File> files = Stream.of(annotation.composeFiles())
+ .map(it -> DolphinScheduler.class.getClassLoader().getResource(it))
+ .filter(Objects::nonNull)
+ .map(URL::getPath)
+ .map(File::new)
+ .collect(Collectors.toList());
+
+ compose = new DockerComposeContainer<>(files)
+ .withPull(true)
+ .withTailChildContainers(true)
+ .withLogConsumer(SERVICE_NAME, outputFrame -> LOGGER.info(outputFrame.getUtf8String()))
+ .waitingFor(SERVICE_NAME, Wait.forHealthcheck().withStartupTimeout(Duration.ofSeconds(180)));
Review Comment:
Fixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1156032291
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10442)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=coverage&view=list) No Coverage information
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=duplicated_lines_density&view=list) No Duplication information
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155068114
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10442)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL) [26 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_coverage&view=list) [14.8% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155921777
> LGTM overall, but it seem some of the files are or smilar to module dolphinscheduler-e2e, I wonder if there exists some way to combine them into some parent module?
The two modules are completely functionally unrelated. API testing only uses the basic framework of E2E for reference and starts third-party services based on docker.
I have discussed this issue with @kezhenxu94 . So I created a new ```API-Test``` module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155926150
> Hi @SbloodyS , can you check whether we can refactor the existing `dolphinscheduler-e2e` to some modules, `dolphinscheduler-testing` containing the `core` part, and let `dolphinscheduler-e2e-test` and `dolphinscheduler-api-test` depends on `dolphinscheduler-testing`?
I'll take a look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155954164
After a deep checking. I found the following differences in both core module's ```DolphinSchedulerExtension.java```.
1. The ```beforeAll``` and ```afterAll``` function would set ```webdriver``` and ``` browser``` in ```E2E-Test``` module. However, ```API-Test``` modules do not need these.
2. Based on the first point, there is no need to pull other additional containers in the ```API-Test``` module for local mode and non local mode. So there is no problem of distinguishing container images of M1 chip.
As we can see, the implementation of the ```core``` module of the ```API-Test``` and ```E2E-Test``` is quite different. So if we want to combine these two modules into one. I think we need to create an abstract class. And then implement its own logic in the ```core``` of the two modules.
I'm not sure whether we need to do this since it's not much different from now. WDYT? @zhongjiajie @kezhenxu94
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897552641
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/resources/docker/file-manage/common.properties:
##########
@@ -0,0 +1,90 @@
+#
Review Comment:
It only need to when the test covers the modified function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897482748
##########
dolphinscheduler-api-test/README.md:
##########
@@ -0,0 +1,44 @@
+# DolphinScheduler Backend API Test
+
+## Page Object Model
+
+DolphinScheduler API test respects
+the [Page Object Model (POM)](https://www.selenium.dev/documentation/guidelines/page_object_models/) design pattern.
+Every page of DolphinScheduler's api is abstracted into a class for better maintainability.
+
+### Example
+
+The login page's api is abstracted
+as [`LoginPage`](dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/pages/LoginPage.java), with the
+following fields,
+
+```java
+public HttpResponse login(String username, String password) {
+ Map<String, Object> params = new HashMap<>();
+
+ params.put("userName", username);
+ params.put("userPassword", password);
+
+ RequestClient requestClient = new RequestClient();
+
+ return requestClient.post("/login", null, params);
+}
+```
+
+where `userName`, `userPassword` are the main elements on UI that we are interested in.
+
+## Test Environment Setup
+
+DolphinScheduler End-to-End test uses [testcontainers](https://www.testcontainers.org) to set up the testing
Review Comment:
No. They are not related.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897533832
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/pom.xml:
##########
@@ -0,0 +1,40 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to Apache Software Foundation (ASF) under one or more contributor
+ ~ license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright
+ ~ ownership. Apache Software Foundation (ASF) licenses this file to you under
+ ~ the Apache License, Version 2.0 (the "License"); you may
+ ~ not use this file except in compliance with the License.
+ ~ You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>dolphinscheduler-api-test</artifactId>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <version>1.0-SNAPSHOT</version>
Review Comment:
Should we use ```dev-SNAPSHOT``` in this? @kezhenxu94 @caishunfeng @zhongjiajie
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] caishunfeng commented on a diff in pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
caishunfeng commented on code in PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#discussion_r897478087
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/utils/JSONUtils.java:
##########
@@ -0,0 +1,384 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.utils;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import static com.fasterxml.jackson.databind.DeserializationFeature.ACCEPT_EMPTY_ARRAY_AS_NULL_OBJECT;
+import static com.fasterxml.jackson.databind.DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES;
+import static com.fasterxml.jackson.databind.DeserializationFeature.READ_UNKNOWN_ENUM_VALUES_AS_NULL;
+import static com.fasterxml.jackson.databind.MapperFeature.REQUIRE_SETTERS_FOR_GETTERS;
+
+import java.io.IOException;
+import java.text.SimpleDateFormat;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.TimeZone;
+
+import org.apache.dolphinscheduler.api.test.core.Constants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.fasterxml.jackson.core.JsonGenerator;
+import com.fasterxml.jackson.core.JsonParser;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.DeserializationContext;
+import com.fasterxml.jackson.databind.JsonDeserializer;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.JsonSerializer;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.ObjectWriter;
+import com.fasterxml.jackson.databind.SerializationFeature;
+import com.fasterxml.jackson.databind.SerializerProvider;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.fasterxml.jackson.databind.node.TextNode;
+import com.fasterxml.jackson.databind.type.CollectionType;
+import org.testcontainers.shaded.org.apache.commons.lang.StringUtils;
+
+/**
+ * json utils
+ */
+public class JSONUtils {
Review Comment:
Is it necessary to create this util class again?
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/pom.xml:
##########
@@ -0,0 +1,40 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to Apache Software Foundation (ASF) under one or more contributor
+ ~ license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright
+ ~ ownership. Apache Software Foundation (ASF) licenses this file to you under
+ ~ the Apache License, Version 2.0 (the "License"); you may
+ ~ not use this file except in compliance with the License.
+ ~ You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>dolphinscheduler-api-test</artifactId>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <version>1.0-SNAPSHOT</version>
Review Comment:
I think we should keep the version consistent
##########
dolphinscheduler-api-test/README.md:
##########
@@ -0,0 +1,44 @@
+# DolphinScheduler Backend API Test
+
+## Page Object Model
+
+DolphinScheduler API test respects
+the [Page Object Model (POM)](https://www.selenium.dev/documentation/guidelines/page_object_models/) design pattern.
+Every page of DolphinScheduler's api is abstracted into a class for better maintainability.
+
+### Example
+
+The login page's api is abstracted
+as [`LoginPage`](dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/pages/LoginPage.java), with the
+following fields,
+
+```java
+public HttpResponse login(String username, String password) {
+ Map<String, Object> params = new HashMap<>();
+
+ params.put("userName", username);
+ params.put("userPassword", password);
+
+ RequestClient requestClient = new RequestClient();
+
+ return requestClient.post("/login", null, params);
+}
+```
+
+where `userName`, `userPassword` are the main elements on UI that we are interested in.
+
+## Test Environment Setup
+
+DolphinScheduler End-to-End test uses [testcontainers](https://www.testcontainers.org) to set up the testing
Review Comment:
Is api-test related to e2e?
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-core/src/main/java/org/apache/dolphinscheduler/api/test/core/DolphinSchedulerExtension.java:
##########
@@ -0,0 +1,79 @@
+/*
+ * Licensed to Apache Software Foundation (ASF) under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Apache Software Foundation (ASF) licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.core;
+
+import java.io.File;
+import java.net.URL;
+import java.time.Duration;
+import java.util.List;
+import java.util.Objects;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
+
+import org.junit.jupiter.api.extension.AfterAllCallback;
+import org.junit.jupiter.api.extension.BeforeAllCallback;
+import org.junit.jupiter.api.extension.ExtensionContext;
+import org.testcontainers.containers.DockerComposeContainer;
+import org.testcontainers.containers.wait.strategy.Wait;
+
+import lombok.extern.slf4j.Slf4j;
+
+@Slf4j
+final class DolphinSchedulerExtension implements BeforeAllCallback, AfterAllCallback {
+ private final boolean LOCAL_MODE = Objects.equals(System.getProperty("local"), "true");
+
+ private final String SERVICE_NAME = "dolphinscheduler_1";
+
+ private DockerComposeContainer<?> compose;
+
+ @Override
+ public void beforeAll(ExtensionContext context) {
+ if (!LOCAL_MODE) {
+ compose = createDockerCompose(context);
+ compose.start();
+ }
+ }
+
+ @Override
+ public void afterAll(ExtensionContext context) {
+ if (compose != null) {
+ compose.stop();
+ }
+ }
+
+ private DockerComposeContainer<?> createDockerCompose(ExtensionContext context) {
+ final Class<?> clazz = context.getRequiredTestClass();
+ final DolphinScheduler annotation = clazz.getAnnotation(DolphinScheduler.class);
+ final List<File> files = Stream.of(annotation.composeFiles())
+ .map(it -> DolphinScheduler.class.getClassLoader().getResource(it))
+ .filter(Objects::nonNull)
+ .map(URL::getPath)
+ .map(File::new)
+ .collect(Collectors.toList());
+
+ compose = new DockerComposeContainer<>(files)
+ .withPull(true)
+ .withTailChildContainers(true)
+ .withLogConsumer(SERVICE_NAME, outputFrame -> LOGGER.info(outputFrame.getUtf8String()))
+ .waitingFor(SERVICE_NAME, Wait.forHealthcheck().withStartupTimeout(Duration.ofSeconds(180)));
Review Comment:
use constant replace 180
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/utils/RequestClient.java:
##########
@@ -0,0 +1,171 @@
+/*
+ * Licensed to Apache Software Foundation (ASF) under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Apache Software Foundation (ASF) licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.utils;
+
+import lombok.SneakyThrows;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.dolphinscheduler.api.test.core.Constants;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponse;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponseBody;
+import org.testcontainers.shaded.okhttp3.FormBody;
+import org.testcontainers.shaded.okhttp3.Headers;
+import org.testcontainers.shaded.okhttp3.MediaType;
+import org.testcontainers.shaded.okhttp3.OkHttpClient;
+import org.testcontainers.shaded.okhttp3.Request;
+import org.testcontainers.shaded.okhttp3.RequestBody;
+import org.testcontainers.shaded.okhttp3.Response;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+@Slf4j
+public class RequestClient {
+
+ private OkHttpClient httpClient = null;
+
+ public RequestClient() {
+ this.httpClient = new OkHttpClient();
+ }
+
+ @SneakyThrows
+ public HttpResponse get(String url, Map<String, String> headers, Map<String, Object> params) {
+ String requestUrl = String.format("%s%s%s", Constants.DOLPHINSCHEDULER_API_URL, url, getParams(params));
+
+ Headers headersBuilder = new Headers.Builder().build();
+ if (headers != null) {
+ headersBuilder = Headers.of(headers);
+ }
+
+ LOGGER.info("GET request to {}, Headers: {}", requestUrl, headersBuilder);
+ Request request = new Request.Builder()
+ .url(requestUrl)
+ .headers(headersBuilder)
+ .get()
+ .build();
+
+ Response response = this.httpClient.newCall(request).execute();
+
+ HttpResponseBody responseData = null;
+ int responseCode = response.code();
+ if (response.body() != null) {
+ responseData = JSONUtils.parseObject(response.body().string(), HttpResponseBody.class);
+ }
+ response.close();
+
+ HttpResponse httpResponse = new HttpResponse(responseCode, responseData);
+
+ LOGGER.info("GET response: {}", httpResponse);
+
+ return httpResponse;
+ }
+
+ public static String getParams(Map<String, Object> params) {
+ StringBuilder sb = new StringBuilder("?");
+ if (params.size() > 0) {
+ for (Map.Entry<String, Object> item : params.entrySet()) {
+ Object value = item.getValue();
+ if (Objects.nonNull(value)) {
+ sb.append("&");
+ sb.append(item.getKey());
+ sb.append("=");
Review Comment:
same here
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-core/pom.xml:
##########
@@ -0,0 +1,32 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to Apache Software Foundation (ASF) under one or more contributor
+ ~ license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright
+ ~ ownership. Apache Software Foundation (ASF) licenses this file to you under
+ ~ the Apache License, Version 2.0 (the "License"); you may
+ ~ not use this file except in compliance with the License.
+ ~ You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing,
+ ~ software distributed under the License is distributed on an
+ ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ ~ KIND, either express or implied. See the License for the
+ ~ specific language governing permissions and limitations
+ ~ under the License.
+ -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <parent>
+ <artifactId>dolphinscheduler-api-test</artifactId>
+ <groupId>org.apache.dolphinscheduler</groupId>
+ <version>1.0-SNAPSHOT</version>
Review Comment:
same here
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/java/org/apache/dolphinscheduler/api.test/utils/RequestClient.java:
##########
@@ -0,0 +1,171 @@
+/*
+ * Licensed to Apache Software Foundation (ASF) under one or more contributor
+ * license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright
+ * ownership. Apache Software Foundation (ASF) licenses this file to you under
+ * the Apache License, Version 2.0 (the "License"); you may
+ * not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.dolphinscheduler.api.test.utils;
+
+import lombok.SneakyThrows;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.dolphinscheduler.api.test.core.Constants;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponse;
+import org.apache.dolphinscheduler.api.test.entity.HttpResponseBody;
+import org.testcontainers.shaded.okhttp3.FormBody;
+import org.testcontainers.shaded.okhttp3.Headers;
+import org.testcontainers.shaded.okhttp3.MediaType;
+import org.testcontainers.shaded.okhttp3.OkHttpClient;
+import org.testcontainers.shaded.okhttp3.Request;
+import org.testcontainers.shaded.okhttp3.RequestBody;
+import org.testcontainers.shaded.okhttp3.Response;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Objects;
+
+@Slf4j
+public class RequestClient {
+
+ private OkHttpClient httpClient = null;
+
+ public RequestClient() {
+ this.httpClient = new OkHttpClient();
+ }
+
+ @SneakyThrows
+ public HttpResponse get(String url, Map<String, String> headers, Map<String, Object> params) {
+ String requestUrl = String.format("%s%s%s", Constants.DOLPHINSCHEDULER_API_URL, url, getParams(params));
+
+ Headers headersBuilder = new Headers.Builder().build();
+ if (headers != null) {
+ headersBuilder = Headers.of(headers);
+ }
+
+ LOGGER.info("GET request to {}, Headers: {}", requestUrl, headersBuilder);
+ Request request = new Request.Builder()
+ .url(requestUrl)
+ .headers(headersBuilder)
+ .get()
+ .build();
+
+ Response response = this.httpClient.newCall(request).execute();
+
+ HttpResponseBody responseData = null;
+ int responseCode = response.code();
+ if (response.body() != null) {
+ responseData = JSONUtils.parseObject(response.body().string(), HttpResponseBody.class);
+ }
+ response.close();
+
+ HttpResponse httpResponse = new HttpResponse(responseCode, responseData);
+
+ LOGGER.info("GET response: {}", httpResponse);
+
+ return httpResponse;
+ }
+
+ public static String getParams(Map<String, Object> params) {
+ StringBuilder sb = new StringBuilder("?");
+ if (params.size() > 0) {
+ for (Map.Entry<String, Object> item : params.entrySet()) {
+ Object value = item.getValue();
+ if (Objects.nonNull(value)) {
+ sb.append("&");
Review Comment:
use constant.
##########
dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/resources/docker/file-manage/common.properties:
##########
@@ -0,0 +1,90 @@
+#
Review Comment:
If the original file has changed, does it need to be kept in sync here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] kezhenxu94 commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155924949
> > LGTM overall, but it seem some of the files are or smilar to module dolphinscheduler-e2e, I wonder if there exists some way to combine them into some parent module?
>
> The two modules are completely functionally unrelated. API testing only uses the basic framework of E2E for reference and starts third-party services based on docker.
>
> I have discussed this issue with @kezhenxu94 . So I created a new `API-Test` module.
Hi @SbloodyS , can you check whether we can refactor the existing `dolphinscheduler-e2e` to some modules, `dolphinscheduler-testing` containing the `core` part, and let `dolphinscheduler-e2e-test` and `dolphinscheduler-api-test` depends on `dolphinscheduler-testing`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155028093
# [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10442?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#10442](https://codecov.io/gh/apache/dolphinscheduler/pull/10442?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (fdd7a03) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/190f253083e2b1d50f235b07c53fbbe6b66c1aa2?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (190f253) will **increase** coverage by `0.03%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## dev #10442 +/- ##
============================================
+ Coverage 40.55% 40.58% +0.03%
- Complexity 4774 4776 +2
============================================
Files 878 878
Lines 35741 35746 +5
Branches 3969 3970 +1
============================================
+ Hits 14493 14509 +16
+ Misses 19805 19788 -17
- Partials 1443 1449 +6
```
| [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/10442?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...er/api/service/impl/TaskDefinitionServiceImpl.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10442/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3NlcnZpY2UvaW1wbC9UYXNrRGVmaW5pdGlvblNlcnZpY2VJbXBsLmphdmE=) | `24.01% <0.00%> (ø)` | |
| [...che/dolphinscheduler/api/python/PythonGateway.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10442/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1hcGkvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvYXBpL3B5dGhvbi9QeXRob25HYXRld2F5LmphdmE=) | `17.00% <0.00%> (+7.76%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10442?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10442?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [190f253...fdd7a03](https://codecov.io/gh/apache/dolphinscheduler/pull/10442?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155033511
SonarCloud Quality Gate failed. [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10442)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=BUG)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL) [26 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10442&resolved=false&types=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_coverage&view=list) [14.8% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_coverage&view=list)
[](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10442&metric=new_duplicated_lines_density&view=list)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] kezhenxu94 commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1155963263
> After a deep checking. I found the following differences in both core module's ```DolphinSchedulerExtension.java```.
>
>
>
> 1. The ```beforeAll``` and ```afterAll``` function would set ```webdriver``` and ``` browser``` in ```E2E-Test``` module. However, ```API-Test``` modules do not need these.
>
> 2. Based on the first point, there is no need to pull other additional containers in the ```API-Test``` module for local mode and non local mode. So there is no need to distinguishing container images of M1 chip.
>
>
>
> As we can see, the implementation of the ```core``` module of the ```API-Test``` and ```E2E-Test``` is quite different. So if we want to combine these two modules into one. I think we need to create an abstract class. And then implement its own logic in the ```core``` of the two modules.
>
>
>
> I'm not sure whether we need to do this since it's not much different from now. WDYT? @zhongjiajie @kezhenxu94
>
>
Make sense. Let's keep them 2 modules for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on pull request #10442: [Feature-10411] Add tenant api test
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on PR #10442:
URL: https://github.com/apache/dolphinscheduler/pull/10442#issuecomment-1156160734
> As GitHub Actions now can re-run only failed jobs, we should consider combine the workflows to save resources and time. Also consider adding api-test to required context before merging PRs
I'll try to combine the workflows in a few days. Since the API test is a new component. It is uncertain whether it is stable in github actions. I'll add it to required context before merging PRs after a period of observation and stabilization. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org