You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/30 10:11:37 UTC

[GitHub] [dolphinscheduler] 106umao opened a new pull request, #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

106umao opened a new pull request, #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294

   <!--Thanks very much for contributing to Apache DolphinScheduler. Please review https://dolphinscheduler.apache.org/en-us/community/development/pull-request.html before opening a pull request.-->
   
   
   ## Purpose of the pull request
   
   When testing the test case of HTTPUtilsTest, the connection timeout will occur. Usually, we will run the test case in mvn test. If this unstable problem occurs, it will need to be restarted frequently.
   
   ## Brief change log
   
   <!--*(for example:)*
   
   Modified URLs of two methods in HttpUtilsTest
   
   ## Verify this pull request
   
   <!--*(Please pick either of the following options)*-->
   
   This pull request is code cleanup without any test coverage.
   
   *(or)*
   
   This pull request is already covered by existing tests, such as *(please describe tests)*.
   
   (or)
   
   This change added tests and can be verified as follows:
   
   <!--*(example:)*
     - *Added dolphinscheduler-dao tests for end-to-end.*
     - *Added CronUtilsTest to verify the change.*
     - *Manually verified the change by testing locally.* -->
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#issuecomment-1140994140

   SonarCloud Quality Gate failed.&nbsp; &nbsp; [![Quality Gate failed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/failed-16px.png 'Quality Gate failed')](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10294)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL) [1 Code Smell](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL)
   
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/0-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_coverage&view=list) [0.0% Coverage](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_coverage&view=list)  
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_duplicated_lines_density&view=list)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r890058649


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalServerHttpUtilsTest.java:
##########
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import junit.framework.Test;
+import junit.framework.TestCase;
+import junit.framework.TestSuite;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.junit.Assert;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class LocalServerHttpUtilsTest extends TestCase{
+
+    private HadoopUtils hadoopUtils = HadoopUtils.getInstance();

Review Comment:
   Can you remove that and I think it's ready to merge



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r889807570


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalServerHttpUtilsTest.java:
##########
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import junit.framework.Test;
+import junit.framework.TestCase;
+import junit.framework.TestSuite;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.junit.Assert;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class LocalServerHttpUtilsTest extends TestCase{
+
+    private HadoopUtils hadoopUtils = HadoopUtils.getInstance();

Review Comment:
   I think it should be removed, but I have asked questions about it in the WeChat group and no one has answered my questions



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r885738208


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   > Actually I'd just remove this test case or set up an http server before the tests. Unit tests should not rely on external third party services. No matter what links you use there must be someone with poor connections and will timeout.
   
   So do you need to merge this pr now, or do I hide these two test cases in this pr because they will cause the test to be unstable



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r890058216


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalServerHttpUtilsTest.java:
##########
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import junit.framework.Test;
+import junit.framework.TestCase;
+import junit.framework.TestSuite;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.junit.Assert;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class LocalServerHttpUtilsTest extends TestCase{
+
+    private HadoopUtils hadoopUtils = HadoopUtils.getInstance();

Review Comment:
   > I think it should be removed, but I have asked questions about it in the WeChat group and no one has answered my questions
   
   Oh sorry to hear no one answered but please ask in issues/PR next time so it should be addressed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] codecov-commenter commented on pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#issuecomment-1140987314

   # [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10294?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
   > Merging [#10294](https://codecov.io/gh/apache/dolphinscheduler/pull/10294?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (6eafbd8) into [dev](https://codecov.io/gh/apache/dolphinscheduler/commit/8d26d4fe72aa81437103716b22f52ef6dc515a14?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8d26d4f) will **decrease** coverage by `0.00%`.
   > The diff coverage is `n/a`.
   
   ```diff
   @@             Coverage Diff              @@
   ##                dev   #10294      +/-   ##
   ============================================
   - Coverage     41.15%   41.14%   -0.01%     
   + Complexity     4772     4770       -2     
   ============================================
     Files           857      857              
     Lines         34659    34659              
     Branches       3828     3828              
   ============================================
   - Hits          14263    14260       -3     
   - Misses        19019    19024       +5     
   + Partials       1377     1375       -2     
   ```
   
   
   | [Impacted Files](https://codecov.io/gh/apache/dolphinscheduler/pull/10294?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
   |---|---|---|
   | [...org/apache/dolphinscheduler/remote/utils/Host.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10294/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL3V0aWxzL0hvc3QuamF2YQ==) | `37.77% <0.00%> (-2.23%)` | :arrow_down: |
   | [...e/dolphinscheduler/remote/NettyRemotingClient.java](https://codecov.io/gh/apache/dolphinscheduler/pull/10294/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-ZG9scGhpbnNjaGVkdWxlci1yZW1vdGUvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2RvbHBoaW5zY2hlZHVsZXIvcmVtb3RlL05ldHR5UmVtb3RpbmdDbGllbnQuamF2YQ==) | `52.11% <0.00%> (-1.41%)` | :arrow_down: |
   
   ------
   
   [Continue to review full report at Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10294?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
   > `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
   > Powered by [Codecov](https://codecov.io/gh/apache/dolphinscheduler/pull/10294?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [12a09f6...6eafbd8](https://codecov.io/gh/apache/dolphinscheduler/pull/10294?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886696465


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalJettyHttpServer.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.BindException;
+
+import junit.extensions.TestSetup;
+import junit.framework.Test;
+
+
+import org.mortbay.jetty.*;
+import org.mortbay.jetty.handler.AbstractHandler;
+import org.mortbay.jetty.handler.ContextHandler;
+import org.mortbay.util.ByteArrayISO8859Writer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+public class LocalJettyHttpServer extends TestSetup {
+    protected static Server server;
+    private static Logger logger = LoggerFactory.getLogger(LocalJettyHttpServer.class);
+    private Integer serverPort = 8888;
+
+    public Integer getServerPort() {
+        return serverPort;
+    }
+
+    public LocalJettyHttpServer(Test suite) {
+        super(suite);
+    }
+
+    protected void setUp() throws Exception {
+        while (true) {
+            try {
+                server = new Server(serverPort);
+                ContextHandler context = new ContextHandler("/test.json");
+                context.setHandler(new AbstractHandler() {
+                    @Override
+                    public void handle(String s, HttpServletRequest request, HttpServletResponse response, int i) throws IOException {
+                        ByteArrayISO8859Writer writer = new ByteArrayISO8859Writer();
+                        writer.write("{\"name\":\"Github\"}");
+                        writer.flush();
+                        response.setContentLength(writer.size());
+                        OutputStream out = response.getOutputStream();
+                        writer.writeTo(out);
+                        out.flush();
+                        Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
+                        baseRequest.setHandled(true);
+                    }
+                });
+                server.setHandler(context);
+                logger.info("server for " + context.getBaseResource());
+                server.start();
+                logger.info("server is starting in port: "+serverPort);
+            } catch (BindException e) {
+                server.stop();
+                logger.info("port: "+serverPort + " has been bind");
+                serverPort++;
+                continue;
+            }
+            break;
+        }

Review Comment:
   @kezhenxu94 
   I should have misunderstood before, I thought he would assign the port when constructing the object, but in fact it is assigned in start.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886370216


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalJettyHttpServer.java:
##########
@@ -0,0 +1,56 @@
+package org.apache.dolphinscheduler.common.utils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+
+import junit.extensions.TestSetup;
+import junit.framework.Test;
+
+
+import org.mortbay.jetty.*;
+import org.mortbay.jetty.handler.AbstractHandler;
+import org.mortbay.jetty.handler.ContextHandler;
+import org.mortbay.util.ByteArrayISO8859Writer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+public class LocalJettyHttpServer extends TestSetup {
+    protected static Server server;
+    private static Logger logger = LoggerFactory.getLogger(LocalJettyHttpServer.class);
+
+    public LocalJettyHttpServer(Test suite) {
+        super(suite);
+    }
+
+    protected void setUp() throws Exception {
+        logger.info("server si starting...");
+        server = new Server(8888);

Review Comment:
   Do not hardcode this port, it will also fail when users' machine has service running at port 8888,
   
   ```java
   
       Server server = new Server();
       Connector connector = new SelectChannelConnector();
   
       connector.setPort(0);
       server.addConnector(connector);
       server.start();
   
   //... and use this to get the dynamic port
       connector.getLocalPort();
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#issuecomment-1142144214

   Hi.@SbloodyS
   It is true that the network is not very good, after replacing the airport you recommended the timeout frequency became lower, but I still have 5 timeouts after 20 consecutive tests, so this retry should still be necessary


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r885738208


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   > Actually I'd just remove this test case or set up an http server before the tests. Unit tests should not rely on external third party services. No matter what links you use there must be someone with poor connections and will timeout.
   So do I need to merge this pr now, or do I hide these two test cases in this pr because they will cause the test to be unstable



##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   > Actually I'd just remove this test case or set up an http server before the tests. Unit tests should not rely on external third party services. No matter what links you use there must be someone with poor connections and will timeout.
   
   So do I need to merge this pr now, or do I hide these two test cases in this pr because they will cause the test to be unstable



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#issuecomment-1147561441

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10294)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=coverage&view=list) No Coverage information  
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_duplicated_lines_density&view=list)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r885715370


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   Actually I'd just remove this test case or set up an http server before the tests. Unit tests should not rely on external third party services. No matter what links you use there must be someone with poor connections and will timeout. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r889776458


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalServerHttpUtilsTest.java:
##########
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import junit.framework.Test;
+import junit.framework.TestCase;
+import junit.framework.TestSuite;
+import org.apache.dolphinscheduler.common.Constants;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.methods.HttpGet;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClients;
+import org.junit.Assert;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class LocalServerHttpUtilsTest extends TestCase{
+
+    private HadoopUtils hadoopUtils = HadoopUtils.getInstance();

Review Comment:
   Can we remove Hadoop-related things in this pure http test?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 merged pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 merged PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] songjianet commented on pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
songjianet commented on PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#issuecomment-1141021717

   Please associate the corresponding issue. @106umao 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886566229


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalJettyHttpServer.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.BindException;
+
+import junit.extensions.TestSetup;
+import junit.framework.Test;
+
+
+import org.mortbay.jetty.*;
+import org.mortbay.jetty.handler.AbstractHandler;
+import org.mortbay.jetty.handler.ContextHandler;
+import org.mortbay.util.ByteArrayISO8859Writer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+public class LocalJettyHttpServer extends TestSetup {
+    protected static Server server;
+    private static Logger logger = LoggerFactory.getLogger(LocalJettyHttpServer.class);
+    private Integer serverPort = 8888;
+
+    public Integer getServerPort() {
+        return serverPort;
+    }
+
+    public LocalJettyHttpServer(Test suite) {
+        super(suite);
+    }
+
+    protected void setUp() throws Exception {
+        while (true) {
+            try {
+                server = new Server(serverPort);
+                ContextHandler context = new ContextHandler("/test.json");
+                context.setHandler(new AbstractHandler() {
+                    @Override
+                    public void handle(String s, HttpServletRequest request, HttpServletResponse response, int i) throws IOException {
+                        ByteArrayISO8859Writer writer = new ByteArrayISO8859Writer();
+                        writer.write("{\"name\":\"Github\"}");
+                        writer.flush();
+                        response.setContentLength(writer.size());
+                        OutputStream out = response.getOutputStream();
+                        writer.writeTo(out);
+                        out.flush();
+                        Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
+                        baseRequest.setHandled(true);
+                    }
+                });
+                server.setHandler(context);
+                logger.info("server for " + context.getBaseResource());
+                server.start();
+                logger.info("server is starting in port: "+serverPort);
+            } catch (BindException e) {
+                server.stop();
+                logger.info("port: "+serverPort + " has been bind");
+                serverPort++;
+                continue;
+            }
+            break;
+        }

Review Comment:
   Please don't do this kind of ugly workarounds, do things in a right way instead of using all kinds of workarounds. see https://stackoverflow.com/questions/8884865/how-to-discover-embedded-jetty-port-after-requesting-random-available-port



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886245086


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   > > Actually I'd just remove this test case or set up an http server before the tests. Unit tests should not rely on external third party services. No matter what links you use there must be someone with poor connections and will timeout.
   > 
   > 
   > 
   > So do you need to merge this pr now, or do I hide these two test cases in this pr because they will cause the test to be unstable
   
   Can you just modify the test to set up an http server with Jetty, and use localhost:port to test this http util? That would be stable and no timeout. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r884912534


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   I don't think this is a good idea. Due to the instability of Baidu. This url may cause unstable in unit test. 
   
   And If GitHub cannot be accessed, the DS code can not be pulled.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r885262296


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   > Perhaps because of the github anti-ddos attack, different policies are used for different URLs, but multiple tries always work.
   
   Maybe your network is in poor condition.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886628952


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalJettyHttpServer.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.BindException;
+
+import junit.extensions.TestSetup;
+import junit.framework.Test;
+
+
+import org.mortbay.jetty.*;
+import org.mortbay.jetty.handler.AbstractHandler;
+import org.mortbay.jetty.handler.ContextHandler;
+import org.mortbay.util.ByteArrayISO8859Writer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+public class LocalJettyHttpServer extends TestSetup {
+    protected static Server server;
+    private static Logger logger = LoggerFactory.getLogger(LocalJettyHttpServer.class);
+    private Integer serverPort = 8888;
+
+    public Integer getServerPort() {
+        return serverPort;
+    }
+
+    public LocalJettyHttpServer(Test suite) {
+        super(suite);
+    }
+
+    protected void setUp() throws Exception {
+        while (true) {
+            try {
+                server = new Server(serverPort);
+                ContextHandler context = new ContextHandler("/test.json");
+                context.setHandler(new AbstractHandler() {
+                    @Override
+                    public void handle(String s, HttpServletRequest request, HttpServletResponse response, int i) throws IOException {
+                        ByteArrayISO8859Writer writer = new ByteArrayISO8859Writer();
+                        writer.write("{\"name\":\"Github\"}");
+                        writer.flush();
+                        response.setContentLength(writer.size());
+                        OutputStream out = response.getOutputStream();
+                        writer.writeTo(out);
+                        out.flush();
+                        Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
+                        baseRequest.setHandled(true);
+                    }
+                });
+                server.setHandler(context);
+                logger.info("server for " + context.getBaseResource());
+                server.start();
+                logger.info("server is starting in port: "+serverPort);
+            } catch (BindException e) {
+                server.stop();
+                logger.info("port: "+serverPort + " has been bind");
+                serverPort++;
+                continue;
+            }
+            break;
+        }

Review Comment:
   Thanks for the reminder, but I think this loop can not be saved, you think about it, before the server.start (), the randomly obtained port does not belong to this server, it is likely to be occupied, when occupied will lead to an exception exit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r885260270


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   Perhaps because of the github anti-ddos attack, different policies are used for different URLs, but multiple tries always work.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r885263739


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/HttpUtilsTest.java:
##########
@@ -40,10 +40,10 @@ public class HttpUtilsTest {
     @Test
     public void testGetTest() {
 	// success
-	String result = HttpUtils.get("https://github.com/manifest.json");
+	String result = HttpUtils.get("https://www.baidu.com/sugrec?prod=pc_his&from=pc_web&json=1&sid=36427_36454_31253_36422_36165_36487_36055_36376_36234_26350_36469_36316&hisdata=&_t=1653904731156&req=2&csor=0");

Review Comment:
   You can consider changing to some high-performance ```Airports```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] sonarcloud[bot] commented on pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
sonarcloud[bot] commented on PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#issuecomment-1141750249

   Kudos, SonarCloud Quality Gate passed!&nbsp; &nbsp; [![Quality Gate passed](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=10294)
   
   [![Bug](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/bug-16px.png 'Bug')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=BUG)  
   [![Vulnerability](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/vulnerability-16px.png 'Vulnerability')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=VULNERABILITY)  
   [![Security Hotspot](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/security_hotspot-16px.png 'Security Hotspot')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=SECURITY_HOTSPOT)  
   [![Code Smell](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/common/code_smell-16px.png 'Code Smell')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL) [![A](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/RatingBadge/A-16px.png 'A')](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL) [0 Code Smells](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=10294&resolved=false&types=CODE_SMELL)
   
   [![No Coverage information](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/CoverageChart/NoCoverageInfo-16px.png 'No Coverage information')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=coverage&view=list) No Coverage information  
   [![0.0%](https://sonarsource.github.io/sonarcloud-github-static-resources/v2/checks/Duplications/3-16px.png '0.0%')](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_duplicated_lines_density&view=list) [0.0% Duplication](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=10294&metric=new_duplicated_lines_density&view=list)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] 106umao commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
106umao commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886628952


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalJettyHttpServer.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.BindException;
+
+import junit.extensions.TestSetup;
+import junit.framework.Test;
+
+
+import org.mortbay.jetty.*;
+import org.mortbay.jetty.handler.AbstractHandler;
+import org.mortbay.jetty.handler.ContextHandler;
+import org.mortbay.util.ByteArrayISO8859Writer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+public class LocalJettyHttpServer extends TestSetup {
+    protected static Server server;
+    private static Logger logger = LoggerFactory.getLogger(LocalJettyHttpServer.class);
+    private Integer serverPort = 8888;
+
+    public Integer getServerPort() {
+        return serverPort;
+    }
+
+    public LocalJettyHttpServer(Test suite) {
+        super(suite);
+    }
+
+    protected void setUp() throws Exception {
+        while (true) {
+            try {
+                server = new Server(serverPort);
+                ContextHandler context = new ContextHandler("/test.json");
+                context.setHandler(new AbstractHandler() {
+                    @Override
+                    public void handle(String s, HttpServletRequest request, HttpServletResponse response, int i) throws IOException {
+                        ByteArrayISO8859Writer writer = new ByteArrayISO8859Writer();
+                        writer.write("{\"name\":\"Github\"}");
+                        writer.flush();
+                        response.setContentLength(writer.size());
+                        OutputStream out = response.getOutputStream();
+                        writer.writeTo(out);
+                        out.flush();
+                        Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
+                        baseRequest.setHandled(true);
+                    }
+                });
+                server.setHandler(context);
+                logger.info("server for " + context.getBaseResource());
+                server.start();
+                logger.info("server is starting in port: "+serverPort);
+            } catch (BindException e) {
+                server.stop();
+                logger.info("port: "+serverPort + " has been bind");
+                serverPort++;
+                continue;
+            }
+            break;
+        }

Review Comment:
   Thanks for the reminder, but I think this loop can not be saved, you think about it, when the server.start () before the randomly obtained port does not belong to the server, it is likely to be occupied, when occupied will lead to an exception exit.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [dolphinscheduler] kezhenxu94 commented on a diff in pull request #10294: [Fix] fix the HttpUtilsTest' test case , it is always time out.

Posted by GitBox <gi...@apache.org>.
kezhenxu94 commented on code in PR #10294:
URL: https://github.com/apache/dolphinscheduler/pull/10294#discussion_r886659572


##########
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/LocalJettyHttpServer.java:
##########
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *    http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.dolphinscheduler.common.utils;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.BindException;
+
+import junit.extensions.TestSetup;
+import junit.framework.Test;
+
+
+import org.mortbay.jetty.*;
+import org.mortbay.jetty.handler.AbstractHandler;
+import org.mortbay.jetty.handler.ContextHandler;
+import org.mortbay.util.ByteArrayISO8859Writer;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.servlet.http.HttpServletRequest;
+import javax.servlet.http.HttpServletResponse;
+
+public class LocalJettyHttpServer extends TestSetup {
+    protected static Server server;
+    private static Logger logger = LoggerFactory.getLogger(LocalJettyHttpServer.class);
+    private Integer serverPort = 8888;
+
+    public Integer getServerPort() {
+        return serverPort;
+    }
+
+    public LocalJettyHttpServer(Test suite) {
+        super(suite);
+    }
+
+    protected void setUp() throws Exception {
+        while (true) {
+            try {
+                server = new Server(serverPort);
+                ContextHandler context = new ContextHandler("/test.json");
+                context.setHandler(new AbstractHandler() {
+                    @Override
+                    public void handle(String s, HttpServletRequest request, HttpServletResponse response, int i) throws IOException {
+                        ByteArrayISO8859Writer writer = new ByteArrayISO8859Writer();
+                        writer.write("{\"name\":\"Github\"}");
+                        writer.flush();
+                        response.setContentLength(writer.size());
+                        OutputStream out = response.getOutputStream();
+                        writer.writeTo(out);
+                        out.flush();
+                        Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
+                        baseRequest.setHandled(true);
+                    }
+                });
+                server.setHandler(context);
+                logger.info("server for " + context.getBaseResource());
+                server.start();
+                logger.info("server is starting in port: "+serverPort);
+            } catch (BindException e) {
+                server.stop();
+                logger.info("port: "+serverPort + " has been bind");
+                serverPort++;
+                continue;
+            }
+            break;
+        }

Review Comment:
   > Thanks for the reminder, but I think this loop can not be saved, you think about it, before the server.start (), the randomly obtained port does not belong to this server, it is likely to be occupied, when occupied will lead to an exception exit.
   
   Why do you think so? When you set port to 0 jetty will find an available port and use it.The key point is "available" not "random".



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org