You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampark.apache.org by GitBox <gi...@apache.org> on 2022/09/14 14:26:35 UTC
[GitHub] [incubator-streampark] RocMarshal opened a new pull request, #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
RocMarshal opened a new pull request, #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606
<!--
Thank you for contributing to StreamPark! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
### What problem does this PR solve?
Issue Number: close #xxx <!-- REMOVE this line if no issue to close -->
Problem Summary:
### What is changed and how it works?
Proposal: [xxx](url) <!-- REMOVE this line if not applicable -->
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/streampark/issues).
- Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
## Purpose of this pull request
Re-arrange the test package structure of the streampark-console-service module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] RocMarshal commented on a diff in pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
RocMarshal commented on code in PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606#discussion_r971960271
##########
streampark-console/streampark-console-service/src/test/java/com/github/benmanes/caffeine/cache/RefreshCacheTest.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.github.benmanes.caffeine.cache;
+
+import org.apache.streampark.common.util.DateUtils;
+
+import org.junit.Test;
+
+import java.util.Date;
+import java.util.TimeZone;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+public class RefreshCacheTest {
+
+ Cache<String, String> caffeine = null;
+
+ @Test
+ public void cache() throws Exception {
+ if (caffeine == null) {
+ caffeine = Caffeine.newBuilder()
+ .refreshAfterWrite(10, TimeUnit.SECONDS)
+ .build(this::refresh);
+ }
+ caffeine.put("config", "hadoop");
+ while (true) {
+ System.out.println(caffeine.getIfPresent("config"));
+ Thread.sleep(2000);
+ }
+ }
+
+ public String refresh(String value) {
+ return UUID.randomUUID() + "@" + value;
+ }
+
+ @Test
+ public void task() throws InterruptedException {
+ System.out.println(DateUtils.format(new Date(), DateUtils.fullFormat(), TimeZone.getDefault()));
+ Timer timer = new Timer();
+ timer.schedule(new TimerTask() {
+ @Override
+ public void run() {
+ System.out.println(DateUtils.format(new Date(), DateUtils.fullFormat(), TimeZone.getDefault()));
+ }
+ }, 1000 * 10, 1000 * 10);
+
+ Thread.sleep(Long.MAX_VALUE);
Review Comment:
@wolfboys Thx for the comment.
Updated. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] wolfboys commented on pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
wolfboys commented on PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606#issuecomment-1246888634
Thanks for your contribution. I've seen many package name has changed, it is not under the same package name, Can you tell me why we need to change like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] RocMarshal commented on pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
RocMarshal commented on PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606#issuecomment-1246927287
@wolfboys
- Some test cases about built-in classes like ```CompletableFutureTest.java```、```RegexTest``` should not be located in the repo. Because It looks more like a usage example. However, the tested classes have already be stable.
- Some test cases about third-part dependencies like ```caffeine``` are also same as mentioned above.
- The test cases classes files would be better located in the same package which contains target tested classes to test under the corresponding test module. In addition, other modules' test cases package structure also follow this rule
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] wolfboys merged pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
wolfboys merged PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] RocMarshal commented on a diff in pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
RocMarshal commented on code in PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606#discussion_r971960538
##########
streampark-common/src/test/scala/org/apache/streampark/common/util/YarnTest.scala:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.streampark.common.util
+
+import org.apache.hadoop.yarn.api.records.{NodeReport, NodeState}
+import org.apache.hadoop.yarn.exceptions.YarnException
+import org.junit.jupiter.api.Test
+
+import java.io.{File, IOException}
+
+class YarnTest {
+
+ @Test
+ @throws[IOException]
+ @throws[YarnException]
+ def vcore(): Unit = {
+ val numYarnMaxVcores = HadoopUtils.yarnClient.getNodeReports(NodeState.RUNNING).stream.mapToInt(report => report.getCapability.getVirtualCores).max.orElse(0)
+ System.out.println(numYarnMaxVcores)
+ }
+
+ @Test def getURL(): Unit = {
+ /**
+ * 将hadoop的配置文件放到一个目录下,
+ * 在运行该类的时候加上jvm级别的参数(idea里的 vmOption ) -DHADOOP_CONF_DIR=${目录}
+ */
+ val url = YarnUtils.getRMWebAppURL
Review Comment:
@lvshaokang Thx for the comment.
Updated. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] lvshaokang commented on a diff in pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
lvshaokang commented on code in PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606#discussion_r971025292
##########
streampark-common/src/test/scala/org/apache/streampark/common/util/YarnTest.scala:
##########
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.streampark.common.util
+
+import org.apache.hadoop.yarn.api.records.{NodeReport, NodeState}
+import org.apache.hadoop.yarn.exceptions.YarnException
+import org.junit.jupiter.api.Test
+
+import java.io.{File, IOException}
+
+class YarnTest {
+
+ @Test
+ @throws[IOException]
+ @throws[YarnException]
+ def vcore(): Unit = {
+ val numYarnMaxVcores = HadoopUtils.yarnClient.getNodeReports(NodeState.RUNNING).stream.mapToInt(report => report.getCapability.getVirtualCores).max.orElse(0)
+ System.out.println(numYarnMaxVcores)
+ }
+
+ @Test def getURL(): Unit = {
+ /**
+ * 将hadoop的配置文件放到一个目录下,
+ * 在运行该类的时候加上jvm级别的参数(idea里的 vmOption ) -DHADOOP_CONF_DIR=${目录}
+ */
+ val url = YarnUtils.getRMWebAppURL
Review Comment:
@RocMarshal Thanks for you feedback. Could you handle these non-ascaii characters by remove or translate ways?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-streampark] wolfboys commented on a diff in pull request #1606: [Hotfix] Re-arrange the test package structure of streampark-console-service.
Posted by GitBox <gi...@apache.org>.
wolfboys commented on code in PR #1606:
URL: https://github.com/apache/incubator-streampark/pull/1606#discussion_r971525093
##########
streampark-console/streampark-console-service/src/test/java/com/github/benmanes/caffeine/cache/RefreshCacheTest.java:
##########
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.github.benmanes.caffeine.cache;
+
+import org.apache.streampark.common.util.DateUtils;
+
+import org.junit.Test;
+
+import java.util.Date;
+import java.util.TimeZone;
+import java.util.Timer;
+import java.util.TimerTask;
+import java.util.UUID;
+import java.util.concurrent.TimeUnit;
+
+public class RefreshCacheTest {
+
+ Cache<String, String> caffeine = null;
+
+ @Test
+ public void cache() throws Exception {
+ if (caffeine == null) {
+ caffeine = Caffeine.newBuilder()
+ .refreshAfterWrite(10, TimeUnit.SECONDS)
+ .build(this::refresh);
+ }
+ caffeine.put("config", "hadoop");
+ while (true) {
+ System.out.println(caffeine.getIfPresent("config"));
+ Thread.sleep(2000);
+ }
+ }
+
+ public String refresh(String value) {
+ return UUID.randomUUID() + "@" + value;
+ }
+
+ @Test
+ public void task() throws InterruptedException {
+ System.out.println(DateUtils.format(new Date(), DateUtils.fullFormat(), TimeZone.getDefault()));
+ Timer timer = new Timer();
+ timer.schedule(new TimerTask() {
+ @Override
+ public void run() {
+ System.out.println(DateUtils.format(new Date(), DateUtils.fullFormat(), TimeZone.getDefault()));
+ }
+ }, 1000 * 10, 1000 * 10);
+
+ Thread.sleep(Long.MAX_VALUE);
Review Comment:
`Thread.sleep(Long.MAX_VALUE)` will block the thread. You can delete this testcase, see: https://github.com/apache/incubator-streampark/pull/1595
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampark.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org