You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@rocketmq.apache.org by ji...@apache.org on 2023/05/11 06:45:51 UTC
[rocketmq-e2e] branch master updated: [ISSUE #24] Add cicd workflow to repo (#25)
This is an automated email from the ASF dual-hosted git repository.
jinrongtong pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/rocketmq-e2e.git
The following commit(s) were added to refs/heads/master by this push:
new 33ca29d [ISSUE #24] Add cicd workflow to repo (#25)
33ca29d is described below
commit 33ca29d587208b3f37a09613664eea2fb31d8963
Author: yueya <10...@users.noreply.github.com>
AuthorDate: Thu May 11 14:45:46 2023 +0800
[ISSUE #24] Add cicd workflow to repo (#25)
* Add github action CI
* fix bug
* update push-ci.yml
* Update README.md
* Update README.md
* Create README.md
* update pom.xml
* update push-ci.yml
* update push-ci.yml
* update push-ci.yml
* update push-ci.yml
* update push-ci.yml
* Create README.md
* fix bug
* update push-ci.yml
* update push-ci.yml
* Create README.md
* Create README.md
* update workflows
---
.github/ISSUE_TEMPLATE/issue_template.md | 50 ++++++++
.github/PULL_REQUEST_TEMPLATE.md | 28 +++++
.github/asf-deploy-settings.xml | 33 +++++
.github/workflows/license-checker.yaml | 34 +++++
.github/workflows/pr-ci.yml | 15 +++
.github/workflows/pr-e2e-test.yml | 136 ++++++++++++++++++++
.github/workflows/push-ci.yml | 138 +++++++++++++++++++++
README.md | 10 +-
java/e2e-v4/pom.xml | 1 +
java/e2e/pom.xml | 1 +
.../java/org/apache/rocketmq/util/VerifyUtils.java | 2 +-
11 files changed, 442 insertions(+), 6 deletions(-)
diff --git a/.github/ISSUE_TEMPLATE/issue_template.md b/.github/ISSUE_TEMPLATE/issue_template.md
new file mode 100644
index 0000000..9357bb2
--- /dev/null
+++ b/.github/ISSUE_TEMPLATE/issue_template.md
@@ -0,0 +1,50 @@
+---
+name: ISSUE_TEMPLATE
+about: Describe this issue template's purpose here.
+
+---
+
+The issue tracker is used for bug reporting purposes **ONLY** whereas feature request needs to follow the [RIP process](https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal). To avoid unnecessary duplication, please check whether there is a previous issue before filing a new one.
+
+It is recommended to start a discussion thread in the [mailing lists](http://rocketmq.apache.org/about/contact/) or [github discussions](https://github.com/apache/rocketmq/discussions) in cases of discussing your deployment plan, API clarification, and other non-bug-reporting issues.
+We welcome any friendly suggestions, bug fixes, collaboration, and other improvements.
+
+Please ensure that your bug report is clear and self-contained. Otherwise, it would take additional rounds of communication, thus more time, to understand the problem itself.
+
+Generally, fixing an issue goes through the following steps:
+1. Understand the issue reported;
+1. Reproduce the unexpected behavior locally;
+1. Perform root cause analysis to identify the underlying problem;
+1. Create test cases to cover the identified problem;
+1. Work out a solution to rectify the behavior and make the newly created test cases pass;
+1. Make a pull request and go through peer review;
+
+As a result, it would be very helpful yet challenging if you could provide an isolated project reproducing your reported issue. Anyway, please ensure your issue report is informative enough for the community to pick up. At a minimum, include the following hints:
+
+**BUG REPORT**
+
+1. Please describe the issue you observed:
+
+- What did you do (The steps to reproduce)?
+
+- What is expected to see?
+
+- What did you see instead?
+
+2. Please tell us about your environment:
+
+3. Other information (e.g. detailed explanation, logs, related issues, suggestions on how to fix, etc):
+
+**FEATURE REQUEST**
+
+1. Please describe the feature you are requesting.
+
+2. Provide any additional detail on your proposed use case for this feature.
+
+3. Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
+
+4. If there are some sub-tasks involved, use -[] for each sub-task and create a corresponding issue to map to the sub-task:
+
+- [sub-task1-issue-number](example_sub_issue1_link_here): sub-task1 description here,
+- [sub-task2-issue-number](example_sub_issue2_link_here): sub-task2 description here,
+- ...
diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md
new file mode 100644
index 0000000..550a6fd
--- /dev/null
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -0,0 +1,28 @@
+
+**Make sure set the target branch to `develop`**
+
+## What is the purpose of the change
+
+<!--
+If this PR fixes a GitHub issue, please add `fixes #<XXX>` or `closes #<XXX>`. Please refer to the documentation for more information:
+https://docs.github.com/en/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
+-->
+
+fix #<xxx> <!-- <xxx> replace with issue id -->
+
+## Brief changelog
+
+XX
+
+## Verifying this change
+
+XXXX
+
+Follow this checklist to help us incorporate your contribution quickly and easily. Notice, `it would be helpful if you could finish the following 5 checklist(the last one is not necessary)before request the community to review your PR`.
+
+- [x] Make sure there is a [Github issue](https://github.com/apache/rocketmq/issues) filed for the change (usually before you start working on it). Trivial changes like typos do not require a Github issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
+- [x] Format the pull request title like `[ISSUE #123] Fix UnknownException when host config not exist`. Each commit in the pull request should have a meaningful subject line and body.
+- [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
+- [x] Write necessary unit-test(over 80% coverage) to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in [test module](https://github.com/apache/rocketmq/tree/master/test).
+- [x] Run `mvn -B clean apache-rat:check findbugs:findbugs checkstyle:checkstyle` to make sure basic checks pass. Run `mvn clean install -DskipITs` to make sure unit-test pass. Run `mvn clean test-compile failsafe:integration-test` to make sure integration-test pass.
+- [ ] If this contribution is large, please file an [Apache Individual Contributor License Agreement](http://www.apache.org/licenses/#clas).
diff --git a/.github/asf-deploy-settings.xml b/.github/asf-deploy-settings.xml
new file mode 100644
index 0000000..fad16cf
--- /dev/null
+++ b/.github/asf-deploy-settings.xml
@@ -0,0 +1,33 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ ~ Licensed to the Apache Software Foundation (ASF) under one or more
+ ~ contributor license agreements. See the NOTICE file distributed with
+ ~ this work for additional information regarding copyright ownership.
+ ~ The ASF licenses this file to You under the Apache License, Version 2.0
+ ~ (the "License"); you may not use this file except in compliance with
+ ~ the License. You may obtain a copy of the License at
+ ~
+ ~ http://www.apache.org/licenses/LICENSE-2.0
+ ~
+ ~ Unless required by applicable law or agreed to in writing, software
+ ~ distributed under the License is distributed on an "AS IS" BASIS,
+ ~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ ~ See the License for the specific language governing permissions and
+ ~ limitations under the License.
+ -->
+
+<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="
+ http://maven.apache.org/SETTINGS/1.0.0
+ http://maven.apache.org/xsd/settings-1.0.0.xsd">
+
+ <servers>
+ <server>
+ <id>apache.snapshots.https</id>
+ <username>${env.NEXUS_DEPLOY_USERNAME}</username>
+ <password>${env.NEXUS_DEPLOY_PASSWORD}</password>
+ </server>
+ </servers>
+
+</settings>
\ No newline at end of file
diff --git a/.github/workflows/license-checker.yaml b/.github/workflows/license-checker.yaml
new file mode 100644
index 0000000..259fdd7
--- /dev/null
+++ b/.github/workflows/license-checker.yaml
@@ -0,0 +1,34 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: License checker
+
+on:
+ pull_request:
+ branches:
+ - develop
+ - master
+
+jobs:
+ check-license:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ - name: Check License Header
+ uses: apache/skywalking-eyes@v0.2.0
+ with:
+ log: info
+ config: .licenserc.yaml
diff --git a/.github/workflows/pr-ci.yml b/.github/workflows/pr-ci.yml
new file mode 100644
index 0000000..5a6980d
--- /dev/null
+++ b/.github/workflows/pr-ci.yml
@@ -0,0 +1,15 @@
+name: PR-CI
+
+on:
+ pull_request:
+ types: [opened, reopened, synchronize]
+
+jobs:
+ dist-tar:
+ name: Build distribution tar
+ runs-on: ubuntu-latest
+ timeout-minutes: 120
+ steps:
+ - uses: actions/checkout@v3
+ with:
+ submodules: true
diff --git a/.github/workflows/pr-e2e-test.yml b/.github/workflows/pr-e2e-test.yml
new file mode 100644
index 0000000..c46e7b5
--- /dev/null
+++ b/.github/workflows/pr-e2e-test.yml
@@ -0,0 +1,136 @@
+name: E2E test for pull request
+
+# read-write repo token
+# access to secrets
+on:
+ workflow_run:
+ workflows: ["PR-CI"]
+ types:
+ - completed
+
+env:
+ DOCKER_REPO: apache/rocketmq-ci
+
+jobs:
+ deploy:
+ if: ${{ success() }}
+ name: Deploy RocketMQ
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: Deploy rocketmq
+ with:
+ action: "deploy"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ chart-git: "https://ghproxy.com/https://github.com/apache/rocketmq-docker.git"
+ chart-branch: "master"
+ chart-path: "./rocketmq-k8s-helm"
+ job-id: ${{ strategy.job-index }}
+ helm-values: |
+ nameserver:
+ image:
+ repository: ${{env.DOCKER_REPO}}
+ tag: ${{ matrix.version }}
+ broker:
+ image:
+ repository: ${{env.DOCKER_REPO}}
+ tag: ${{ matrix.version }}
+ proxy:
+ image:
+ repository: ${{env.DOCKER_REPO}}
+ tag: ${{ matrix.version }}
+
+ e2e-grpc-java-test:
+ if: ${{ success() }}
+ name: e2e-grpc-java-test
+ needs: [ deploy ]
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: e2e test
+ with:
+ action: "test"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ test-code-git: "https://ghproxy.com/https://github.com/apache/rocketmq-e2e.git"
+ test-code-branch: "master"
+ test-code-path: java/e2e
+ test-cmd: "mvn -B test"
+ job-id: 0
+ - name: Publish Test Report
+ uses: mikepenz/action-junit-report@v3
+ if: always() # always run even if the previous step fails
+ with:
+ report_paths: '**/test_report/TEST-*.xml'
+ annotate_only: true
+ include_passed: true
+ detailed_summary: true
+ - uses: actions/upload-artifact@v3
+ if: always()
+ name: Upload test log
+ with:
+ name: test-e2e-grpc-java-log.txt
+ path: testlog.txt
+
+ e2e-remoting-java-test:
+ if: ${{ success() }}
+ name: e2e-remoting-java-test
+ needs: [ deploy ]
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: e2e test
+ with:
+ action: "test"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ test-code-git: "https://ghproxy.com/https://github.com/apache/rocketmq-e2e.git"
+ test-code-branch: "master"
+ test-code-path: java/e2e-v4
+ test-cmd: "mvn -B test"
+ job-id: 0
+ - name: Publish Test Report
+ uses: mikepenz/action-junit-report@v3
+ if: always() # always run even if the previous step fails
+ with:
+ report_paths: '**/test_report/TEST-*.xml'
+ annotate_only: true
+ include_passed: true
+ detailed_summary: true
+ - uses: actions/upload-artifact@v3
+ if: always()
+ name: Upload test log
+ with:
+ name: test-e2e-remoting-java-log.txt
+ path: testlog.txt
+
+ clean:
+ if: always()
+ name: Clean
+ needs: [ e2e-remoting-java-test,e2e-grpc-java-test ]
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: clean
+ with:
+ action: "clean"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ job-id: ${{ strategy.job-index }}
\ No newline at end of file
diff --git a/.github/workflows/push-ci.yml b/.github/workflows/push-ci.yml
new file mode 100644
index 0000000..6196a91
--- /dev/null
+++ b/.github/workflows/push-ci.yml
@@ -0,0 +1,138 @@
+name: PUSH-CI
+
+on:
+ push:
+ branches: [ master ]
+ #schedule:
+ # - cron: "0 18 * * *" # TimeZone: UTC 0
+
+concurrency:
+ group: rocketmq-${{ github.ref }}
+
+env:
+ MAVEN_OPTS: -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false -Dmaven.wagon.httpconnectionManager.ttlSeconds=120
+ DOCKER_REPO: apache/rocketmq
+
+jobs:
+ deploy:
+ if: ${{ success() }}
+ name: Deploy RocketMQ
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: Deploy rocketmq
+ with:
+ action: "deploy"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ chart-git: "https://ghproxy.com/https://github.com/apache/rocketmq-docker.git"
+ chart-branch: "master"
+ chart-path: "./rocketmq-k8s-helm"
+ job-id: ${{ strategy.job-index }}
+ helm-values: |
+ nameserver:
+ image:
+ repository: ${{env.DOCKER_REPO}}
+ tag: ${{ matrix.version }}
+ broker:
+ image:
+ repository: ${{env.DOCKER_REPO}}
+ tag: ${{ matrix.version }}
+ proxy:
+ image:
+ repository: ${{env.DOCKER_REPO}}
+ tag: ${{ matrix.version }}
+
+ e2e-grpc-java-test:
+ if: ${{ success() }}
+ name: e2e-grpc-java-test
+ needs: [ deploy ]
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: e2e test
+ with:
+ action: "test"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ test-code-git: "https://ghproxy.com/https://github.com/apache/rocketmq-e2e.git"
+ test-code-branch: "master"
+ test-code-path: java/e2e
+ test-cmd: "mvn -B test"
+ job-id: 0
+ - name: Publish Test Report
+ uses: mikepenz/action-junit-report@v3
+ if: always() # always run even if the previous step fails
+ with:
+ report_paths: '**/test_report/TEST-*.xml'
+ annotate_only: true
+ include_passed: true
+ detailed_summary: true
+ - uses: actions/upload-artifact@v3
+ if: always()
+ name: Upload test log
+ with:
+ name: test-e2e-grpc-java-log.txt
+ path: testlog.txt
+
+ e2e-remoting-java-test:
+ if: ${{ success() }}
+ name: e2e-remoting-java-test
+ needs: [ deploy ]
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: e2e test
+ with:
+ action: "test"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ test-code-git: "https://ghproxy.com/https://github.com/apache/rocketmq-e2e.git"
+ test-code-branch: "master"
+ test-code-path: java/e2e-v4
+ test-cmd: "mvn -B test"
+ job-id: 0
+ - name: Publish Test Report
+ uses: mikepenz/action-junit-report@v3
+ if: always() # always run even if the previous step fails
+ with:
+ report_paths: '**/test_report/TEST-*.xml'
+ annotate_only: true
+ include_passed: true
+ detailed_summary: true
+ - uses: actions/upload-artifact@v3
+ if: always()
+ name: Upload test log
+ with:
+ name: test-e2e-remoting-java-log.txt
+ path: testlog.txt
+
+ clean:
+ if: always()
+ name: Clean
+ needs: [ e2e-remoting-java-test,e2e-grpc-java-test ]
+ runs-on: ubuntu-latest
+ timeout-minutes: 60
+ strategy:
+ matrix:
+ version: [ 5.1.0 ]
+ steps:
+ - uses: apache/rocketmq-test-tool@07147d677bc032edf30eee27fe94c0f531e075a2
+ name: clean
+ with:
+ action: "clean"
+ ask-config: "${{ secrets.ASK_CONFIG_VIRGINA }}"
+ test-version: "${{ matrix.version }}"
+ job-id: ${{ strategy.job-index }}
diff --git a/README.md b/README.md
index d37fc5f..3d0bf9a 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
-## Apache RocketMQ E2E
+## Apache RocketMQ E2E
[](https://www.apache.org/licenses/LICENSE-2.0.html)
-RocketMQ E2E Test
+RocketMQ E2E Test
### Test Case Coverage
* Message Type
@@ -15,7 +15,7 @@ RocketMQ E2E Test
* PushConsumer
* SimpleConsumer
* Order/Delay/Transaction/Normal
- * Sync receive/Async receive
+ * Sync receive/Async receive
* Sync ack/Async ack
* Client init (Producer/PushConsumer/SimpleConsumer)
* Parameter settings
@@ -39,6 +39,6 @@ RocketMQ E2E Test
mvn clean test -B -Dgroups=smoke -Dcluster=DefaultCluster
```
##### Options
-* `ALL_IP` : not required, default is null
+* `ALL_IP` : not required, default is null
* `cluster`: not required, default `DefaultCluster`
-* `groups`: param of junit5, `src/main/java/org/apache/rocketmq/enums/TESTSET.java`
\ No newline at end of file
+* `groups`: param of junit5, `src/main/java/org/apache/rocketmq/enums/TESTSET.java`
diff --git a/java/e2e-v4/pom.xml b/java/e2e-v4/pom.xml
index 13874d2..e10d53b 100644
--- a/java/e2e-v4/pom.xml
+++ b/java/e2e-v4/pom.xml
@@ -130,6 +130,7 @@
<configuration>
<argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
<rerunFailingTestsCount>1</rerunFailingTestsCount>
+ <trimStackTrace>false</trimStackTrace>
</configuration>
</plugin>
<plugin>
diff --git a/java/e2e/pom.xml b/java/e2e/pom.xml
index 76ba5bf..ed3b404 100644
--- a/java/e2e/pom.xml
+++ b/java/e2e/pom.xml
@@ -146,6 +146,7 @@
<version>3.0.0-M5</version>
<configuration>
<argLine>-Xmx1024m -XX:MaxPermSize=256m</argLine>
+ <trimStackTrace>false</trimStackTrace>
<!-- <rerunFailingTestsCount>1</rerunFailingTestsCount>-->
</configuration>
</plugin>
diff --git a/java/e2e/src/main/java/org/apache/rocketmq/util/VerifyUtils.java b/java/e2e/src/main/java/org/apache/rocketmq/util/VerifyUtils.java
index 8b94073..5d3a7ca 100644
--- a/java/e2e/src/main/java/org/apache/rocketmq/util/VerifyUtils.java
+++ b/java/e2e/src/main/java/org/apache/rocketmq/util/VerifyUtils.java
@@ -316,7 +316,7 @@ public class VerifyUtils {
long consumeTime = System.currentTimeMillis();
for (Object receivedMessage : receivedMessages) {
MessageView messageView = (MessageView) receivedMessage;
-// Assertions.assertTrue(messageView.getDeliveryTimestamp().isPresent());
+ Assertions.assertTrue(messageView.getDeliveryTimestamp().isPresent(),"DeliveryTimestamp is empty");
//Check the current time and the distribution time. If the difference is within 5s, the requirements are met
long bornTimestamp = messageView.getBornTimestamp();
if (Math.abs((consumeTime - bornTimestamp) / 1000 - delayTimeSec) > 5) {