You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by "SbloodyS (via GitHub)" <gi...@apache.org> on 2024/09/05 09:22:47 UTC
[PR] [Chore] Improve the robustness of CI [dolphinscheduler]
SbloodyS opened a new pull request, #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592
<!--Thanks very much for contributing to Apache DolphinScheduler, we are happy that you want to help us improve DolphinScheduler! -->
## Purpose of the pull request
<!--(For example: This pull request adds checkstyle plugin).-->
## Brief change log
- Fix the instability in https://github.com/apache/dolphinscheduler/actions/runs/10715805776/job/29714737751?pr=16590
- Reuse maven cache
## Verify this pull request
<!--*(Please pick either of the following options)*-->
This pull request is code cleanup without any test coverage.
*(or)*
This pull request is already covered by existing tests, such as *(please describe tests)*.
(or)
This change added tests and can be verified as follows:
<!--*(example:)*
- *Added dolphinscheduler-dao tests for end-to-end.*
- *Added CronUtilsTest to verify the change.*
- *Manually verified the change by testing locally.* -->
(or)
## Pull Request Notice
[Pull Request Notice](https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md)
If your pull request contain incompatible change, you should also add it to `docs/docs/en/guide/upgrede/incompatible.md`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746377531
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
not sure use tsinghua mirrors is good idea or not
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746405947
##########
.github/workflows/e2e.yml:
##########
@@ -151,7 +151,7 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
- key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-e2e
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
Review Comment:
Thanks for clarification.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746381939
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
how about using apache cdn, such as `https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-bin.tar.gz` for version 3.2.2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746404804
##########
.github/workflows/e2e.yml:
##########
@@ -151,7 +151,7 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
- key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-e2e
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
Review Comment:
I have half a memory we have some error with the same cache key https://github.com/apache/dolphinscheduler/pull/14808 but I am not sure, maybe we can keep this change to see whether regression in the further
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746385107
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
> how about using apache cdn, such as `https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-bin.tar.gz` for version 3.2.2?
And how about apache official cdn?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2331653800
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2331655542
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2331814018
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746379303
##########
.github/workflows/e2e.yml:
##########
@@ -151,7 +151,7 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
- key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-e2e
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
Review Comment:
e2e has different pom.xml with the backend module, as you can see it uses `dolphinscheduler-e2e/pom.xml` instead of `/pom.xml`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746386380
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
> how about using apache cdn, such as `https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-bin.tar.gz` for version 3.2.2?
This is the same speed as `This is the same speed as archive`...
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
> how about using apache cdn, such as `https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-bin.tar.gz` for version 3.2.2?
This is the same speed as `archive.apache.org`...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746381258
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
> not sure use tsinghua mirrors is good idea or not
After I tested a dozen mirrors, I found this source to be the fastest in github action.
> if accept can we add some desc in L144?
It seems comments don't seem to be supported in `run`. I tried, but I got an error...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746382572
##########
.github/workflows/e2e.yml:
##########
@@ -151,7 +151,7 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
- key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-e2e
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
Review Comment:
It doesn't matter. The maven cache path is `~/.m2/repository`. The old dependency cache is retained and new dependencies are added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746383323
##########
.github/workflows/api-test.yml:
##########
@@ -62,7 +62,7 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
- key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-api-test
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
Review Comment:
The total cache storage limit is 10GB. We add binary package to it will cause overuse. So I reuse it. Please refer to the doc in
https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2332983630
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2332984691
## [](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592) **Quality Gate passed**
Issues
 [0 New issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0 Accepted issues](https://sonarcloud.io/project/issues?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=ACCEPTED)
Measures
 [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache-dolphinscheduler&pullRequest=16592&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true)
 [0.0% Coverage on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_coverage&view=list)
 [0.0% Duplication on New Code](https://sonarcloud.io/component_measures?id=apache-dolphinscheduler&pullRequest=16592&metric=new_duplicated_lines_density&view=list)
[See analysis details on SonarCloud](https://sonarcloud.io/dashboard?id=apache-dolphinscheduler&pullRequest=16592)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2331815035
Please retry analysis of this Pull-Request directly on SonarCloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "SbloodyS (via GitHub)" <gi...@apache.org>.
SbloodyS commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1745597675
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
Since `https://archive.apache.org` download speed is too slow and often causes timeout failures. So using `https://mirrors.tuna.tsinghua.edu.cn` to do the cache instead.
Notice: This site does not contains the archived versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "Gallardot (via GitHub)" <gi...@apache.org>.
Gallardot merged PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746399000
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
all right, so maybe we can keep tsinghua mirror here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "sonarcloud[bot] (via GitHub)" <gi...@apache.org>.
sonarcloud[bot] commented on PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#issuecomment-2331149806
Please retry analysis of this Pull-Request directly on SonarCloud
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "ruanwenjun (via GitHub)" <gi...@apache.org>.
ruanwenjun commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746381965
##########
.github/workflows/api-test.yml:
##########
@@ -62,7 +62,7 @@ jobs:
uses: actions/cache@v4
with:
path: ~/.m2/repository
- key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-api-test
+ key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}-backend
Review Comment:
Why change the cache key?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746384869
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
> It seems comments don't seem to be supported in run. I tried, but I got an error...
I mean we and comment before the line for `run` property.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Chore] Improve the robustness of CI [dolphinscheduler]
Posted by "zhongjiajie (via GitHub)" <gi...@apache.org>.
zhongjiajie commented on code in PR #16592:
URL: https://github.com/apache/dolphinscheduler/pull/16592#discussion_r1746377960
##########
.github/workflows/backend.yml:
##########
@@ -131,9 +124,39 @@ jobs:
- name: Running cluster test
run: |
/bin/bash ${{ matrix.case.script }}
+ schema-check-prepare:
+ name: schema-check-prepare-${{ matrix.version }}
+ needs: paths-filter
+ runs-on: ubuntu-latest
+ timeout-minutes: 30
+ strategy:
+ fail-fast: false
+ matrix:
+ version: [ "3.1.9", "3.2.0" ]
+ steps:
+ - uses: actions/cache/restore@v4
+ name: Download Binary Package Cache
+ with:
+ path: /tmp/${{ matrix.version }}
+ key: apache-dolphinscheduler-${{ matrix.version }}-bin
+ restore-keys: apache-dolphinscheduler-${{ matrix.version }}-bin
+ - name: Download Binary Package if not cached
+ run: |
+ if [ ! -f /tmp/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz ]; then
+ echo "Binary package not found in cache, downloading..."
+ mkdir -p /tmp/${{ matrix.version }}
+ wget https://mirrors.tuna.tsinghua.edu.cn/apache/dolphinscheduler/${{ matrix.version }}/apache-dolphinscheduler-${{ matrix.version }}-bin.tar.gz -P /tmp/${{ matrix.version }}
Review Comment:
if accept can we add some desc in L144?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org