You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by GitBox <gi...@apache.org> on 2022/01/09 07:56:38 UTC
[GitHub] [incubator-seatunnel] asdf2014 opened a new pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
asdf2014 opened a new pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984
<!--
Thank you for contributing to SeaTunnel! Please make sure that your code changes
are covered with tests. And in case of new features or big changes
remember to adjust the documentation.
Feel free to ping committers for the review!
## Contribution Checklist
- Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/incubator-seatunnel/issues).
- Name the pull request in the form "[SeaTunnel #XXXX] [component] Title of the pull request", where *SeaTunnel #XXXX* should be replaced by the actual issue number.
- Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
-->
## Purpose of this pull request
<!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
Fixes #983
## Check list
* [x] Code changed are covered with tests, or it does not need tests for reason:
* [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#issuecomment-1008257306
Looks like this problem still not be fixed in the latest version of `httpcomponents`. FYI, https://issues.apache.org/jira/browse/HTTPCORE-634
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#issuecomment-1008248922
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on a change in pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on a change in pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#discussion_r780877319
##########
File path: .github/workflows/backend.yml
##########
@@ -52,7 +52,7 @@ jobs:
key: ${{ runner.os }}-maven
- name: Check Style
run: |
- ./mvnw -B checkstyle:check
+ ./mvnw -T 16 -B checkstyle:check
Review comment:
Sure, let me address
##########
File path: .github/workflows/backend.yml
##########
@@ -64,4 +64,4 @@ jobs:
-Dmaven.wagon.httpconnectionManager.ttlSeconds=120
- name: Run Unit tests
run: |
- ./mvnw -B clean verify -Dmaven.test.skip=false -Dgpg.skip=true
+ ./mvnw -T 16 -B clean verify -Dmaven.test.skip=false -Dgpg.skip=true
Review comment:
Sure, let me address
##########
File path: .github/workflows/publish-docker.yaml
##########
@@ -61,7 +61,7 @@ jobs:
env:
MAVEN_OPTS: -Xmx2G -Xms2G
run: |
- ./mvnw -B clean deploy \
+ ./mvnw -T 16 -B clean deploy \
Review comment:
Sure, let me remove it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#issuecomment-1008249163
It's weird, because this dependency is right there. FYI, https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpasyncclient/4.1.4/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] leo65535 commented on pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
leo65535 commented on pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#issuecomment-1008502612
Overall LGTM, left some places need to be updated in this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] leo65535 commented on a change in pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
leo65535 commented on a change in pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#discussion_r780871619
##########
File path: .github/workflows/backend.yml
##########
@@ -52,7 +52,7 @@ jobs:
key: ${{ runner.os }}-maven
- name: Check Style
run: |
- ./mvnw -B checkstyle:check
+ ./mvnw -T 16 -B checkstyle:check
Review comment:
hi @asdf2014, 16 threads is too big, let's use `-T 2C`.
##########
File path: .github/workflows/backend.yml
##########
@@ -64,4 +64,4 @@ jobs:
-Dmaven.wagon.httpconnectionManager.ttlSeconds=120
- name: Run Unit tests
run: |
- ./mvnw -B clean verify -Dmaven.test.skip=false -Dgpg.skip=true
+ ./mvnw -T 16 -B clean verify -Dmaven.test.skip=false -Dgpg.skip=true
Review comment:
ditto.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on a change in pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on a change in pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#discussion_r780884823
##########
File path: .github/workflows/backend.yml
##########
@@ -64,4 +64,4 @@ jobs:
-Dmaven.wagon.httpconnectionManager.ttlSeconds=120
- name: Run Unit tests
run: |
- ./mvnw -B clean verify -Dmaven.test.skip=false -Dgpg.skip=true
+ ./mvnw -T 16 -B clean verify -Dmaven.test.skip=false -Dgpg.skip=true
Review comment:
Same as above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#issuecomment-1008516914
Hi, @leo65535 . Thanks a lot for your comments, which have been addressed. PTAL.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on a change in pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on a change in pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#discussion_r780885227
##########
File path: .github/workflows/publish-docker.yaml
##########
@@ -61,7 +61,7 @@ jobs:
env:
MAVEN_OPTS: -Xmx2G -Xms2G
run: |
- ./mvnw -B clean deploy \
+ ./mvnw -T 16 -B clean deploy \
Review comment:
@leo65535 Thanks a lot for your comment. However, I didn't fully understand what's your concern. Would you please explain more details. Anyway, I have removed this in case something goes wrong.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on a change in pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on a change in pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#discussion_r780884653
##########
File path: .github/workflows/backend.yml
##########
@@ -52,7 +52,7 @@ jobs:
key: ${{ runner.os }}-maven
- name: Check Style
run: |
- ./mvnw -B checkstyle:check
+ ./mvnw -T 16 -B checkstyle:check
Review comment:
@leo65535 Actually, the `-T 16` option is suitable, when the number of our connectors becomes larger, it will be very necessary, and 16 threads is used by other projects as well, for example, Apache Pinot. Anyway, I reduced this number of threads for now, and will raise up another PR to increase it when it is not enough later.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] CalvinKirs merged pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
CalvinKirs merged pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] leo65535 commented on a change in pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
leo65535 commented on a change in pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#discussion_r780871814
##########
File path: .github/workflows/publish-docker.yaml
##########
@@ -61,7 +61,7 @@ jobs:
env:
MAVEN_OPTS: -Xmx2G -Xms2G
run: |
- ./mvnw -B clean deploy \
+ ./mvnw -T 16 -B clean deploy \
Review comment:
`-T` is an unsafe option, I think we shouldn't use it in deploy command.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-seatunnel] asdf2014 commented on pull request #984: [SeaTunnel#983] Speed up CI/CD with multi threads
Posted by GitBox <gi...@apache.org>.
asdf2014 commented on pull request #984:
URL: https://github.com/apache/incubator-seatunnel/pull/984#issuecomment-1008257060
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org