You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by GitBox <gi...@apache.org> on 2023/01/10 02:35:42 UTC
[GitHub] [incubator-uniffle] kaijchen opened a new pull request, #463: [ISSUE-461] Add spark 3.3 profile and CI workflow
kaijchen opened a new pull request, #463:
URL: https://github.com/apache/incubator-uniffle/pull/463
### What changes were proposed in this pull request?
1. Add Spark 3.3 profile
2. Enable tests for Spark 3.3 in CI
3. Fix broken tests
### Why are the changes needed?
#461
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
https://github.com/kaijchen/incubator-uniffle/actions/runs/3879507232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] advancedxy commented on a diff in pull request #463: [ISSUE-461] Add spark 3.3 profile and CI workflow
Posted by GitBox <gi...@apache.org>.
advancedxy commented on code in PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#discussion_r1065309053
##########
.github/workflows/parallel.yml:
##########
@@ -62,6 +62,7 @@ jobs:
- spark3
- spark3.2
- spark3.2.0
+ - spark3.3
Review Comment:
This profile list is extremely strange.. Is it possible to do some refactor and make the profile name consistent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] kaijchen commented on a diff in pull request #463: [ISSUE-461] Add spark 3.3 profile and CI workflow
Posted by GitBox <gi...@apache.org>.
kaijchen commented on code in PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#discussion_r1065310397
##########
.github/workflows/parallel.yml:
##########
@@ -62,6 +62,7 @@ jobs:
- spark3
- spark3.2
- spark3.2.0
+ - spark3.3
Review Comment:
`spark3` is actually Spark 3.1.
`spark3.2.0` is listed apart from `spark3.2` due to some unexpected API change.
We should change it in another PR (if needed).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] advancedxy commented on a diff in pull request #463: [ISSUE-461] Support Spark 3.3
Posted by GitBox <gi...@apache.org>.
advancedxy commented on code in PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#discussion_r1065650890
##########
.github/workflows/parallel.yml:
##########
@@ -62,6 +62,7 @@ jobs:
- spark3
- spark3.2
- spark3.2.0
+ - spark3.3
Review Comment:
Would you create an issue for that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] kaijchen commented on a diff in pull request #463: [ISSUE-461] Add spark 3.3 profile and CI workflow
Posted by GitBox <gi...@apache.org>.
kaijchen commented on code in PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#discussion_r1065310397
##########
.github/workflows/parallel.yml:
##########
@@ -62,6 +62,7 @@ jobs:
- spark3
- spark3.2
- spark3.2.0
+ - spark3.3
Review Comment:
`spark3` is actually Spark 3.1.
`spark3.2.0` is listed apart from `spark3.2` due to some accidental API change.
We should change it in another PR (if needed).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] codecov-commenter commented on pull request #463: [ISSUE-461] Add spark 3.3 profile and CI workflow
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#issuecomment-1376655883
# [Codecov](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#463](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c015168) into [master](https://codecov.io/gh/apache/incubator-uniffle/commit/ebaff6aa4eccfe4f54a967bc271da7e345f6c34a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ebaff6a) will **decrease** coverage by `0.66%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #463 +/- ##
============================================
- Coverage 58.74% 58.08% -0.67%
+ Complexity 1664 1445 -219
============================================
Files 199 177 -22
Lines 11236 9399 -1837
Branches 999 839 -160
============================================
- Hits 6601 5459 -1142
+ Misses 4243 3591 -652
+ Partials 392 349 -43
```
| [Impacted Files](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...org/apache/spark/shuffle/writer/AddBlockEvent.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvQWRkQmxvY2tFdmVudC5qYXZh) | | |
| [...preduce/task/reduce/RssRemoteMergeManagerImpl.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL3Rhc2svcmVkdWNlL1Jzc1JlbW90ZU1lcmdlTWFuYWdlckltcGwuamF2YQ==) | | |
| [...rg/apache/hadoop/mapred/RssMapOutputCollector.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkL1Jzc01hcE91dHB1dENvbGxlY3Rvci5qYXZh) | | |
| [...n/java/org/apache/hadoop/mapreduce/MRIdHelper.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL01SSWRIZWxwZXIuamF2YQ==) | | |
| [.../org/apache/spark/shuffle/writer/WriterBuffer.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvV3JpdGVyQnVmZmVyLmphdmE=) | | |
| [.../java/org/apache/spark/shuffle/RssSparkConfig.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS9Sc3NTcGFya0NvbmZpZy5qYXZh) | | |
| [...n/java/org/apache/hadoop/mapreduce/RssMRUtils.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL1Jzc01SVXRpbHMuamF2YQ==) | | |
| [...pache/hadoop/mapreduce/task/reduce/RssFetcher.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL3Rhc2svcmVkdWNlL1Jzc0ZldGNoZXIuamF2YQ==) | | |
| [...mapreduce/task/reduce/RssInMemoryRemoteMerger.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LW1yL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9oYWRvb3AvbWFwcmVkdWNlL3Rhc2svcmVkdWNlL1Jzc0luTWVtb3J5UmVtb3RlTWVyZ2VyLmphdmE=) | | |
| [...k/shuffle/writer/WrappedByteArrayOutputStream.java](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-Y2xpZW50LXNwYXJrL2NvbW1vbi9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvc3Bhcmsvc2h1ZmZsZS93cml0ZXIvV3JhcHBlZEJ5dGVBcnJheU91dHB1dFN0cmVhbS5qYXZh) | | |
| ... and [12 more](https://codecov.io/gh/apache/incubator-uniffle/pull/463?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] kaijchen merged pull request #463: [ISSUE-461] Support Spark 3.3
Posted by GitBox <gi...@apache.org>.
kaijchen merged PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] kaijchen commented on a diff in pull request #463: [ISSUE-461] Add spark 3.3 profile and CI workflow
Posted by GitBox <gi...@apache.org>.
kaijchen commented on code in PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#discussion_r1065310397
##########
.github/workflows/parallel.yml:
##########
@@ -62,6 +62,7 @@ jobs:
- spark3
- spark3.2
- spark3.2.0
+ - spark3.3
Review Comment:
`spark3` is actually Spark 3.1.
`spark3.2.0` is listed apart from `spark3.2` due to some accidental API change.
We should change is in another PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
[GitHub] [incubator-uniffle] kaijchen commented on pull request #463: [ISSUE-461] Support Spark 3.3
Posted by GitBox <gi...@apache.org>.
kaijchen commented on PR #463:
URL: https://github.com/apache/incubator-uniffle/pull/463#issuecomment-1377077651
Thanks @jerqi and @zuston for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org