You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "xianjingfeng (via GitHub)" <gi...@apache.org> on 2025/04/02 09:03:51 UTC
[PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
xianjingfeng opened a new pull request, #2427:
URL: https://github.com/apache/uniffle/pull/2427
# NOTICE: Please remove all these generated template comments before request review(include this line)
<!--
1. Title: [#<issue>] <type>(<scope>): <subject>
Examples:
- "[#123] feat(operator): support xxx"
- "[#123] feat(server,coordinator): support xxx"
- "[#123] feat(common): support xxx"
- "[MINOR] improvement(script): fix style"
- "[MINOR] improvement(ci): add some examples and notice"
- "[#233] fix: check null before access result in xxx"
- "[#233][FOLLOWUP] improvement(dashboard): display something xxx"
- "[MINOR] refactor: fix typo in variable name"
- "[MINOR] docs: fix typo in README"
- "[#255] test: fix flaky test NameOfTheTest"
Reference: https://www.conventionalcommits.org/en/v1.0.0/
2. Contributor guidelines:
https://github.com/apache/incubator-uniffle/blob/master/CONTRIBUTING.md
3. If the PR is unfinished, please mark this PR as draft.
-->
### What changes were proposed in this pull request?
<!--
(Please outline the changes and how this PR fixes the issue.)
-->
Skip the application which is expired when flushing buffers.
### Why are the changes needed?
<!--
(Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, describe the bug.)
-->
All RPC executors are blocked when flushBuffer and removeResources are triggered at the same time in the same app.
Fix: #2426
### Does this PR introduce _any_ user-facing change?
<!--
(Please list the user-facing changes introduced by your change, including
1. Change in user-facing APIs.
2. Addition or removal of property keys.)
-->
No.
### How was this patch tested?
<!--
(Please test your changes, and provide instructions on how to test it:
1. If you add a feature or fix a bug, add a test to cover your changes.
2. If you fix a flaky test, repeat it for many times to prove it works.)
-->
Existing UTs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng closed pull request #2427: [#2426] Skip the application which is expired when flushing buffers
URL: https://github.com/apache/uniffle/pull/2427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #2427:
URL: https://github.com/apache/uniffle/pull/2427#issuecomment-2772230102
CI failed. https://github.com/apache/uniffle/actions/runs/14215737936 Could you fix this error first?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng closed pull request #2427: [#2426] Skip the application which is expired when flushing buffers
URL: https://github.com/apache/uniffle/pull/2427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] improvement: Skip the application which is expired when flushing buffers [uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng commented on PR #2427:
URL: https://github.com/apache/uniffle/pull/2427#issuecomment-2788766204
Thanks for your review. @zuston @jerqi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2427:
URL: https://github.com/apache/uniffle/pull/2427#issuecomment-2774953287
## Test Results
3 018 files - 1 3 018 suites - 1 6h 44m 8s ⏱️ - 1m 17s
1 177 tests - 1 1 175 ✅ - 2 1 💤 ±0 0 ❌ ±0 1 🔥 +1
14 921 runs - 1 14 905 ✅ - 2 15 💤 ±0 0 ❌ ±0 1 🔥 +1
For more details on these errors, see [this check](https://github.com/apache/uniffle/runs/39904411262).
Results for commit f80f0ba3. ± Comparison against base commit 6085a7df.
<details>
<summary>This pull request <b>removes</b> 1 test.</summary>
```
org.apache.uniffle.test.GetReaderTest ‑ test{File}
```
</details>
[test-results]:data:application/gzip;base64,H4sIAHhN7mcC/03Myw7CIBCF4VdpWLvg6kx9GQMUEmIvhsKq8d2ltUWW55vJvxEfRreSRycow1tH1hxSu4ccdQrLXIRLLncq97R/MAZwzeearf2Z+tsrvHer4HUYC9AKLsYlni8xz0dV9vyatSp7qqqd1QpN9dht1C7TFFIZxCP11GihJHLvBhAcJVIcwFlQ2gtzB4UGgHy+ZQMD7BQBAAA=
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] improvement: Skip the application which is expired when flushing buffers [uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng merged PR #2427:
URL: https://github.com/apache/uniffle/pull/2427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng commented on PR #2427:
URL: https://github.com/apache/uniffle/pull/2427#issuecomment-2774425744
> CI failed. https://github.com/apache/uniffle/actions/runs/14215737936 Could you fix this error first?
Do you know how to fix this issue?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #2427:
URL: https://github.com/apache/uniffle/pull/2427#issuecomment-2774594721
> > CI failed. https://github.com/apache/uniffle/actions/runs/14215737936 Could you fix this error first?
>
> Do you know how to fix this issue?
You can update the `dorny/paths-filter@v2 ` in the CI pipeline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on code in PR #2427:
URL: https://github.com/apache/uniffle/pull/2427#discussion_r2024635625
##########
server/src/main/java/org/apache/uniffle/server/ShuffleServerConf.java:
##########
@@ -480,6 +480,14 @@ public class ShuffleServerConf extends RssBaseConf {
+ " The cpu usage of the shuffle server will be reduced."
+ " But SKIP_LIST doesn't support the slow-start feature of MR.");
+ public static final ConfigOption<Integer> SERVER_SHUFFLE_FLUSH_TRYLOCK_TIMEOUT =
+ ConfigOptions.key("rss.server.shuffleBuffer.flush.tryLock.timeout.ms")
Review Comment:
Maybe you can use `"rss.server.flush.tryLockTimeoutMs`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#2426] Skip the application which is expired when flushing buffers [uniffle]
Posted by "xianjingfeng (via GitHub)" <gi...@apache.org>.
xianjingfeng closed pull request #2427: [#2426] Skip the application which is expired when flushing buffers
URL: https://github.com/apache/uniffle/pull/2427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org