You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "rickyma (via GitHub)" <gi...@apache.org> on 2024/07/03 06:20:42 UTC
[PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
rickyma opened a new pull request, #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859
### What changes were proposed in this pull request?
Introduce a fine-grained force commit switch.
### Why are the changes needed?
For https://github.com/apache/incubator-uniffle/issues/1858.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "rickyma (via GitHub)" <gi...@apache.org>.
rickyma commented on PR #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859#issuecomment-2205326046
For a single app, indeed there is a performance decrease, but if running a large number of apps simultaneously, the overall throughput remains the same. We provide an optional choice, and the default value is false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859#issuecomment-2205236533
It will have bad performance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "rickyma (via GitHub)" <gi...@apache.org>.
rickyma commented on PR #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859#issuecomment-2216843642
Closed. We don't use this flag anymore.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859#issuecomment-2205224671
## Test Results
2 657 files ±0 2 657 suites ±0 5h 31m 35s :stopwatch: +15s
946 tests ±0 945 :white_check_mark: ±0 1 :zzz: ±0 0 :x: ±0
11 789 runs ±0 11 774 :white_check_mark: ±0 15 :zzz: ±0 0 :x: ±0
Results for commit 67a2e7df. ± Comparison against base commit 88152e98.
[test-results]:data:application/gzip;base64,H4sIADz1hGYC/03PSw7CIBhG0a00jB1A5elmDM+E2BZDYWTcu3+bggy/A9yEDwpx8Tt6TDNn4jahvcYyblezLjFtIERJxYDgvBw3FOVtPfdq7UnsT6/4Pl51CDouALiDzznlS3LdjiYhQqprtiiYoN2uKmswVM89Rm1a11hgIC707IUL1MtgLPZ3h4mR8EWFqZsJNUQK6i1H3x8MvWwCEgEAAA==
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "rickyma (via GitHub)" <gi...@apache.org>.
rickyma commented on PR #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859#issuecomment-2205432145
This can only indicate that there are not enough running apps. When there are enough apps, no app's shuffle data can stay in memory for a long time. Therefore, in this scenario, the performance will not be affected very much, and only in this scenario will we consider enabling this switch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "rickyma (via GitHub)" <gi...@apache.org>.
rickyma closed pull request #1859: [#1858] feat(spark): Introduce a fine-grained force commit switch
URL: https://github.com/apache/incubator-uniffle/pull/1859
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [PR] [#1858] feat(spark): Introduce a fine-grained force commit switch [incubator-uniffle]
Posted by "jerqi (via GitHub)" <gi...@apache.org>.
jerqi commented on PR #1859:
URL: https://github.com/apache/incubator-uniffle/pull/1859#issuecomment-2205381183
> For a single app, indeed there is a performance decrease, but if running a large number of apps simultaneously, the overall throughput remains the same. We provide an optional choice, and the default value is false.
If we running multiple applications, the performance will be worse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org