You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/08 08:54:10 UTC
[I] [Bug] The StorageManager cache might not function effectively under heavy IO pressure [incubator-uniffle]
rickyma opened a new issue, #1626:
URL: https://github.com/apache/incubator-uniffle/issues/1626
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### Describe the bug
I think the following code snipet in `HybridStorageManager` could be refactored after https://github.com/apache/incubator-uniffle/pull/383:
```
public boolean write(Storage storage, ShuffleWriteHandler handler, ShuffleDataFlushEvent event) {
StorageManager underStorageManager = eventOfUnderStorageManagers.getIfPresent(event);
if (underStorageManager == null) {
return false;
}
return underStorageManager.write(storage, handler, event);
}
```
When we cannot obtain the cached storageManager from `eventOfUnderStorageManagers`, we should try other methods to acquire the storageManager and refresh the cache, instead of directly returning `false`.
Currently, the actual write logic is not truly executed.
There may be a scenario where retry failures occur rapidly, and such retries are meaningless:
<img width="1720" alt="35879725f6a9ba276ca6a37c5249f29" src="https://github.com/apache/incubator-uniffle/assets/13834479/94b7034a-5ebe-422a-a56a-07b5676cda88">
### Affects Version(s)
master
### Uniffle Server Log Output
_No response_
### Uniffle Engine Log Output
_No response_
### Uniffle Server Configurations
_No response_
### Uniffle Engine Configurations
_No response_
### Additional context
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org
Re: [I] [Bug] The StorageManager cache might not function effectively under heavy IO pressure [incubator-uniffle]
Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston closed issue #1626: [Bug] The StorageManager cache might not function effectively under heavy IO pressure
URL: https://github.com/apache/incubator-uniffle/issues/1626
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org