You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "rickyma (via GitHub)" <gi...@apache.org> on 2024/04/08 08:54:10 UTC

[I] [Bug] The StorageManager cache might not function effectively under heavy IO pressure [incubator-uniffle]

rickyma opened a new issue, #1626:
URL: https://github.com/apache/incubator-uniffle/issues/1626

   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
   
   
   ### Search before asking
   
   - [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
   
   
   ### Describe the bug
   
   I think the following code snipet in `HybridStorageManager` could be refactored after https://github.com/apache/incubator-uniffle/pull/383:
   ```
   public boolean write(Storage storage, ShuffleWriteHandler handler, ShuffleDataFlushEvent event) {
     StorageManager underStorageManager = eventOfUnderStorageManagers.getIfPresent(event);
     if (underStorageManager == null) {
       return false;
     }
     return underStorageManager.write(storage, handler, event);
   }
   ```
   When we cannot obtain the cached storageManager from `eventOfUnderStorageManagers`, we should try other methods to acquire the storageManager and refresh the cache, instead of directly returning `false`. 
   
   Currently, the actual write logic is not truly executed.
   There may be a scenario where retry failures occur rapidly, and such retries are meaningless:
   <img width="1720" alt="35879725f6a9ba276ca6a37c5249f29" src="https://github.com/apache/incubator-uniffle/assets/13834479/94b7034a-5ebe-422a-a56a-07b5676cda88">
   
   
   ### Affects Version(s)
   
   master
   
   ### Uniffle Server Log Output
   
   _No response_
   
   ### Uniffle Engine Log Output
   
   _No response_
   
   ### Uniffle Server Configurations
   
   _No response_
   
   ### Uniffle Engine Configurations
   
   _No response_
   
   ### Additional context
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [I] [Bug] The StorageManager cache might not function effectively under heavy IO pressure [incubator-uniffle]

Posted by "zuston (via GitHub)" <gi...@apache.org>.
zuston closed issue #1626: [Bug] The StorageManager cache might not function effectively under heavy IO pressure
URL: https://github.com/apache/incubator-uniffle/issues/1626


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org