You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@uniffle.apache.org by GitBox <gi...@apache.org> on 2023/01/04 08:49:46 UTC
[GitHub] [incubator-uniffle] xianjingfeng opened a new issue, #455: [Improvement] Lazily create `uncompressedData`
xianjingfeng opened a new issue, #455:
URL: https://github.com/apache/incubator-uniffle/issues/455
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.
### What would you like to be improved?
This part as follows will cost much memory if we use AQE. We should create `uncompressedData` lazily.
https://github.com/apache/incubator-uniffle/blob/9572b8423fff8f3dfa52411dae1caa4d33f08663/client-spark/common/src/main/java/org/apache/spark/shuffle/reader/RssShuffleDataIterator.java#L73-L78
### How should we improve?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] xianjingfeng commented on issue #455: [Improvement] Lazily create `uncompressedData`
Posted by GitBox <gi...@apache.org>.
xianjingfeng commented on issue #455:
URL: https://github.com/apache/incubator-uniffle/issues/455#issuecomment-1370632006
PTAL @zuston
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi closed issue #455: [Improvement] Lazily create `uncompressedData`
Posted by GitBox <gi...@apache.org>.
jerqi closed issue #455: [Improvement] Lazily create `uncompressedData`
URL: https://github.com/apache/incubator-uniffle/issues/455
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] zuston commented on issue #455: [Improvement] Lazily create `uncompressedData`
Posted by GitBox <gi...@apache.org>.
zuston commented on issue #455:
URL: https://github.com/apache/incubator-uniffle/issues/455#issuecomment-1370745842
Nice catch, could you help fix this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-uniffle] jerqi commented on issue #455: [Improvement] Lazily create `uncompressedData`
Posted by GitBox <gi...@apache.org>.
jerqi commented on issue #455:
URL: https://github.com/apache/incubator-uniffle/issues/455#issuecomment-1373458127
closed by #457
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@uniffle.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org