You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2022/05/11 08:33:16 UTC
[GitHub] [systemds] ywcb00 opened a new pull request, #1611: [SYSTEMDS-3185] Federated Response Serialization Caching
ywcb00 opened a new pull request, #1611:
URL: https://github.com/apache/systemds/pull/1611
Hi,
This PR introduces the reuse of serialized federated responses. In order to avoid redundant serialization, we put the serialized byte array of federated responses containing a CacheBlock into the lineage cache and mark it with a separate lineage item for serialization derived from the lineage item of the underlying data. The next time the same data is requested, the serialization process for the federated response is skipped and the cached bytes are loaded directly into the output buffer.
The implementation overrides the encode method of netty's ObjectEncoder to check whether we can reuse the serialization prior to the actual encoding, and to obtain and store the serialized bytes after the encoding.
Thanks for review :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] mboehm7 commented on pull request #1611: [SYSTEMDS-3185] Federated Response Serialization Caching
Posted by GitBox <gi...@apache.org>.
mboehm7 commented on PR #1611:
URL: https://github.com/apache/systemds/pull/1611#issuecomment-1126960424
LGTM - thanks @ywcb00 for the patch. During the merge, I fixed a few unnecessary imports, formatting, and the stringbuilder handling in federated statistics.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] asfgit closed pull request #1611: [SYSTEMDS-3185] Federated Response Serialization Caching
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #1611: [SYSTEMDS-3185] Federated Response Serialization Caching
URL: https://github.com/apache/systemds/pull/1611
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@systemds.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org