You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@gluten.apache.org by "acvictor (via GitHub)" <gi...@apache.org> on 2026/02/19 12:32:28 UTC
[PR] [VL] Reduce triple map lookup to single find in dictionary serialization [incubator-gluten]
acvictor opened a new pull request, #11628:
URL: https://github.com/apache/incubator-gluten/pull/11628
## What changes are proposed in this pull request?
This PR replaces three separate map look ups on the same key with a single `find()` and iterator reuse in `ArrowShuffleDictionaryWriter::serialize()`
## How was this patch tested?
Existing UTs
## Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Reduce triple map lookup to single find in dictionary serialization [incubator-gluten]
Posted by "marin-ma (via GitHub)" <gi...@apache.org>.
marin-ma commented on code in PR #11628:
URL: https://github.com/apache/incubator-gluten/pull/11628#discussion_r2873074150
##########
cpp/velox/shuffle/ArrowShuffleDictionaryWriter.cc:
##########
@@ -404,14 +404,14 @@ arrow::Status ArrowShuffleDictionaryWriter::serialize(arrow::io::OutputStream* o
ARROW_RETURN_NOT_OK(out->Write(bitMap.data(), bitMapSize));
for (auto fieldIdx : dictionaryFields_) {
+ auto it = dictionaries_.find(fieldIdx);
GLUTEN_DCHECK(
- dictionaries_.find(fieldIdx) != dictionaries_.end(),
+ it != dictionaries_.end(),
Review Comment:
@jinchengchenghh This change makes sense as `dictionaries_.erase(fieldIdx)` also performs another lookup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Reduce triple map lookup to single find in dictionary serialization [incubator-gluten]
Posted by "marin-ma (via GitHub)" <gi...@apache.org>.
marin-ma merged PR #11628:
URL: https://github.com/apache/incubator-gluten/pull/11628
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Reduce triple map lookup to single find in dictionary serialization [incubator-gluten]
Posted by "jinchengchenghh (via GitHub)" <gi...@apache.org>.
jinchengchenghh commented on code in PR #11628:
URL: https://github.com/apache/incubator-gluten/pull/11628#discussion_r2872830358
##########
cpp/velox/shuffle/ArrowShuffleDictionaryWriter.cc:
##########
@@ -404,14 +404,14 @@ arrow::Status ArrowShuffleDictionaryWriter::serialize(arrow::io::OutputStream* o
ARROW_RETURN_NOT_OK(out->Write(bitMap.data(), bitMapSize));
for (auto fieldIdx : dictionaryFields_) {
+ auto it = dictionaries_.find(fieldIdx);
GLUTEN_DCHECK(
- dictionaries_.find(fieldIdx) != dictionaries_.end(),
+ it != dictionaries_.end(),
Review Comment:
GLUTEN_DCHECK means the code only executes on debug compilation mode, so we don't need to add complexity for it. Only if the code logic has something wrong, the check mai fail.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org
Re: [PR] [VL] Reduce triple map lookup to single find in dictionary serialization [incubator-gluten]
Posted by "acvictor (via GitHub)" <gi...@apache.org>.
acvictor commented on PR #11628:
URL: https://github.com/apache/incubator-gluten/pull/11628#issuecomment-3928204079
@marin-ma can you please review this PR? Thanks in advance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@gluten.apache.org
For additional commands, e-mail: commits-help@gluten.apache.org