[Impala-ASF-CR] IMPALA-11539: Mitigate intra-node skew of file scans with MT DOP

["Joe McDonnell (Code Review)" <ge...@cloudera.org>]

Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/18929 )

Change subject: IMPALA-11539: Mitigate intra-node skew of file scans with MT_DOP
......................................................................


Patch Set 5: Code-Review+1

(1 comment)

This makes sense to me.

I don't think this would have an impact on data caching except in pretty extreme cases. It's possible that we could get smarter with this logic in future, but this seems like an improvement on its own.

We are moving from O(1) to O(log(N)), but that doesn't seem like a big problem for this data structure. Other costs should dwarf the cost of accessing this structure.

http://gerrit.cloudera.org:8080/#/c/18929/5/be/src/exec/scan-range-queue-mt.h
File be/src/exec/scan-range-queue-mt.h:

http://gerrit.cloudera.org:8080/#/c/18929/5/be/src/exec/scan-range-queue-mt.h@37
PS5, Line 37: class ScanRangeQueueMt {
Nit: Maybe add DISALLOW_COPY_AND_ASSIGN



-- 
To view, visit http://gerrit.cloudera.org:8080/18929
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib7dc1f1665565da6c0e155c1e585f7089b18a180
Gerrit-Change-Number: 18929
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 01 Sep 2022 19:28:12 +0000
Gerrit-HasComments: Yes