You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Joe McDonnell (Code Review)" <ge...@cloudera.org> on 2024/10/30 00:13:49 UTC

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Joe McDonnell has uploaded this change for review. ( http://gerrit.cloudera.org:8080/21991


Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................

IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

This adds counters for the number of bytes written / read
from the tuple cache. This gives visibility into whether
certain locations have enormous result sizes. This will be
used to tune the placement of tuple cache nodes.

Tests:
 - Added checks of the TupleCacheBytesWritten/Read counters
   to existing tests in test_tuple_cache.py

Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
---
M be/src/exec/tuple-file-reader.cc
M be/src/exec/tuple-file-reader.h
M be/src/exec/tuple-file-writer.cc
M be/src/exec/tuple-file-writer.h
M tests/custom_cluster/test_tuple_cache.py
5 files changed, 36 insertions(+), 4 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/91/21991/1
-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Yida Wu (Code Review)" <ge...@cloudera.org>.
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................


Patch Set 1: Code-Review+1

LGTM


-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 30 Oct 2024 16:13:55 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................


Patch Set 1: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 30 Oct 2024 21:39:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/17302/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Wed, 30 Oct 2024 00:38:08 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11084/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 30 Oct 2024 16:19:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/11087/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 30 Oct 2024 18:25:26 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................

IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

This adds counters for the number of bytes written / read
from the tuple cache. This gives visibility into whether
certain locations have enormous result sizes. This will be
used to tune the placement of tuple cache nodes.

Tests:
 - Added checks of the TupleCacheBytesWritten/Read counters
   to existing tests in test_tuple_cache.py

Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Reviewed-on: http://gerrit.cloudera.org:8080/21991
Reviewed-by: Yida Wu <wy...@gmail.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/tuple-file-reader.cc
M be/src/exec/tuple-file-reader.h
M be/src/exec/tuple-file-writer.cc
M be/src/exec/tuple-file-writer.h
M tests/custom_cluster/test_tuple_cache.py
5 files changed, 36 insertions(+), 4 deletions(-)

Approvals:
  Yida Wu: Looks good to me, but someone else must approve
  Michael Smith: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 2
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>

[Impala-ASF-CR] IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/21991 )

Change subject: IMPALA-13497: Add TupleCacheBytesWritten/Read to the profile
......................................................................


Patch Set 1: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/21991
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ib5c9249049d8d46116a65929896832d02c2d9f1f
Gerrit-Change-Number: 21991
Gerrit-PatchSet: 1
Gerrit-Owner: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 30 Oct 2024 18:25:06 +0000
Gerrit-HasComments: No