You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org> on 2017/12/06 21:30:14 UTC

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Bikramjeet Vig has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8781


Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................

IMPALA-6222: Add details to error msg on failure to get min reservation

This patch adds the following details to the error message encountered
on failure to get minimum memory reservation:
- which ReservationTracker or MemTracker hit its limit
- top 5 admitted queries that are consuming the most memory under the
tracker that hit its limit

Testing:
- added tests to reservation-tracker-test.cc that verify the error
message returned for different cases.
- tested "initial reservation failed" condition manually to verify
the error message returned (Request suggestion for an automated test).

Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
---
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-tracker.cc
M be/src/runtime/bufferpool/reservation-tracker.h
M be/src/runtime/initial-reservations.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M common/thrift/generate_error_codes.py
7 files changed, 169 insertions(+), 38 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/8781/1
-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 1
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/8781

to look at the new patch set (#2).

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................

IMPALA-6222: Add details to error msg on failure to get min reservation

This patch adds the following details to the error message encountered
on failure to get minimum memory reservation:
- which ReservationTracker or MemTracker hit its limit
- top 5 admitted queries that are consuming the most memory under the
tracker that hit its limit

Testing:
- added tests to reservation-tracker-test.cc that verify the error
message returned for different cases.
- tested "initial reservation failed" condition manually to verify
the error message returned (Request suggestion for an automated test).

Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
---
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-tracker.cc
M be/src/runtime/bufferpool/reservation-tracker.h
M be/src/runtime/initial-reservations.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M common/thrift/generate_error_codes.py
M tests/custom_cluster/test_mem_reservations.py
8 files changed, 187 insertions(+), 40 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/8781/2
-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 1:

(11 comments)

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker-test.cc
File be/src/runtime/bufferpool/reservation-tracker-test.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker-test.cc@44
PS1, Line 44: NULL
> nullptr?
Done


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.h
File be/src/runtime/bufferpool/reservation-tracker.h:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.h@121
PS1, Line 121:   /// 'error_status'.
> Maybe add something like (if 'error_status' is non-NULL) just to clarify th
Done


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc
File be/src/runtime/bufferpool/reservation-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@168
PS1, Line 168: memtracker
> nit: mem_tracker for consistency
Done


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@174
PS1, Line 174:       *error_status = Status::Expected(Substitute(
> Seems like we should probably add this one to generate_error_codes.py (we d
I was thinking of adding it to generate_error_codes.py but that would mean that I wont be able to use Status::expected here to avoid printing the stacktrace.

Also, i will be breaking this error message as described in the following comment.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@177
PS1, Line 177: he top 5 queries "
             :           "that allocated memory under this tracker are:\n$6",
would it make more sense to have the top 5 queries based on reservation rather than memory consumption, I can add a flag to LogTopNQueries() to sort by amount of reservation instead. Or will this extension of LogTopNQueries() be superfluous?


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@185
PS1, Line 185:  memtracker->LogTopNQueries(5
> This is kind of assuming that only query mem trackers and above have reserv
I see what you mean, I think error message would make more sense if I only append the "The top 5 queries..." part only if memtracker->LogTopNQueries(5) returns a non empty string (since it only returns something if it is executed for query mem trackers and above)


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@185
PS1, Line 185:           memtracker->LogTopNQueries(5)));
> I feel like we should probably document the lock order implied here since w
Done


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@199
PS1, Line 199:             Substitute("Failed to increase reservation by $0 because it would exceed the "
> Also move to generate_error_codes.py? Or maybe we should call MemTracker::M
We should probably stick to using this over MemLimitExceeded as this one is more descriptive.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@203
PS1, Line 203:                 mem_tracker_->LogUsage(0), mem_tracker_->LogTopNQueries(5)));
> We don't actually know that the limit hit was on mem_tracker_ itself, unfor
TryConsumeFromMemTracker only returns false if the memtracker has no parent and hence no ancestors. So if the consume call fails, we can be sure that it is the same memtracker


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/mem-tracker.cc
File be/src/runtime/mem-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/mem-tracker.cc@336
PS1, Line 336: void MemTracker::GetTopNQueries(std::priority_queue<MemTracker*, vector<MemTracker*>,
> Returning the pointers to the MemTrackers isn't safe - they're only guarant
Done


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/mem-tracker.cc@338
PS1, Line 338:   {
> nit: curly braces don't seem necessary since the lock is held for the whole
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 1
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Sat, 09 Dec 2017 02:31:00 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................

IMPALA-6222: Add details to error msg on failure to get min reservation

This patch adds the following details to the error message encountered
on failure to get minimum memory reservation:
- which ReservationTracker hit its limit
- top 5 admitted queries that are consuming the most memory under the
ReservationTracker that hit its limit

Testing:
- added tests to reservation-tracker-test.cc that verify the error
message returned for different cases.
- tested "initial reservation failed" condition manually to verify
the error message returned.

Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Reviewed-on: http://gerrit.cloudera.org:8080/8781
Reviewed-by: Bikramjeet Vig <bi...@cloudera.com>
Tested-by: Impala Public Jenkins
---
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-tracker.cc
M be/src/runtime/bufferpool/reservation-tracker.h
M be/src/runtime/initial-reservations.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M common/thrift/generate_error_codes.py
7 files changed, 176 insertions(+), 39 deletions(-)

Approvals:
  Bikramjeet Vig: Looks good to me, approved
  Impala Public Jenkins: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 5
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc
File be/src/runtime/bufferpool/reservation-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@203
PS1, Line 203:                 mem_tracker_->LogUsage(0));
> That's not quite right. E.g. if this is the query reservation tracker, TryC
Calling MemLimitExceeded here like you suggested earlier.



-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 2
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Tue, 12 Dec 2017 22:51:34 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/8781

to look at the new patch set (#3).

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................

IMPALA-6222: Add details to error msg on failure to get min reservation

This patch adds the following details to the error message encountered
on failure to get minimum memory reservation:
- which ReservationTracker hit its limit
- top 5 admitted queries that are consuming the most memory under the
ReservationTracker that hit its limit

Testing:
- added tests to reservation-tracker-test.cc that verify the error
message returned for different cases.
- tested "initial reservation failed" condition manually to verify
the error message returned.

Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
---
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-tracker.cc
M be/src/runtime/bufferpool/reservation-tracker.h
M be/src/runtime/initial-reservations.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M common/thrift/generate_error_codes.py
7 files changed, 174 insertions(+), 39 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/8781/3
-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 1:

(10 comments)

I like the overall approach and the tests. Comments are mainly about the invariants we can assume about where limits are placed in the hierarchy.

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker-test.cc
File be/src/runtime/bufferpool/reservation-tracker-test.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker-test.cc@44
PS1, Line 44: NULL
nullptr?


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.h
File be/src/runtime/bufferpool/reservation-tracker.h:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.h@121
PS1, Line 121:   /// 'error_status'.
Maybe add something like (if 'error_status' is non-NULL) just to clarify that it's valid to pass in NULL?


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc
File be/src/runtime/bufferpool/reservation-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@168
PS1, Line 168: memtracker
nit: mem_tracker for consistency


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@174
PS1, Line 174:       *error_status = Status::Expected(Substitute(
Seems like we should probably add this one to generate_error_codes.py (we don't have terribly consistent criteria for doing so but it's usually a good practice).


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@185
PS1, Line 185:  memtracker->LogTopNQueries(5
This is kind of assuming that only query mem trackers and above have reservation limits - this would produce a weird result if we called it on a lower one.

The assumption is true for now but I'm wary of adding an undocumented assumption here since we might want to add constraints to other plan nodes, e.g. for partial sort. I guess we could document on this function that the error message only makes sense if you call this on a query-level reservation tracker (or above).


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@185
PS1, Line 185:           memtracker->LogTopNQueries(5)));
I feel like we should probably document the lock order implied here since we're calling a function that acquires MemTracker::child_trackers_lock_ while holding ReservationTracker::lock_. Maybe document in the ReservationTracker::lock_ comment?


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@199
PS1, Line 199:             Substitute("Failed to increase reservation by $0 because it would exceed the "
Also move to generate_error_codes.py? Or maybe we should call MemTracker::MemLimitExceeded() here instead for consistency?


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@203
PS1, Line 203:                 mem_tracker_->LogUsage(0), mem_tracker_->LogTopNQueries(5)));
We don't actually know that the limit hit was on mem_tracker_ itself, unfortunately, it could be further up. I think calling MemTracker::MemLimitExceeded() might solve this issue though.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/mem-tracker.cc
File be/src/runtime/mem-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/mem-tracker.cc@336
PS1, Line 336: void MemTracker::GetTopNQueries(std::priority_queue<MemTracker*, vector<MemTracker*>,
Returning the pointers to the MemTrackers isn't safe - they're only guaranteed to live as long as 'child_trackers_lock_' is held. I think we need to copy out any state from the child trackers that we need before dropping the lock.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/mem-tracker.cc@338
PS1, Line 338:   {
nit: curly braces don't seem necessary since the lock is held for the whole function.



-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 1
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 06 Dec 2017 23:45:15 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 4
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Dec 2017 22:34:28 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 4: Code-Review+2

carrying over Tim's +2


-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 4
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Dec 2017 18:52:25 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 1:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc
File be/src/runtime/bufferpool/reservation-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@174
PS1, Line 174:       *error_status = Status::Expected(Substitute(
> I was thinking of adding it to generate_error_codes.py but that would mean 
This seems ok then.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@177
PS1, Line 177: he top 5 queries "
             :           "that allocated memory under this tracker are:\n$6",
> would it make more sense to have the top 5 queries based on reservation rat
I think just sorting them by consumption is good enough. We should be moving more things to reservation anyway so the difference will get smaller over time.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@185
PS1, Line 185:  memtracker->LogTopNQueries(5
> I see what you mean, I think error message would make more sense if I only 
Yeah that makes sense to me.


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@199
PS1, Line 199:             Substitute("Failed to increase reservation by $0 because it would exceed the "
> We should probably stick to using this over MemLimitExceeded as this one is
WFM


http://gerrit.cloudera.org:8080/#/c/8781/1/be/src/runtime/bufferpool/reservation-tracker.cc@203
PS1, Line 203:                 mem_tracker_->LogUsage(0), mem_tracker_->LogTopNQueries(5)));
> TryConsumeFromMemTracker only returns false if the memtracker has no parent
That's not quite right. E.g. if this is the query reservation tracker, TryConsume() on the query mem tracker will check both the query and process limits.

The subtlety is that GetParentMemTracker() returns NULL if this reservation tracker's parent is not linked to a mem tracker. That doesn't actually imply that the linked mem tracker does not have a parent. I.e. parent_->mem_tracker_ can be null even if mem_tracker_->parent_ is non-NULL. Maybe the name of the function could be improved...



-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 1
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Mon, 11 Dec 2017 22:18:57 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Hello Tim Armstrong, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/8781

to look at the new patch set (#4).

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................

IMPALA-6222: Add details to error msg on failure to get min reservation

This patch adds the following details to the error message encountered
on failure to get minimum memory reservation:
- which ReservationTracker hit its limit
- top 5 admitted queries that are consuming the most memory under the
ReservationTracker that hit its limit

Testing:
- added tests to reservation-tracker-test.cc that verify the error
message returned for different cases.
- tested "initial reservation failed" condition manually to verify
the error message returned.

Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
---
M be/src/runtime/bufferpool/reservation-tracker-test.cc
M be/src/runtime/bufferpool/reservation-tracker.cc
M be/src/runtime/bufferpool/reservation-tracker.h
M be/src/runtime/initial-reservations.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/mem-tracker.h
M common/thrift/generate_error_codes.py
7 files changed, 176 insertions(+), 39 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/81/8781/4
-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 4
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/1619/


-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 4
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Dec 2017 18:59:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Tim Armstrong (Code Review)" <ge...@cloudera.org>.
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 3: Code-Review+2

(2 comments)

http://gerrit.cloudera.org:8080/#/c/8781/3/be/src/runtime/mem-tracker.cc
File be/src/runtime/mem-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/3/be/src/runtime/mem-tracker.cc@324
PS3, Line 324:   std::priority_queue<pair<int64_t, string>, vector<pair<int64_t, string>>,
nit: add a using up the top of the file or in common/names.h instead of using the std:: prefixes.


http://gerrit.cloudera.org:8080/#/c/8781/3/be/src/runtime/mem-tracker.cc@340
PS3, Line 340:     int limit) {
nit: weird wrapping.



-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Wed, 13 Dec 2017 22:51:05 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-6222: Add details to error msg on failure to get min reservation

Posted by "Bikramjeet Vig (Code Review)" <ge...@cloudera.org>.
Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/8781 )

Change subject: IMPALA-6222: Add details to error msg on failure to get min reservation
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/8781/3/be/src/runtime/mem-tracker.cc
File be/src/runtime/mem-tracker.cc:

http://gerrit.cloudera.org:8080/#/c/8781/3/be/src/runtime/mem-tracker.cc@324
PS3, Line 324:   std::priority_queue<pair<int64_t, string>, vector<pair<int64_t, string>>,
> nit: add a using up the top of the file or in common/names.h instead of usi
Done


http://gerrit.cloudera.org:8080/#/c/8781/3/be/src/runtime/mem-tracker.cc@340
PS3, Line 340:     int limit) {
> nit: weird wrapping.
4 extra spaces before std::greater as it is a part of the template parameter list of std::priority_queue

"int limit" at the same level as  "std::priority_queue" as both are a part of the function parameter list



-- 
To view, visit http://gerrit.cloudera.org:8080/8781
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic4675fe923b33fdc4ddefd1872e6d6b803993d74
Gerrit-Change-Number: 8781
Gerrit-PatchSet: 3
Gerrit-Owner: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Bikramjeet Vig <bi...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <ta...@cloudera.com>
Gerrit-Comment-Date: Thu, 14 Dec 2017 18:48:40 +0000
Gerrit-HasComments: Yes