You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2019/11/07 04:21:41 UTC

[kudu-CR] wip KUDU-2929: don't do nothing when under memory pressure 

Andrew Wong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/14650


Change subject: wip KUDU-2929: don't do nothing when under memory pressure
......................................................................

wip KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure.

While nice in theory, I don't think this holds up in production. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops.

wip: I need to test this on a real cluster.

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
---
M src/kudu/util/maintenance_manager.cc
1 file changed, 2 insertions(+), 9 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/14650/1
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>

[kudu-CR] wip KUDU-2929: don't do nothing when under memory pressure

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: wip KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 1:

(1 comment)

Any way to unit test this? Maybe via some mocking in maintenance_manager-test.cc?

http://gerrit.cloudera.org:8080/#/c/14650/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14650/1//COMMIT_MSG@19
PS1, Line 19: let's continue walking through valid ops.
Might want to add here what you'd expect to happen (i.e. now we'll do compactions).



-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 07 Nov 2019 04:38:26 +0000
Gerrit-HasComments: Yes

[kudu-CR] wip KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14650

to look at the new patch set (#2).

Change subject: wip KUDU-2929: don't do nothing when under memory pressure
......................................................................

wip KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure.

While nice in theory, I don't think this holds up in production. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops (in order of what frees the
most WALs, then what frees the most on-disk data, then what has the best
perf improvement).

wip: I need to test this on a real cluster.

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
---
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
2 files changed, 27 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/14650/2
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14650

to look at the new patch set (#6).

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................

KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure if we don't know that op is anchoring
memory (obvious in the case of MRS or DMS flushes, less so in the case
of compactions).

While nice in theory, I don't think this behavior was desirable. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops (in order of what frees the
most WALs, then what frees the most on-disk data, then what has the best
perf improvement).

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
---
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
3 files changed, 72 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/14650/6
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] wip KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: wip KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 1:

(1 comment)

> Patch Set 1:
> 
> (1 comment)
> 
> Any way to unit test this? Maybe via some mocking in maintenance_manager-test.cc?

Sure, should be able to test that we schedule _something_ instead of nothing, even if there's no MRS or DMS.

http://gerrit.cloudera.org:8080/#/c/14650/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14650/1//COMMIT_MSG@19
PS1, Line 19: let's continue walking through valid ops.
> Might want to add here what you'd expect to happen (i.e. now we'll do compa
It won't necessarily be compaction. It'll be whatever has we would do next if we weren't under memory pressure.



-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 07 Nov 2019 04:56:30 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 6: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Sat, 09 Nov 2019 06:29:07 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14650

to look at the new patch set (#3).

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................

KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure if we don't know that op is anchoring
memory (obvious in the case of MRS or DMS flushes, less so in the case
of compactions).

While nice in theory, I don't think this behavior was desirable. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops (in order of what frees the
most WALs, then what frees the most on-disk data, then what has the best
perf improvement).

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
---
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
2 files changed, 27 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/14650/3
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14650

to look at the new patch set (#4).

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................

KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure if we don't know that op is anchoring
memory (obvious in the case of MRS or DMS flushes, less so in the case
of compactions).

While nice in theory, I don't think this behavior was desirable. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops (in order of what frees the
most WALs, then what frees the most on-disk data, then what has the best
perf improvement).

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
---
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
3 files changed, 69 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/14650/4
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 4:

> Patch Set 4:
> 
> Looks good but the test failures seem real.

Haven't been able to repro the failures, but maybe this'll do it.


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Sat, 09 Nov 2019 02:39:50 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 3:

Deployed this on a real cluster.

First, I set the memory pressure limit to 3% of a hard limit of 1GiB. Running a small write workload quickly ballooned the tablet's rowset tree height to 117, at which point, no ops would happen. Even after restarting the node, the tserver was effectively stuck.

After deploying this patch and restarting, the tablet quickly compacted itself down to a height of 1.


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 07 Nov 2019 07:49:12 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 3:

Still haven't tested it on a real cluster, but the code here probably won't change unless it breaks something.


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 07 Nov 2019 06:47:36 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................

KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure if we don't know that op is anchoring
memory (obvious in the case of MRS or DMS flushes, less so in the case
of compactions).

While nice in theory, I don't think this behavior was desirable. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops (in order of what frees the
most WALs, then what frees the most on-disk data, then what has the best
perf improvement).

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Reviewed-on: http://gerrit.cloudera.org:8080/14650
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Tested-by: Andrew Wong <aw...@cloudera.com>
---
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
3 files changed, 72 insertions(+), 10 deletions(-)

Approvals:
  Adar Dembo: Looks good to me, approved
  Andrew Wong: Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 7
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 6: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Sat, 09 Nov 2019 03:55:06 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 4:

Looks good but the test failures seem real.


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Fri, 08 Nov 2019 23:10:13 +0000
Gerrit-HasComments: No

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Kudu Jenkins, Adar Dembo, Todd Lipcon, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/14650

to look at the new patch set (#5).

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................

KUDU-2929: don't do nothing when under memory pressure

When we're under memory pressure, we may exit early from considering
ops if there are none that we know will free memory. This might be
because we don't want to spend more memory performing an op when we are
already under memory pressure if we don't know that op is anchoring
memory (obvious in the case of MRS or DMS flushes, less so in the case
of compactions).

While nice in theory, I don't think this behavior was desirable. Yes, we
want to prioritize ops that definitely free memory over those that might
not, but doing _something_ seems better than nothing.

So rather than returning early if there aren't ops that free memory,
let's continue walking through valid ops (in order of what frees the
most WALs, then what frees the most on-disk data, then what has the best
perf improvement).

Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
---
M src/kudu/tserver/tablet_server-test.cc
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
3 files changed, 71 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/50/14650/5
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 5
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has removed a vote on this change.

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Removed Verified-1 by Kudu Jenkins (120)
-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] wip KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: wip KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/14650/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/14650/1//COMMIT_MSG@19
PS1, Line 19: let's continue walking through valid ops 
> It won't necessarily be compaction. It'll be whatever has we would do next 
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Thu, 07 Nov 2019 06:31:21 +0000
Gerrit-HasComments: Yes

[kudu-CR] KUDU-2929: don't do nothing when under memory pressure

Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/14650 )

Change subject: KUDU-2929: don't do nothing when under memory pressure
......................................................................


Patch Set 6:

Both failures seem unrelated.


-- 
To view, visit http://gerrit.cloudera.org:8080/14650
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I030f9ef379af501fe7bd2f42906ec2f9ea16dbde
Gerrit-Change-Number: 14650
Gerrit-PatchSet: 6
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-Comment-Date: Sat, 09 Nov 2019 06:29:03 +0000
Gerrit-HasComments: No