You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Mike Percy (Code Review)" <ge...@cloudera.org> on 2017/03/03 21:53:15 UTC

[kudu-CR] env: Add support for getting FS capacity 

Hello Adar Dembo,

I'd like you to do a code review.  Please visit

    http://gerrit.cloudera.org:8080/6255

to review the following change.

Change subject: env: Add support for getting FS capacity
......................................................................

env: Add support for getting FS capacity

We need a way to get FS capacity in a follow-up patch. This change adds
that capability and changes the Env API to allow for fetching both
capacity and free bytes with a single call. This also allows us to fetch
both values with a single syscall. This API is inspired by
boost::filesystem::space().

This patch also removes the difference in the "free" space returned when
this API is invoked as a superuser vs. a non-privileged user. Now, only
the space available to non-privileged processes is returned. This is
also consistent with the boost API and makes the API more predictable.

Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
---
M src/kudu/util/env-test.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M src/kudu/util/env_util.cc
4 files changed, 43 insertions(+), 30 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/55/6255/1
-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>

[kudu-CR] env: Add support for getting FS capacity

Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has submitted this change and it was merged.

Change subject: env: Add support for getting FS capacity
......................................................................


env: Add support for getting FS capacity

We need a way to get FS capacity in a follow-up patch. This change adds
that capability and changes the Env API to allow for fetching both
capacity and free bytes with a single call. This also allows us to fetch
both values with a single syscall. This API is inspired by
boost::filesystem::space().

This patch also removes the difference in the "free" space returned when
this API is invoked as a superuser vs. a non-privileged user. Now, only
the space available to non-privileged processes is returned. This is
also consistent with the boost API and makes the API more predictable.

Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Reviewed-on: http://gerrit.cloudera.org:8080/6255
Tested-by: Kudu Jenkins
Reviewed-by: Todd Lipcon <to...@apache.org>
Reviewed-by: David Ribeiro Alves <dr...@apache.org>
---
M src/kudu/util/env-test.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M src/kudu/util/env_util.cc
4 files changed, 49 insertions(+), 32 deletions(-)

Approvals:
  David Ribeiro Alves: Looks good to me, approved
  Todd Lipcon: Looks good to me, but someone else must approve
  Kudu Jenkins: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>

[kudu-CR] env: Add support for getting FS capacity

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env-test.cc
File src/kudu/util/env-test.cc:

Line 814:   NO_FATALS(AssertEventually([&] {
> Generally it does need it, and it should have it. In this case, it's option
isn't this a bug in AssertEventually then? assert is clearly expected to fail immediately while expect is expected to continue despite the failure.


-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] env: Add support for getting FS capacity

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env-test.cc
File src/kudu/util/env-test.cc:

Line 814:   NO_FATALS(AssertEventually([&] {
> isn't this a bug in AssertEventually then? assert is clearly expected to fa
ASSERT only fails immediately if it's in the test method itself. Since AssertEventually is itself a function call, it can't 'return' from the test method.

I suppose we could make it a macro that does NO_FATALS(AssertEventuallyInternal(...)) to accomplish this if you think that's better?


-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] env: Add support for getting FS capacity

Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 1: Verified+1

Overriding jenkins flakiness

-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-HasComments: No

[kudu-CR] env: Add support for getting FS capacity

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env-test.cc
File src/kudu/util/env-test.cc:

Line 806: TEST_F(TestEnv, TestGetBytesFree) {
Nit: maybe rename the test too?


Line 814:   NO_FATALS(AssertEventually([&] {
I don't think AssertEventually() needs to be wrapped in NO_FATALS(). I've never seen that done before.


http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env_posix.cc
File src/kudu/util/env_posix.cc:

Line 1023:   virtual Status GetSpaceInfo(const std::string& path, SpaceInfo* space_info) OVERRIDE {
Nit: don't need std:: prefix.


-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] env: Add support for getting FS capacity

Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 2:

Yeah I was thinking about adding ASSERT_EVENTUALLY as a macro, would be an improvement. Anyway, as you say, not directly related to this patch. Thanks for the review!

-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No

[kudu-CR] env: Add support for getting FS capacity

Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 2: Code-Review+1

-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No

[kudu-CR] env: Add support for getting FS capacity

Posted by "David Ribeiro Alves (Code Review)" <ge...@cloudera.org>.
David Ribeiro Alves has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 2: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env-test.cc
File src/kudu/util/env-test.cc:

Line 814:   // filesystem.
> ASSERT only fails immediately if it's in the test method itself. Since Asse
I think we should so that, yes, since it kinda breaks what we'd expect from an Assert method. Don't think we need to predicate this patch on that though.


-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: David Ribeiro Alves <dr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes

[kudu-CR] env: Add support for getting FS capacity

Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has uploaded a new patch set (#2).

Change subject: env: Add support for getting FS capacity
......................................................................

env: Add support for getting FS capacity

We need a way to get FS capacity in a follow-up patch. This change adds
that capability and changes the Env API to allow for fetching both
capacity and free bytes with a single call. This also allows us to fetch
both values with a single syscall. This API is inspired by
boost::filesystem::space().

This patch also removes the difference in the "free" space returned when
this API is invoked as a superuser vs. a non-privileged user. Now, only
the space available to non-privileged processes is returned. This is
also consistent with the boost API and makes the API more predictable.

Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
---
M src/kudu/util/env-test.cc
M src/kudu/util/env.h
M src/kudu/util/env_posix.cc
M src/kudu/util/env_util.cc
4 files changed, 49 insertions(+), 32 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/55/6255/2
-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>

[kudu-CR] env: Add support for getting FS capacity

Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change.

Change subject: env: Add support for getting FS capacity
......................................................................


Patch Set 1:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env-test.cc
File src/kudu/util/env-test.cc:

Line 806: TEST_F(TestEnv, TestGetBytesFree) {
> Nit: maybe rename the test too?
Done


Line 814:   NO_FATALS(AssertEventually([&] {
> I don't think AssertEventually() needs to be wrapped in NO_FATALS(). I've n
Generally it does need it, and it should have it. In this case, it's optional because it's the last statement in the test, but I think it's good practice to use NO_FATALS wherever it's appropriate. If I didn't wrap this in NO_FATALS then if there was a statement following it, even if AssertEventually failed the test would continue executing, usually resulting in a confusing failure message when the test finally completes.


http://gerrit.cloudera.org:8080/#/c/6255/1/src/kudu/util/env_posix.cc
File src/kudu/util/env_posix.cc:

Line 1023:   virtual Status GetSpaceInfo(const std::string& path, SpaceInfo* space_info) OVERRIDE {
> Nit: don't need std:: prefix.
Done


-- 
To view, visit http://gerrit.cloudera.org:8080/6255
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id43275876d3352f5cf943e24ed4513b9f2c131aa
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-HasComments: Yes