You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Dinesh Bhat (Code Review)" <ge...@cloudera.org> on 2016/10/03 07:33:21 UTC

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Hello Dan Burkert, Adar Dembo,

I'd like you to do a code review.  Please visit

    http://gerrit.cloudera.org:8080/4594

to review the following change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................

KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

ReadFdsFully() captures either stdout or stderr or both depending on how
caller has registered. Once the helpers are done with capturing, it needs
take into account that there can be less than 2 fds registered from the
caller and hence can not do std::move(vector[1]) operation on it.

Also added a test code to exercise combination of:
SubProcess::Call(argv[0], nullptr, &stderr);
SubProcess::Call(argv[0], &stdout, nullptr);

Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
---
M src/kudu/util/subprocess-test.cc
M src/kudu/util/subprocess.cc
2 files changed, 33 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/4594/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Dinesh Bhat has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 3:

(5 comments)

TFTR Alexey,

http://gerrit.cloudera.org:8080/#/c/4594/3/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

Line 27: #include "kudu/gutil/strings/numbers.h"
> Is this necessary for the updated version?
Done


PS3, Line 138: string
> Nit: consider 'const string str = ...' or 'static const string str = ...' j
Done


PS3, Line 140:   Status s = Subprocess::Call({"/bin/sh", "-c", cmd_str}, nullptr, &stderr);
             :   ASSERT_TRUE(s.ok());
> Nit: since 's' is used only for ASSERT_TRUE(s.ok()) check, consider replaci
Done


PS3, Line 146:   s = Subprocess::Call({"/bin/ls", "/dev/null"}, &stdout, nullptr);
             :   ASSERT_TRUE(s.ok());
> Ditto for merging these two strings into ASSERT_OK(...)
Done


PS3, Line 150:   s = Subprocess::Call({"/bin/ls", "/dev/zero"}, nullptr, nullptr);
             :   ASSERT_TRUE(s.ok());
> Ditto for merging these two strings into ASSERT_OK(...)
Done


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Dinesh Bhat has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
> I agree with Alexey (and I wrote the same on PS1): generating a random stri
Thanks Adar, agreed on rand() part. I think Alexey & I were discussing a slightly different topic earlier; i.e whether or not we want to be checking stderr content at all. My argument was we want to because we may have collected a garbage in stderr if the program didn't SIGSEGV. I also didn't want to rely on an error code which came out as EACCESS via SubProcess::Call("/").  After some offline discussions, we updated the stderr to be more deterministic. In this case we know certainly:
1) Program has executed via execve(unlike EACCESS case)
2) It has also captured a well-defined string in stderr


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Dinesh Bhat has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
> I don't think it's worth analyzing the stderr in the context of this test: 
We are better off checking little more than no SIGSEGV here, to make sure we don't have garbage stashed in stderr. Previously I had run 50k iterations of original patch without hitting anything, but one of the jenkins run had caught sigsegv.

I am sure you would get a different error if you are running on a VM with root privileges for "/" command. In any case, I can include a simpler version than 'bash -c' if you feel this is too fancy, but checking for sanity output of stderr is important IMHO.


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

ReadFdsFully() captures either stdout or stderr or both depending on how
caller has registered. Once the helpers are done with capturing, it needs
take into account that there can be less than 2 fds registered from the
caller and hence can not do std::move(vector[1]) operation on it.

Also added a test code to exercise combination of:
SubProcess::Call({argv}, nullptr, &stderr);
SubProcess::Call({argv}, &stdout, nullptr);
SubProcess::Call({argv}, nullptr, nullptr);

Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Reviewed-on: http://gerrit.cloudera.org:8080/4594
Tested-by: Kudu Jenkins
Reviewed-by: Alexey Serbin <as...@cloudera.com>
---
M src/kudu/util/subprocess-test.cc
M src/kudu/util/subprocess.cc
2 files changed, 28 insertions(+), 4 deletions(-)

Approvals:
  Alexey Serbin: Looks good to me, approved
  Kudu Jenkins: Verified



-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 5
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/4594/1/src/kudu/util/subprocess.cc
File src/kudu/util/subprocess.cc:

Line 504:     *stdout_out = std::move(outv.front());
> Isn't that redundant if we are already checking for outv.size() as per your
Yep, exactly.  I wrote this comment before commenting on outv.capacity().  So, once there is a check on 'outv' size in accordance with 'fds' contents, that's not needed.


Line 507:     *stderr_out = std::move(outv.back());
> same as above.
Yep, the same response as above.


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Dinesh Bhat has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
> Why do you need such an intricate stuff?  If you want something which is gu
It's true that there are simpler alternatives; I wanted to pick a generic command valid across all unix platforms irrespective of modes. For eg, above command outputs 'permission denied' on mac with user privileges and 'command not found' if I am running it with sudo privileges. I think if we are simply checking stderr.empty() it should be okay. I can change it to that if you are advocating KISS.


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 4: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: No

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
Why do you need such an intricate stuff?  If you want something which is guaranteed to be rejected for execution, you could just do

Subprocess::Call({ "/" }, nullptr, &stderr);


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Dinesh Bhat has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 1:

(9 comments)

TFTR Adar/Alexey, updated the patch, please see responses inline.

http://gerrit.cloudera.org:8080/#/c/4594/1/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

Line 22: #include <cstdlib>
> What's this one for?
needed for std::rand below.


Line 29: #include "kudu/gutil/strings/substitute.h"
> Not used?
yeah, I was initially using strings::Substitute and re-added now while taking care of one of rev comments below.


PS1, Line 133: <random>
> Why randomize the non-existent command? You can just hardcode some nonsensi
That works too, random number prefixed with a nonsensical string could make it more stronger case for stderr I would think. Lemme know. Also I stuck to simple rand() instead of more accurate C++11 versions of rand().  Updated accordingly.


PS1, Line 136: TestKUDU1674
> You've already got the bug in the comment just above, so use the test name 
Done


Line 141:   ASSERT_FALSE(stderr.empty());
> Nit: since you check for the presence on non-empty substring in stderr, thi
Done


Line 148:   ASSERT_FALSE(stdout.empty());
> ditto.
Done


http://gerrit.cloudera.org:8080/#/c/4594/1/src/kudu/util/subprocess.cc
File src/kudu/util/subprocess.cc:

PS1, Line 502: capacity
> Why capacity() instead of size()?  Capacity is controlled by the implementa
Sounds good, fixed the code as well as comment above.


Line 504:     *stdout_out = std::move(outv.front());
> Do you think it's worth to check that !outv.empty() before calling outv.fro
Isn't that redundant if we are already checking for outv.size() as per your comment above ? More useful check could have been to recognize that outv.front had captured stdout(and not stderr), but we don't have visibility into that in this line.


Line 507:     *stderr_out = std::move(outv.back());
> Ditto here: do you think it's worth to check that !outv.empty() before call
same as above.


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
> It's true that there are simpler alternatives; I wanted to pick a generic c
I don't think it's worth analyzing the stderr in the context of this test: other (e.g., next one) version of bash could output something else instead of 'command not found'.  Checking !s.ok() would be enough: all we want is to check is that Subprocess::Call() does not die with SIGSEGV, right?

But it's all up to you.  I just wanted to point it's possible to check that the Subprocess::Call() does not SIGSEGV by a simpler means.

BTW, in my case, running Subprocess::Call("/", ...) with and without super-user privileges output the same 'Couldn't exec /: Permission denied [13]' and I wouldn't expect it to be different depending on access level.


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 1:

(4 comments)

Good find.

http://gerrit.cloudera.org:8080/#/c/4594/1/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

Line 22: #include <cstdlib>
What's this one for?


Line 29: #include "kudu/gutil/strings/substitute.h"
Not used?


PS1, Line 133: <random>
Why randomize the non-existent command? You can just hardcode some nonsensical string instead, and save some function calls/includes.


PS1, Line 136: TestKUDU1674
You've already got the bug in the comment just above, so use the test name to describe the bug itself, or what the test is covering.


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
> We are better off checking little more than no SIGSEGV here, to make sure w
I doubt it would be different error for running "/" as a  command.  Do you have an example of setup where different errors are emitted if running with and without super-user access?  Can I see it?


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 3:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/4594/3/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

Line 27: #include "kudu/gutil/strings/numbers.h"
Is this necessary for the updated version?


PS3, Line 138: string
Nit: consider 'const string str = ...' or 'static const string str = ...' just to express it's not about to change during the test.


PS3, Line 140:   Status s = Subprocess::Call({"/bin/sh", "-c", cmd_str}, nullptr, &stderr);
             :   ASSERT_TRUE(s.ok());
Nit: since 's' is used only for ASSERT_TRUE(s.ok()) check, consider replacing there two strings with one like

ASSERT_OK(Subprocess::Call(...));


PS3, Line 146:   s = Subprocess::Call({"/bin/ls", "/dev/null"}, &stdout, nullptr);
             :   ASSERT_TRUE(s.ok());
Ditto for merging these two strings into ASSERT_OK(...)


PS3, Line 150:   s = Subprocess::Call({"/bin/ls", "/dev/zero"}, nullptr, nullptr);
             :   ASSERT_TRUE(s.ok());
Ditto for merging these two strings into ASSERT_OK(...)


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 1:

(5 comments)

http://gerrit.cloudera.org:8080/#/c/4594/1/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

Line 141:   ASSERT_FALSE(stderr.empty());
Nit: since you check for the presence on non-empty substring in stderr, this check could be omitted.


Line 148:   ASSERT_FALSE(stdout.empty());
ditto.


http://gerrit.cloudera.org:8080/#/c/4594/1/src/kudu/util/subprocess.cc
File src/kudu/util/subprocess.cc:

PS1, Line 502: capacity
Why capacity() instead of size()?  Capacity is controlled by the implementation details of the vector, and it might become 4 even if the vector is empty.

Also, why is it necessary to check it here?  Does it make sense to be more specific and check that the result vector contains appropriate number of elements in accordance with the size of the fds passed in?


Line 504:     *stdout_out = std::move(outv.front());
Do you think it's worth to check that !outv.empty() before calling outv.front()?


Line 507:     *stderr_out = std::move(outv.back());
Ditto here: do you think it's worth to check that !outv.empty() before calling outv.back()?


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Hello Adar Dembo, Alexey Serbin, Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/4594

to look at the new patch set (#3).

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................

KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

ReadFdsFully() captures either stdout or stderr or both depending on how
caller has registered. Once the helpers are done with capturing, it needs
take into account that there can be less than 2 fds registered from the
caller and hence can not do std::move(vector[1]) operation on it.

Also added a test code to exercise combination of:
SubProcess::Call(argv[0], nullptr, &stderr);
SubProcess::Call(argv[0], &stdout, nullptr);
SubProcess::Call(argv[0], nullptr, nullptr);

Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
---
M src/kudu/util/subprocess-test.cc
M src/kudu/util/subprocess.cc
2 files changed, 32 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/4594/3
-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Hello Adar Dembo, Alexey Serbin, Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/4594

to look at the new patch set (#4).

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................

KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

ReadFdsFully() captures either stdout or stderr or both depending on how
caller has registered. Once the helpers are done with capturing, it needs
take into account that there can be less than 2 fds registered from the
caller and hence can not do std::move(vector[1]) operation on it.

Also added a test code to exercise combination of:
SubProcess::Call({argv}, nullptr, &stderr);
SubProcess::Call({argv}, &stdout, nullptr);
SubProcess::Call({argv}, nullptr, nullptr);

Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
---
M src/kudu/util/subprocess-test.cc
M src/kudu/util/subprocess.cc
2 files changed, 28 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/4594/4
-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 4
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Dinesh Bhat (Code Review)" <ge...@cloudera.org>.
Hello Adar Dembo, Alexey Serbin, Kudu Jenkins,

I'd like you to reexamine a change.  Please visit

    http://gerrit.cloudera.org:8080/4594

to look at the new patch set (#2).

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................

KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

ReadFdsFully() captures either stdout or stderr or both depending on how
caller has registered. Once the helpers are done with capturing, it needs
take into account that there can be less than 2 fds registered from the
caller and hence can not do std::move(vector[1]) operation on it.

Also added a test code to exercise combination of:
SubProcess::Call(argv[0], nullptr, &stderr);
SubProcess::Call(argv[0], &stdout, nullptr);
SubProcess::Call(argv[0], nullptr, nullptr);

Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
---
M src/kudu/util/subprocess-test.cc
M src/kudu/util/subprocess.cc
2 files changed, 33 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/94/4594/2
-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins

[kudu-CR] KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone

Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.

Change subject: KUDU-1674: Fix SubProcess:Call SEGV when trying to capture stderr alone
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4594/2/src/kudu/util/subprocess-test.cc
File src/kudu/util/subprocess-test.cc:

PS2, Line 139: SimpleItoa(std::rand())
> I doubt it would be different error for running "/" as a  command.  Do you 
I agree with Alexey (and I wrote the same on PS1): generating a random string so as to "guarantee output in stderr" is overkill. Why would it guarantee anything stronger or weaker than a hardcoded string?

Any version of bash will yield at least "command not found" if  you pass in a hardcoded string that you know isn't going to be a command. Something like "asdfghjkl". Or "suohvkxuhvoqie". Or whatever you want. I guarantee you it will be robust enough. If not, I'll eat my words and fix it myself. :)


-- 
To view, visit http://gerrit.cloudera.org:8080/4594
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I67bd462098526a9ba032669b621b8139b56ee5b8
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@cloudera.com>
Gerrit-Reviewer: Dinesh Bhat <di...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes