You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Andrew Wong (Code Review)" <ge...@cloudera.org> on 2020/03/11 00:28:06 UTC
[kudu-CR] subprocess: maintain a thread for fork/exec
Hello Attila Bukor, Adar Dembo, Hao Hao,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/15398
to review the following change.
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
subprocess: maintain a thread for fork/exec
If SubprocessServer::Init() is called from a short-lived thread, because
it does a fork/exec, the subprocess will silently be reaped when the
thread exits.
Specifically, we saw this happening upon initializing the
CatalogManager, which happens in a short-lived thread, when starting up
the Ranger client. This surfaced as us getting an EOF from the
subprocess receiver thread.
Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
---
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/subprocess/subprocess_server-test.cc
3 files changed, 44 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/15398/1
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15398 )
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 11 Mar 2020 00:58:33 +0000
Gerrit-HasComments: No
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/15398 )
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
Patch Set 3: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 11 Mar 2020 02:57:44 +0000
Gerrit-HasComments: No
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15398 )
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
subprocess: maintain a thread for fork/exec
If SubprocessServer::Init() is called from a short-lived thread, because
it does a fork/exec, the subprocess will silently be reaped when the
thread exits.
Specifically, we saw this happening upon initializing the
CatalogManager, which happens in a short-lived thread, when starting up
the Ranger client. This surfaced as us getting an EOF from the
subprocess receiver thread.
Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Reviewed-on: http://gerrit.cloudera.org:8080/15398
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <gr...@apache.org>
---
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/subprocess/subprocess_server-test.cc
3 files changed, 45 insertions(+), 1 deletion(-)
Approvals:
Kudu Jenkins: Verified
Grant Henke: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 4
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Attila Bukor, Kudu Jenkins, Adar Dembo, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15398
to look at the new patch set (#2).
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
subprocess: maintain a thread for fork/exec
If SubprocessServer::Init() is called from a short-lived thread, because
it does a fork/exec, the subprocess will silently be reaped when the
thread exits.
Specifically, we saw this happening upon initializing the
CatalogManager, which happens in a short-lived thread, when starting up
the Ranger client. This surfaced as us getting an EOF from the
subprocess receiver thread.
Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
---
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/subprocess/subprocess_server-test.cc
3 files changed, 44 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/15398/2
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change. ( http://gerrit.cloudera.org:8080/15398 )
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15398/1/src/kudu/subprocess/server.cc
File src/kudu/subprocess/server.cc:
http://gerrit.cloudera.org:8080/#/c/15398/1/src/kudu/subprocess/server.cc@109
PS1, Line 109: while (!closing_.WaitFor(MonoDelta::FromSeconds(60))) {}
How about just closing_.Wait()? The latch already loops internally.
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 1
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 11 Mar 2020 00:42:55 +0000
Gerrit-HasComments: Yes
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/15398 )
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/15398/1/src/kudu/subprocess/server.cc
File src/kudu/subprocess/server.cc:
http://gerrit.cloudera.org:8080/#/c/15398/1/src/kudu/subprocess/server.cc@109
PS1, Line 109: closing_.Wait();
> How about just closing_.Wait()? The latch already loops internally.
Done
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 2
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Comment-Date: Wed, 11 Mar 2020 00:50:24 +0000
Gerrit-HasComments: Yes
[kudu-CR] subprocess: maintain a thread for fork/exec
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Hello Attila Bukor, Kudu Jenkins, Adar Dembo, Hao Hao,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/15398
to look at the new patch set (#3).
Change subject: subprocess: maintain a thread for fork/exec
......................................................................
subprocess: maintain a thread for fork/exec
If SubprocessServer::Init() is called from a short-lived thread, because
it does a fork/exec, the subprocess will silently be reaped when the
thread exits.
Specifically, we saw this happening upon initializing the
CatalogManager, which happens in a short-lived thread, when starting up
the Ranger client. This surfaced as us getting an EOF from the
subprocess receiver thread.
Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
---
M src/kudu/subprocess/server.cc
M src/kudu/subprocess/server.h
M src/kudu/subprocess/subprocess_server-test.cc
3 files changed, 45 insertions(+), 1 deletion(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/98/15398/3
--
To view, visit http://gerrit.cloudera.org:8080/15398
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I803b1613ef1a988df1da4c908c2c37e1fbbdcf81
Gerrit-Change-Number: 15398
Gerrit-PatchSet: 3
Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)