You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2017/04/17 23:21:25 UTC
[kudu-CR] pstack watcher: set a limit on backtrace depth
Hello Mike Percy,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/6658
to review the following change.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
pstack_watcher: set a limit on backtrace depth
On RHEL 6.8, pstack_watcher-test times out fairly reliably, with a 'gdb'
process left spinning 100% CPU.
Some googling found a report of a similar spinning GDB while taking a
backtrace[1]. This report suggests setting a backtrace limit --
apparently the spinning is due to gdb getting in some infinite loop
trying to walk the stack with insufficient/incorrect debug info.
This patch sets such a limit. I verified on a box which was reliably
reproducing the timeout that it is now fixed.
[1] https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/434168
Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
---
M src/kudu/util/pstack_watcher.cc
1 file changed, 2 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/6658/1
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/6658/1//COMMIT_MSG
Commit Message:
PS1, Line 14: apparently the spinning is due to gdb getting in some infinite loop
: trying to walk the stack with insufficient/incorrect debug info.
> Do we have a reference for this part though? I didn't see that conclusion r
i'm guessing it's that it's not able to interpret the more modern debuginfo or something. It's hard to debug (when I tried to gdb gdb things didn't go well) and since it's not our bug, I'm satisfied with the workaround.
http://gerrit.cloudera.org:8080/#/c/6658/1/src/kudu/util/pstack_watcher.cc
File src/kudu/util/pstack_watcher.cc:
Line 139: argv.push_back("set backtrace limit 100");
> Could you add a comment explaining why we do this?
will do
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Hello Mike Percy,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/6658
to look at the new patch set (#2).
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
pstack_watcher: set a limit on backtrace depth
On RHEL 6.8, pstack_watcher-test times out fairly reliably, with a 'gdb'
process left spinning 100% CPU.
Some googling found a report of a similar spinning GDB while taking a
backtrace[1]. This report suggests setting a backtrace limit --
apparently the spinning is due to gdb getting in some infinite loop
trying to walk the stack with insufficient/incorrect debug info.
This patch sets such a limit. I verified on a box which was reliably
reproducing the timeout that it is now fixed.
[1] https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/434168
Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
---
M src/kudu/util/pstack_watcher.cc
1 file changed, 10 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/58/6658/2
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
Patch Set 1:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/6658/1//COMMIT_MSG
Commit Message:
PS1, Line 14: apparently the spinning is due to gdb getting in some infinite loop
: trying to walk the stack with insufficient/incorrect debug info.
Do we have a reference for this part though? I didn't see that conclusion reached in the linked bug report or any of its related bugs.
http://gerrit.cloudera.org:8080/#/c/6658/1/src/kudu/util/pstack_watcher.cc
File src/kudu/util/pstack_watcher.cc:
Line 139: argv.push_back("set backtrace limit 100");
Could you add a comment explaining why we do this?
I think the rest of the commands are self-explanatory, but feel free to also comment any that you think may not be.
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: No
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Mike Percy (Code Review)" <ge...@cloudera.org>.
Mike Percy has posted comments on this change.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-HasComments: No
[kudu-CR] pstack watcher: set a limit on backtrace depth
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged.
Change subject: pstack_watcher: set a limit on backtrace depth
......................................................................
pstack_watcher: set a limit on backtrace depth
On RHEL 6.8, pstack_watcher-test times out fairly reliably, with a 'gdb'
process left spinning 100% CPU.
Some googling found a report of a similar spinning GDB while taking a
backtrace[1]. This report suggests setting a backtrace limit --
apparently the spinning is due to gdb getting in some infinite loop
trying to walk the stack with insufficient/incorrect debug info.
This patch sets such a limit. I verified on a box which was reliably
reproducing the timeout that it is now fixed.
[1] https://bugs.launchpad.net/ubuntu/+source/gdb/+bug/434168
Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Reviewed-on: http://gerrit.cloudera.org:8080/6658
Reviewed-by: Adar Dembo <ad...@cloudera.com>
Reviewed-by: Mike Percy <mp...@apache.org>
Tested-by: Kudu Jenkins
---
M src/kudu/util/pstack_watcher.cc
1 file changed, 10 insertions(+), 0 deletions(-)
Approvals:
Mike Percy: Looks good to me, approved
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/6658
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ide46209256fd4ced6b08065c2ae3c0046de49565
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Mike Percy <mp...@apache.org>
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>