You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Todd Lipcon (Code Review)" <ge...@cloudera.org> on 2017/06/12 22:59:33 UTC
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Hello Jean-Daniel Cryans, Hao Hao,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/7160
to review the following change.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
KUDU-2003. Fix fd-cache related tests on high-core systems
These two tests would fail on systems with a number of cores that was
not a divisor of 192. This patch switches the tests to use a non-sharded
LRU cache so that the behavior is not dependent on the core count.
I verified that these tests now pass on a machine with 88 logical cores.
Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
---
M src/kudu/fs/block_manager-stress-test.cc
M src/kudu/util/file_cache-stress-test.cc
2 files changed, 24 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/60/7160/1
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7160/2/src/kudu/util/file_cache-stress-test.cc
File src/kudu/util/file_cache-stress-test.cc:
Line 93: // Use a single shard. Otherwise, the cache can be a little bit "sloppy"
> The main downside of using a single shard (and the reason I didn't do that
yea, I figure the coverage via all of the end-to-end system/integration tests is sufficient.
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has posted comments on this change.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
Patch Set 1:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7160/1/src/kudu/util/file_cache-stress-test.cc
File src/kudu/util/file_cache-stress-test.cc:
Line 83: static const int kTestMaxOpenFiles = 100;
> Any rules to pick this magic number?
No particular reason. Just small enough that we don't need any particularly fancy ulimit.
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Posted by "Jean-Daniel Cryans (Code Review)" <ge...@cloudera.org>.
Jean-Daniel Cryans has posted comments on this change.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
Patch Set 1: Code-Review+1
Seems fine but I really don't know that part of the code very well.
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: No
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Posted by "Hao Hao (Code Review)" <ge...@cloudera.org>.
Hao Hao has posted comments on this change.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7160/1/src/kudu/util/file_cache-stress-test.cc
File src/kudu/util/file_cache-stress-test.cc:
Line 83: static const int kTestMaxOpenFiles = 100;
Any rules to pick this magic number?
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Posted by "Adar Dembo (Code Review)" <ge...@cloudera.org>.
Adar Dembo has posted comments on this change.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
Patch Set 2: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/7160/2/src/kudu/util/file_cache-stress-test.cc
File src/kudu/util/file_cache-stress-test.cc:
Line 93: // Use a single shard. Otherwise, the cache can be a little bit "sloppy"
The main downside of using a single shard (and the reason I didn't do that when I added the test) is poorer coverage of the cache sharding. Do we get adequate coverage of that elsewhere?
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2003. Fix fd-cache related tests on high-core systems
Posted by "Todd Lipcon (Code Review)" <ge...@cloudera.org>.
Todd Lipcon has submitted this change and it was merged.
Change subject: KUDU-2003. Fix fd-cache related tests on high-core systems
......................................................................
KUDU-2003. Fix fd-cache related tests on high-core systems
These two tests would fail on systems with a number of cores that was
not a divisor of 192. This patch switches the tests to use a non-sharded
LRU cache so that the behavior is not dependent on the core count.
I verified that these tests now pass on a machine with 88 logical cores.
Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Reviewed-on: http://gerrit.cloudera.org:8080/7160
Tested-by: Kudu Jenkins
Reviewed-by: Adar Dembo <ad...@cloudera.com>
---
M src/kudu/fs/block_manager-stress-test.cc
M src/kudu/util/file_cache-stress-test.cc
2 files changed, 24 insertions(+), 16 deletions(-)
Approvals:
Adar Dembo: Looks good to me, approved
Kudu Jenkins: Verified
--
To view, visit http://gerrit.cloudera.org:8080/7160
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: merged
Gerrit-Change-Id: Ibd901730f16c70bb2e4a60464ab08a9a6f14c32f
Gerrit-PatchSet: 3
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <to...@apache.org>
Gerrit-Reviewer: Adar Dembo <ad...@cloudera.com>
Gerrit-Reviewer: Dan Burkert <da...@apache.org>
Gerrit-Reviewer: Hao Hao <ha...@cloudera.com>
Gerrit-Reviewer: Jean-Daniel Cryans <jd...@apache.org>
Gerrit-Reviewer: Kudu Jenkins
Gerrit-Reviewer: Todd Lipcon <to...@apache.org>