You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@helix.apache.org by GitBox <gi...@apache.org> on 2022/09/20 17:48:51 UTC
[GitHub] [helix] mgao0 opened a new pull request, #2219: Upgrade to zookeeper 3.6.3
mgao0 opened a new pull request, #2219:
URL: https://github.com/apache/helix/pull/2219
### Issues
- [ ] My PR addresses the following Helix issues and references them in the PR description:
(#200 - Link your issue number here: You can write "Fixes #XXX". Please use the proper keyword so that the issue gets closed automatically. See https://docs.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue
Any of the following keywords can be used: close, closes, closed, fix, fixes, fixed, resolve, resolves, resolved)
### Description
- [ ] Here are some details about my PR, including screenshots of any UI changes:
(Write a concise description including what, why, how)
### Tests
- [ ] The following tests are written for this issue:
(List the names of added unit/integration tests)
- The following is the result of the "mvn test" command on the appropriate module:
(If CI test fails due to known issue, please specify the issue and test PR locally. Then copy & paste the result of "mvn test" to here.)
### Changes that Break Backward Compatibility (Optional)
- My PR contains changes that break backward compatibility or previous assumptions for certain methods or API. They include:
(Consider including all behavior changes for public methods or API. Also include these changes in merge description so that other developers are aware of these changes. This allows them to make relevant code changes in feature branches accounting for the new method/API behavior.)
### Documentation (Optional)
- In case of new functionality, my PR adds documentation in the following wiki page:
(Link the GitHub wiki you added)
### Commits
- My commits all reference appropriate Apache Helix GitHub issues in their subject lines. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Code Quality
- My diff has been formatted using helix-style.xml
(helix-style-intellij.xml if IntelliJ IDE is used)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] mgao0 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
mgao0 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975900922
##########
helix-core/helix-core-1.0.5-SNAPSHOT.ivy:
##########
@@ -63,6 +63,7 @@ under the License.
<dependency org="org.yaml" name="snakeyaml" rev="1.30" conf="compile->compile(default);runtime->runtime(default);default->default"/>
<dependency org="commons-logging" name="commons-logging-api" rev="1.1" conf="compile->compile(*),master(*);runtime->runtime(*)"/>
<dependency org="io.dropwizard.metrics" name="metrics-core" rev="4.1.14" conf="compile->compile(default);runtime->runtime(default);default->default"/>
+ <dependency org="org.xerial.snappy" name="snappy-java" rev="1.1.7" conf="compile->compile(default);runtime->runtime(default);default->default"/>
Review Comment:
My understanding is to keep pom file and ivy file in sync, pom is for maven, and ivy is for gradle. But they should be same. I'll look more into this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] mgao0 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
mgao0 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975900454
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,29 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ int numOps = 110;
+ int numOpInOps = 1000;
+ // All the paths that are going to be created as children nodes, plus one parent node
+ // Record paths so can be deleted at the end of the test
+ String[] nodePaths = new String[numOps * numOpInOps + 1];
+ nodePaths[numOps * numOpInOps] = path;
_zkClient.createPersistent(path);
- for (int i = 0; i < 110; i++) {
- List<Op> ops = new ArrayList<>(1000);
- for (int j = 0; j < 1000; j++) {
+ for (int i = 0; i < numOps; i++) {
+ List<Op> ops = new ArrayList<>(numOpInOps);
+ for (int j = 0; j < numOpInOps; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
- // Create ephemeral nodes so closing zkClient deletes them for cleanup
+ nodePaths[numOpInOps * i + j] = childPath;
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
Review Comment:
Thanks for the questions. Bottom line is that we should limit the number of ephemeral nodes that someone can create in one session. You were working with ephemeral nodes throttling before so you might also have some context.
1. It worked before because the before bump up, it doesn't need to read transaction log. Please see my 2nd point. It is not necessarily about jute max buffer. But if jute max buffer is large enough (>9MB) this test will pass without changing the test because it won't fail when reading transaction log.
2. I updated the description for a more specific commit. You can see that in 3.6.0 it added a zookeeper server shutdown mode which is non-fullyshutdown. And it will require a fastForwardDataBase in order to do this kind of shutdown. The error occurred when reading the transaction log during fastForwardDataBase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] mgao0 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
mgao0 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975824989
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,26 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
+ String[] nodePaths = new String[110001];
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ nodePaths[110000] = path;
_zkClient.createPersistent(path);
for (int i = 0; i < 110; i++) {
List<Op> ops = new ArrayList<>(1000);
for (int j = 0; j < 1000; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
+ nodePaths[1000 * i + j] = childPath;
// Create ephemeral nodes so closing zkClient deletes them for cleanup
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
}
// Reduce total creation time by batch creating znodes
_zkClient.multi(ops);
Review Comment:
Thanks for your comment!
`ops` is an iterable of `op` (which is a single operation https://zookeeper.apache.org/doc/r3.5.5/api/org/apache/zookeeper/Op.html). Transaction support is a zk server side feature so we already have it.
` multi() isn't supported at the moment` this means we don't have the multi() API in Helix ZKClient.
Please let me know if it makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] rahulrane50 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
rahulrane50 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975874399
##########
helix-core/helix-core-1.0.5-SNAPSHOT.ivy:
##########
@@ -63,6 +63,7 @@ under the License.
<dependency org="org.yaml" name="snakeyaml" rev="1.30" conf="compile->compile(default);runtime->runtime(default);default->default"/>
<dependency org="commons-logging" name="commons-logging-api" rev="1.1" conf="compile->compile(*),master(*);runtime->runtime(*)"/>
<dependency org="io.dropwizard.metrics" name="metrics-core" rev="4.1.14" conf="compile->compile(default);runtime->runtime(default);default->default"/>
+ <dependency org="org.xerial.snappy" name="snappy-java" rev="1.1.7" conf="compile->compile(default);runtime->runtime(default);default->default"/>
Review Comment:
just curious, do we need this in snapshot as well? I thought just adding this in pom.xml is good enough?
Adding it to snapshot would affect backward compatibility or not?
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,29 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ int numOps = 110;
+ int numOpInOps = 1000;
+ // All the paths that are going to be created as children nodes, plus one parent node
+ // Record paths so can be deleted at the end of the test
+ String[] nodePaths = new String[numOps * numOpInOps + 1];
+ nodePaths[numOps * numOpInOps] = path;
_zkClient.createPersistent(path);
- for (int i = 0; i < 110; i++) {
- List<Op> ops = new ArrayList<>(1000);
- for (int j = 0; j < 1000; j++) {
+ for (int i = 0; i < numOps; i++) {
+ List<Op> ops = new ArrayList<>(numOpInOps);
+ for (int j = 0; j < numOpInOps; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
- // Create ephemeral nodes so closing zkClient deletes them for cleanup
+ nodePaths[numOpInOps * i + j] = childPath;
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
Review Comment:
I understand for this test creating persistent node is also fine but let's discuss if this a problem if someone is trying to create large number of ephemeral nodes? Couple of questions :
1. Why was it working previously? Was it because jutemax buffer was 4M and now it's 1M? because if that's the case then we should ideally keep jutemaxbuffer for this test same as what we kept in prod and then reason out failure/test scenario.
2. I'm still not understanding why would a large transaction log has anything to do with data consistency. (Issue you mentioned in description). Can you please explain?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] mgao0 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
mgao0 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975900454
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,29 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ int numOps = 110;
+ int numOpInOps = 1000;
+ // All the paths that are going to be created as children nodes, plus one parent node
+ // Record paths so can be deleted at the end of the test
+ String[] nodePaths = new String[numOps * numOpInOps + 1];
+ nodePaths[numOps * numOpInOps] = path;
_zkClient.createPersistent(path);
- for (int i = 0; i < 110; i++) {
- List<Op> ops = new ArrayList<>(1000);
- for (int j = 0; j < 1000; j++) {
+ for (int i = 0; i < numOps; i++) {
+ List<Op> ops = new ArrayList<>(numOpInOps);
+ for (int j = 0; j < numOpInOps; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
- // Create ephemeral nodes so closing zkClient deletes them for cleanup
+ nodePaths[numOpInOps * i + j] = childPath;
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
Review Comment:
Thanks for the questions. Bottom line is that we should limit the number of ephemeral nodes that someone can create in one session. You were working with ephemeral nodes throttling before so you might also have some context.
1. It worked before because the before bump up, it doesn't need to read transaction log. Please see my 2nd point. It is not necessarily about jute max buffer. But if jute max buffer is large enough (>9MB) this test will pass without changing the test because it won't fail when reading transaction log.
2. I updated the description with a more specific commit. You can see that in 3.6.0 it added a zookeeper server shutdown mode which is non-fullyshutdown. And it will require a fastForwardDataBase in order to do this kind of shutdown. The error occurred when reading the transaction log during fastForwardDataBase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] Marcosrico commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
Marcosrico commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r976549393
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,26 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
+ String[] nodePaths = new String[110001];
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ nodePaths[110000] = path;
_zkClient.createPersistent(path);
for (int i = 0; i < 110; i++) {
List<Op> ops = new ArrayList<>(1000);
for (int j = 0; j < 1000; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
+ nodePaths[1000 * i + j] = childPath;
// Create ephemeral nodes so closing zkClient deletes them for cleanup
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
}
// Reduce total creation time by batch creating znodes
_zkClient.multi(ops);
Review Comment:
Got it, makes sense. Thanks for the clarification!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] mgao0 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
mgao0 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975900454
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,29 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ int numOps = 110;
+ int numOpInOps = 1000;
+ // All the paths that are going to be created as children nodes, plus one parent node
+ // Record paths so can be deleted at the end of the test
+ String[] nodePaths = new String[numOps * numOpInOps + 1];
+ nodePaths[numOps * numOpInOps] = path;
_zkClient.createPersistent(path);
- for (int i = 0; i < 110; i++) {
- List<Op> ops = new ArrayList<>(1000);
- for (int j = 0; j < 1000; j++) {
+ for (int i = 0; i < numOps; i++) {
+ List<Op> ops = new ArrayList<>(numOpInOps);
+ for (int j = 0; j < numOpInOps; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
- // Create ephemeral nodes so closing zkClient deletes them for cleanup
+ nodePaths[numOpInOps * i + j] = childPath;
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
Review Comment:
Thanks for the questions. Bottom line is that we should limit the number of ephemeral nodes that someone can create in one session. You were working with ephemeral nodes throttling before so you might also have some context.
1. It worked before because the before bump up, it doesn't need to read transaction log. Please see my 2nd point. It is not necessarily about jute max buffer. But if jute max buffer is large enough (>9MB) this test will pass without changing the test because it won't fail when reading transaction log.
2. I updated the description with a more specific commit. You can see that in 3.6.0 it added a zookeeper server shutdown mode which is non-fullyshutdown. And it will require a fastForwardDataBase in order to do this kind of shutdown. The error occurred when reading the transaction log during fastForwardDataBase. Before 3.6.0, zookeeper shutdown is just clear the zk database.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] Marcosrico commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
Marcosrico commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975811345
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,26 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
+ String[] nodePaths = new String[110001];
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ nodePaths[110000] = path;
_zkClient.createPersistent(path);
for (int i = 0; i < 110; i++) {
List<Op> ops = new ArrayList<>(1000);
for (int j = 0; j < 1000; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
+ nodePaths[1000 * i + j] = childPath;
// Create ephemeral nodes so closing zkClient deletes them for cleanup
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
}
// Reduce total creation time by batch creating znodes
_zkClient.multi(ops);
Review Comment:
Forgive me for I might not have a good understanding on this but what is `ops` and what is the purpose of calling `multi` on it? I believe `multi()` isn't supported at the moment, just wondering if I am missing something. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] NealSun96 merged pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
NealSun96 merged PR #2219:
URL: https://github.com/apache/helix/pull/2219
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] qqu0127 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
qqu0127 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975840709
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,25 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
+ String[] nodePaths = new String[110001];
Review Comment:
I assume it's 110 * 1000 + 1 ?
nit: Let's use constants and multiplication for these numbers. Same as below in the for-loop. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] rahulrane50 commented on a diff in pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
rahulrane50 commented on code in PR #2219:
URL: https://github.com/apache/helix/pull/2219#discussion_r975916486
##########
zookeeper-api/src/test/java/org/apache/helix/zookeeper/impl/client/TestRawZkClient.java:
##########
@@ -1063,23 +1063,29 @@ public void testAsyncWriteByExpectedSession() throws Exception {
* Tests getChildren() when there are an excessive number of children and connection loss happens,
* the operation should terminate and exit retry loop.
*/
- @Test(timeOut = 30 * 1000L)
+ @Test
public void testGetChildrenOnLargeNumChildren() throws Exception {
final String methodName = TestHelper.getTestMethodName();
System.out.println("Start test: " + methodName);
// Create 110K children to make packet length of children exceed 4 MB
// and cause connection loss for getChildren() operation
String path = "/" + methodName;
+ int numOps = 110;
+ int numOpInOps = 1000;
+ // All the paths that are going to be created as children nodes, plus one parent node
+ // Record paths so can be deleted at the end of the test
+ String[] nodePaths = new String[numOps * numOpInOps + 1];
+ nodePaths[numOps * numOpInOps] = path;
_zkClient.createPersistent(path);
- for (int i = 0; i < 110; i++) {
- List<Op> ops = new ArrayList<>(1000);
- for (int j = 0; j < 1000; j++) {
+ for (int i = 0; i < numOps; i++) {
+ List<Op> ops = new ArrayList<>(numOpInOps);
+ for (int j = 0; j < numOpInOps; j++) {
String childPath = path + "/" + UUID.randomUUID().toString();
- // Create ephemeral nodes so closing zkClient deletes them for cleanup
+ nodePaths[numOpInOps * i + j] = childPath;
ops.add(
- Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.EPHEMERAL));
+ Op.create(childPath, new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE, CreateMode.PERSISTENT));
Review Comment:
Ah i got your point now, thanks a lot for explaining!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org
[GitHub] [helix] mgao0 commented on pull request #2219: Upgrade to zookeeper 3.6.3
Posted by GitBox <gi...@apache.org>.
mgao0 commented on PR #2219:
URL: https://github.com/apache/helix/pull/2219#issuecomment-1255280116
This PR is ready to be merged, approved by [rahulrane50](https://github.com/rahulrane50) and [qqu0127](https://github.com/qqu0127). Thanks!
Commit message:
Bump up zookeeper dependency version to 3.6.3
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@helix.apache.org
For additional commands, e-mail: reviews-help@helix.apache.org