You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@zookeeper.apache.org by TyqITstudent <gi...@git.apache.org> on 2018/11/04 04:14:16 UTC
[GitHub] zookeeper pull request #686: Zookeeper 3167
GitHub user TyqITstudent opened a pull request:
https://github.com/apache/zookeeper/pull/686
Zookeeper 3167
1. In production environment, there will be always a situation that there are a lot of recursive sub nodes of one node. We need to count total number of it.
2. Now, we can only use API getChildren which returns the List<String> of first level of sub nodes. We need to iterate every sub node to get recursive sub nodes. It will cost a lot of time.
3. In zookeeper server side, it uses Hasp<String, DataNode> to store node. The key of the map represents the path of the node. We can iterate the map get total number of all levels of sub nodes of one node.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/TyqITstudent/zookeeper ZOOKEEPER-3167
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zookeeper/pull/686.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #686
----
commit 620f9f4c22192e17d0fb9a4bb5486dcd2d8db2bd
Author: Flavio Paiva Junqueira <fp...@...>
Date: 2014-11-29T15:55:50Z
ZOOKEEPER-2064 Prevent resource leak in various classes (Ted Yu via fpj)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1642441 13f79535-47bb-0310-9956-ffa450edef68
commit a9718779da86d9e61e2432e6e87c031db311f42f
Author: Michi Mutsuzaki <mi...@...>
Date: 2015-02-11T07:01:42Z
ZOOKEEPER-1949 recipes jar not included in the distribution package (Rakesh R via michim)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1658890 13f79535-47bb-0310-9956-ffa450edef68
commit 7b3ba5fca5802cec37605f8099c6fdead6d3c872
Author: Michi Mutsuzaki <mi...@...>
Date: 2015-02-22T21:11:11Z
ZOOKEEPER-2114 jute generated allocate_* functions are not externally visible (Tim Crowder via michim)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1661562 13f79535-47bb-0310-9956-ffa450edef68
commit 415bcd091228caef78e0d8b7bbdbbe67c48dee2f
Author: Michi Mutsuzaki <mi...@...>
Date: 2015-02-22T22:08:16Z
ZOOKEEPER-2073 Memory leak on zookeeper_close (Dave Gosselin via michim)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1661573 13f79535-47bb-0310-9956-ffa450edef68
commit 2c271faa1517ddaca0747cdf8760a1fb670f955f
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-04-07T00:50:58Z
ZOOKEEPER-2056 Zookeeper 3.4.x and 3.5.0-alpha is not OSGi compliant (Deiwin Sarjas via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1671716 13f79535-47bb-0310-9956-ffa450edef68
commit 87fc5e19bfae81508de96b4bb86d0aa1d5374c1c
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-04-18T00:47:56Z
ZOOKEEPER-2146 BinaryInputArchive readString should check length before allocating memory (Hongchao Deng via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1674424 13f79535-47bb-0310-9956-ffa450edef68
commit 7c3fe9d224e318a842b27113b7594cc7a33f88ce
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-05-03T18:02:51Z
ZOOKEEPER-2174 JUnit4ZKTestRunner logs test failure for all exceptions
JUnit4ZKTestRunner logs test failure for all exceptions, even if the test
method is annotated with an expected exception (Chris Nauroth via rgs).
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1677463 13f79535-47bb-0310-9956-ffa450edef68
commit c7830c532e6a8fe0d14c82f1ad1a3ebcdfc83187
Author: Rakesh Radhakrishnan <ra...@...>
Date: 2015-05-15T04:06:54Z
ZOOKEEPER-2126 Improve exit log messsage of EventThread and SendThread by adding SessionId (surendra singh lilhore via rakeshr)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1679491 13f79535-47bb-0310-9956-ffa450edef68
commit 1169a5b5eca3982d24359c27f669af429ff71157
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-05-18T07:38:36Z
ZOOKEEPER-1077: C client lib doesn't build on Solaris (Chris Nauroth via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1679953 13f79535-47bb-0310-9956-ffa450edef68
commit 6b8f0cd84e22494a664fcb7369d15286b083a1ed
Author: Michi Mutsuzaki <mi...@...>
Date: 2015-05-24T06:33:43Z
ZOOKEEPER-2186 QuorumCnxManager#receiveConnection may crash with random input (rgs via michim)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1681417 13f79535-47bb-0310-9956-ffa450edef68
commit ba96bd8a68c8eacab3a9e61b401cab0f35c7c314
Author: Michi Mutsuzaki <mi...@...>
Date: 2015-05-24T06:37:32Z
ZOOKEEPER-2124 Allow Zookeeper version string to have underscore '_' (Chris Nauroth via michim)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1681418 13f79535-47bb-0310-9956-ffa450edef68
commit 6ce171a99878f745c0b7f212e27c9064f95cc4e2
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-05-29T19:51:58Z
ZOOKEEPER-2179: Typo in Watcher.java (Archana T via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1682541 13f79535-47bb-0310-9956-ffa450edef68
commit e269cd5f4f76e8e2472de47354525674ad434659
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-02T20:43:29Z
ZOOKEEPER-2096: C client builds with incorrect error codes in VisualStudio 2010+
(Vitaly Stakhovsky via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683180 13f79535-47bb-0310-9956-ffa450edef68
commit a351209b3bd7e3123ce742789063784f6322d7fc
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-04T16:27:19Z
ZOOKEEPER-2194: Let DataNode.getChildren() return an unmodifiable view of its
children set (Hitoshi Mitake via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683589 13f79535-47bb-0310-9956-ffa450edef68
commit 67cb77e74d0d8389a5d58781d7ed9b44ca3b3c38
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-05T18:44:25Z
ZOOKEEPER-2205: Log type of unexpected quorum packet in learner handler loop
(Hitoshi Mitake via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683838 13f79535-47bb-0310-9956-ffa450edef68
commit 67019674379e6fac647d5eb23b694f1172d2849f
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-06T16:53:10Z
ZOOKEEPER-2201: Network issues can cause cluster to hang due to near-deadlock
(Donny Nadolny via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1683931 13f79535-47bb-0310-9956-ffa450edef68
commit 6fdb35ff1e1d3fbd1f5f22f9d4a8581ef8244851
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-11T18:14:21Z
ZOOKEEPER-2213: Empty path in Set crashes server and prevents restart
(Hongchao Deng via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1684958 13f79535-47bb-0310-9956-ffa450edef68
commit 62888591c33968c3bf4fc79bb843cae9ff3ae5cf
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-13T00:49:48Z
ZOOKEEPER-706: Large numbers of watches can cause session re-establishment to fail
(Chris Thunes via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1685205 13f79535-47bb-0310-9956-ffa450edef68
commit b407b1593ec6d1b3879f532a3aff700cec50bf75
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-06-18T18:58:48Z
ZOOKEEPER-602: log all exceptions not caught by ZK threads
(Rakesh R via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1686289 13f79535-47bb-0310-9956-ffa450edef68
commit d66048c73a550aae8d34ccbae6ec384f3e8f74a3
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-07-06T15:48:24Z
ZOOKEEPER-2224: Four letter command hangs when network is slow
(Arshad Mohammad via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1689431 13f79535-47bb-0310-9956-ffa450edef68
commit 0d254c8df43acae08c71edd299bda35b593d2f06
Author: Michi Mutsuzaki <mi...@...>
Date: 2015-07-28T04:39:20Z
ZOOKEEPER-2235: License update (fpj via michim)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1692991 13f79535-47bb-0310-9956-ffa450edef68
commit b2703be61c7d6e31ec2b535f5f130b1703eefad9
Author: Rakesh Radhakrishnan <ra...@...>
Date: 2015-08-01T03:39:21Z
ZOOKEEPER-2237 Port async multi to 3.4 branch (Ivan Kelly via rakeshr)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1693680 13f79535-47bb-0310-9956-ffa450edef68
commit 8de5ad8d99a0f974a4393cc4eda546c66643dd96
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-08-05T20:17:18Z
ZOOKEEPER-2239: JMX State from LocalPeerBean incorrect
(Kevin Lee via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1694318 13f79535-47bb-0310-9956-ffa450edef68
commit cfff3f06ae5c208ab566ab4fd941df835b880e53
Author: Hongchao Deng <hd...@...>
Date: 2015-08-17T20:52:07Z
ZOOKEEPER-1907 Improve Thread handling (Rakesh R via hdeng)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1696337 13f79535-47bb-0310-9956-ffa450edef68
commit 908ea4084ea7b38b74b1fcaab7bfdf108d2a6093
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-08-25T05:13:25Z
ZOOKEEPER-1927: zkServer.sh fails to read dataDir (and others)
from zoo.cfg on Solaris 10 (grep issue, manifests as FAILED TO WRITE PID)
(Chris Nauroth via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1697552 13f79535-47bb-0310-9956-ffa450edef68
commit 3a7c6a5f010086a2305bbbde5efbf6ddb2e767ed
Author: Flavio Paiva Junqueira <fp...@...>
Date: 2015-09-03T23:00:20Z
ZOOKEEPER-2033: zookeeper follower fails to start after a restart immediately following a new epoch (Asad Saeed via fpj)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1701146 13f79535-47bb-0310-9956-ffa450edef68
commit 1c1a2c785b4b18e639ff4497d96d721aaf11eb40
Author: Rakesh Radhakrishnan <ra...@...>
Date: 2015-09-06T17:48:13Z
ZOOKEEPER-2256: Zookeeper is not using specified JMX port in zkEnv.sh(Arshad Mohammad via rakeshr)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1701503 13f79535-47bb-0310-9956-ffa450edef68
commit 8f326d5f5b2af8710ec64941fa5db7bca6bc06da
Author: Raúl Gutiérrez Segalés <rg...@...>
Date: 2015-09-10T04:12:05Z
ZOOKEEPER-2270: Allow MBeanRegistry to be overridden for better unit tests
(Jordan Zimmerman via rgs)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1702157 13f79535-47bb-0310-9956-ffa450edef68
commit 124670eb739d064e6bd01da451e92fe0084bdc6c
Author: Chris Nauroth <cn...@...>
Date: 2015-09-11T06:02:29Z
ZOOKEEPER-2040: Server to log underlying cause of SASL connection problems. (Steve Loughran via cnauroth)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1702381 13f79535-47bb-0310-9956-ffa450edef68
commit 9350096d66c4e405f312a453185471b2306fb9ee
Author: Rakesh Radhakrishnan <ra...@...>
Date: 2015-09-17T07:12:00Z
ZOOKEEPER-2245: SimpleSysTest test cases fails (Arshad Mohammad via rakeshr)
git-svn-id: https://svn.apache.org/repos/asf/zookeeper/branches/branch-3.4@1703504 13f79535-47bb-0310-9956-ffa450edef68
----
---
[GitHub] zookeeper issue #686: Zookeeper 3167
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/zookeeper/pull/686
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2580/
---
[GitHub] zookeeper pull request #686: Zookeeper 3167
Posted by TyqITstudent <gi...@git.apache.org>.
Github user TyqITstudent closed the pull request at:
https://github.com/apache/zookeeper/pull/686
---
[GitHub] zookeeper issue #686: Zookeeper 3167
Posted by maoling <gi...@git.apache.org>.
Github user maoling commented on the issue:
https://github.com/apache/zookeeper/pull/686
@TyqITstudent
some mistakes with the way of PR.if you are chinese,you can also refer to this[ post ](https://zhuanlan.zhihu.com/p/23457016)
---
[GitHub] zookeeper issue #686: Zookeeper 3167
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit commented on the issue:
https://github.com/apache/zookeeper/pull/686
Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/PreCommit-ZOOKEEPER-github-pr-build/2577/
---
[GitHub] zookeeper issue #686: Zookeeper 3167
Posted by maoling <gi...@git.apache.org>.
Github user maoling commented on the issue:
https://github.com/apache/zookeeper/pull/686
@TyqITstudent quick comment:
- you can follow an example of `GetChildren2Test`,create an UT:e.g `GetAllChildrenNumberTest`，create some zk-nodes,then do some asserts of the number of the specific node. do a relevant test in the `DataTreeTest `could be better.
- BTW:
the codes need another rebase.
change the github tilte to "`ZOOKEEPER-3167:add an API to get total count of recursive sub nodes of one node`"
---
[GitHub] zookeeper issue #686: Zookeeper 3167
Posted by anmolnar <gi...@git.apache.org>.
Github user anmolnar commented on the issue:
https://github.com/apache/zookeeper/pull/686
@TyqITstudent Please create pull request for the master branch first. We can talk about integrating this into 3.5, but we don't accept new features for 3.4
Would be better to use `key.startsWith(path)` instead of contains to make sure the path starts with the parent's.
Have you considered limiting the access to this command? I'm not sure if it's needed (say we've merged the recursive flag of `setAcl` command recently which didn't have limits either), but this command could be too expensive, if the ensemble has millions of nodes. As a consequence the command can be used for a DOS attach, similarly to 4lw commands which have been blacklisted previously.
---
[GitHub] zookeeper issue #686: Zookeeper 3167
Posted by maoling <gi...@git.apache.org>.
Github user maoling commented on the issue:
https://github.com/apache/zookeeper/pull/686
- agree with this
> Would be better to use key.startsWith(path) instead of contains to make sure the path starts with the parent's.
- `getAllChildNumber` is just foreach the treemap,no limit may be also ok.
but `setAcl ` does a recursion which really needs a limit.we can create a seperate jira to do this.
> Have you considered limiting the access to this command? I'm not sure if it's needed (say we've merged the recursive flag of setAcl command recently which didn't have limits either)
BTW.
- this patch needs a unit test.
- Do we need a CLI for this feature?we can create a seperate jira to do this.
---