You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@zookeeper.apache.org by GitBox <gi...@apache.org> on 2022/06/30 08:51:15 UTC
[GitHub] [zookeeper] BukrosSzabolcs opened a new pull request, #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
BukrosSzabolcs opened a new pull request, #1902:
URL: https://github.com/apache/zookeeper/pull/1902
add a tool to recursively collect and display child count and data
stored in sub-trees
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] sonatype-lift[bot] commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r911907134
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
Review Comment:
*[PATH_TRAVERSAL_IN](https://find-sec-bugs.github.io/bugs.htm#PATH_TRAVERSAL_IN):* This API (java/io/File.<init>(Ljava/lang/String;)V) reads a file whose location might be specified by user input
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=289522632&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=289522632&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289522632&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289522632&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=289522632&lift_comment_rating=5) ]
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
+ try (InputStream is = SnapStream.getInputStream(snapshotFile)) {
+ InputArchive ia = BinaryInputArchive.getArchive(is);
+
+ FileSnap fileSnap = new FileSnap(null);
+
+ DataTree dataTree = new DataTree();
+ Map<Long,Integer> sessions = new HashMap<Long,Integer>();
+
+ fileSnap.deserialize(dataTree, sessions, ia);
Review Comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 69 is not released after line 76.
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=289522818&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=289522818&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289522818&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289522818&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=289522818&lift_comment_rating=5) ]
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
Review Comment:
*[SC2164](https://github.com/koalaman/shellcheck/wiki/SC2164):* Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=289523767&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=289523767&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523767&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523767&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=289523767&lift_comment_rating=5) ]
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
Review Comment:
*[SC1091](https://github.com/koalaman/shellcheck/wiki/SC1091):* Not following: ./zkEnv.sh was not specified as input (see shellcheck -x).
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=289523769&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=289523769&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523769&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523769&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=289523769&lift_comment_rating=5) ]
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
Review Comment:
*[SC1091](https://github.com/koalaman/shellcheck/wiki/SC1091):* Not following: ./../libexec/zkEnv.sh was not specified as input (see shellcheck -x).
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=289523768&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=289523768&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523768&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523768&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=289523768&lift_comment_rating=5) ]
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
+fi
+
+"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
Review Comment:
*[SC2086](https://github.com/koalaman/shellcheck/wiki/SC2086):* Double quote to prevent globbing and word splitting.
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=289523770&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=289523770&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523770&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=289523770&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=289523770&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] asfgit closed pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
asfgit closed pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
URL: https://github.com/apache/zookeeper/pull/1902
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] sonatype-lift[bot] commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r910787432
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotSumFormatter.java:
##########
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotSumFormatter {
+
+ /**
+ * USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
+ System.exit(2);
+ }
+
+ new SnapshotSumFormatter().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ InputStream is =
Review Comment:
*[PATH_TRAVERSAL_IN](https://find-sec-bugs.github.io/bugs.htm#PATH_TRAVERSAL_IN):* This API (java/io/FileInputStream.<init>(Ljava/lang/String;)V) reads a file whose location might be specified by user input
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288734858&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288734858&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288734858&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288734858&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288734858&lift_comment_rating=5) ]
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotSumFormatter.java:
##########
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotSumFormatter {
+
+ /**
+ * USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
+ System.exit(2);
+ }
+
+ new SnapshotSumFormatter().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ InputStream is =
+ new CheckedInputStream(new BufferedInputStream(new FileInputStream(snapshotFileName)),
+ new Adler32());
+ InputArchive ia = BinaryInputArchive.getArchive(is);
+
+ FileSnap fileSnap = new FileSnap(null);
+
+ DataTree dataTree = new DataTree();
+ Map<Long,Integer> sessions = new HashMap<Long,Integer>();
+
+ fileSnap.deserialize(dataTree, sessions, ia);
Review Comment:
*RESOURCE_LEAK:* resource of type `java.io.DataInputStream` acquired by call to `getArchive(...)` at line 72 is not released after line 79.
**Note**: potential exception at line 79
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288735472&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288735472&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735472&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735472&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288735472&lift_comment_rating=5) ]
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotSumFormatter.java:
##########
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotSumFormatter {
+
+ /**
+ * USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
+ System.exit(2);
+ }
+
+ new SnapshotSumFormatter().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ InputStream is =
+ new CheckedInputStream(new BufferedInputStream(new FileInputStream(snapshotFileName)),
+ new Adler32());
+ InputArchive ia = BinaryInputArchive.getArchive(is);
Review Comment:
*RESOURCE_LEAK:* resource of type `java.util.zip.CheckedInputStream` acquired by call to `new()` at line 69 is not released after line 72.
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288735234&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288735234&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735234&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735234&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288735234&lift_comment_rating=5) ]
##########
bin/zkSnapShotSumToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
Review Comment:
*[SC2164](https://github.com/koalaman/shellcheck/wiki/SC2164):* Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288735989&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288735989&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735989&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735989&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288735989&lift_comment_rating=5) ]
##########
bin/zkSnapShotSumToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
Review Comment:
*[SC1091](https://github.com/koalaman/shellcheck/wiki/SC1091):* Not following: ./../libexec/zkEnv.sh was not specified as input (see shellcheck -x).
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288735990&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288735990&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735990&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735990&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288735990&lift_comment_rating=5) ]
##########
bin/zkSnapShotSumToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
Review Comment:
*[SC1091](https://github.com/koalaman/shellcheck/wiki/SC1091):* Not following: ./zkEnv.sh was not specified as input (see shellcheck -x).
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288735991&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288735991&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735991&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735991&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288735991&lift_comment_rating=5) ]
##########
bin/zkSnapShotSumToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
+fi
+
+"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
Review Comment:
*[SC2086](https://github.com/koalaman/shellcheck/wiki/SC2086):* Double quote to prevent globbing and word splitting.
Reply with *"**@sonatype-lift help**"* for info about LiftBot commands.
Reply with *"**@sonatype-lift ignore**"* to tell LiftBot to leave out the above finding from this PR.
Reply with *"**@sonatype-lift ignoreall**"* to tell LiftBot to leave out all the findings from this PR and from the status bar in Github.
When talking to LiftBot, you need to **refresh** the page to see its response. [Click here](https://help.sonatype.com/lift/talking-to-lift) to get to know more about LiftBot commands.
---
Was this a good recommendation?
[ [🙁 Not relevant](https://www.sonatype.com/lift-comment-rating?comment=288735992&lift_comment_rating=1) ] - [ [😕 Won't fix](https://www.sonatype.com/lift-comment-rating?comment=288735992&lift_comment_rating=2) ] - [ [😑 Not critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735992&lift_comment_rating=3) ] - [ [🙂 Critical, will fix](https://www.sonatype.com/lift-comment-rating?comment=288735992&lift_comment_rating=4) ] - [ [😊 Critical, fixing now](https://www.sonatype.com/lift-comment-rating?comment=288735992&lift_comment_rating=5) ]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] symat commented on pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
symat commented on PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#issuecomment-1183079964
I'm merging it now to master. Thanks @BukrosSzabolcs for the contribution!!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918979226
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
Review Comment:
Okay, adding it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918634107
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
+fi
+
+"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
Review Comment:
@sonatype-lift ignoreall
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] anmolnar commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
anmolnar commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918829491
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
Review Comment:
Same here.
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
+ try (InputStream is = SnapStream.getInputStream(snapshotFile)) {
+ InputArchive ia = BinaryInputArchive.getArchive(is);
+
+ FileSnap fileSnap = new FileSnap(null);
+
+ DataTree dataTree = new DataTree();
+ Map<Long,Integer> sessions = new HashMap<Long,Integer>();
+
+ fileSnap.deserialize(dataTree, sessions, ia);
Review Comment:
I don't get this warning message: the `inputStream` is defined in the `try()` clause which means it will be automatically released at the end of the block. What's wrong?
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
+fi
+
+"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
Review Comment:
Why not adding double-quotes to `$JVMFLAGS` instead?
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
Review Comment:
I think you can ignore this.
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
Review Comment:
Maybe you could try adding:
```
# shellcheck source=somefile
```
Based on: https://github.com/koalaman/shellcheck/wiki/SC1091
Otherwise feel free to ignore this one too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918862079
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
+fi
+
+"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
Review Comment:
Fixing this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r911947557
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
Review Comment:
@sonatype-lift ignore
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] sonatype-lift[bot] commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918634205
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
+else
+ . "$ZOOBINDIR"/zkEnv.sh
+fi
+
+"$JAVA" -cp "$CLASSPATH" $JVMFLAGS \
Review Comment:
The **ignoreall** command is active on this PR, all the existing Lift issues are ignored.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#issuecomment-1181412284
@symat I have fixed the spotbug issues. The test failure looks unrelated. Should we re-run it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] anmolnar commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
anmolnar commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918968567
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
+ try (InputStream is = SnapStream.getInputStream(snapshotFile)) {
+ InputArchive ia = BinaryInputArchive.getArchive(is);
+
+ FileSnap fileSnap = new FileSnap(null);
+
+ DataTree dataTree = new DataTree();
+ Map<Long,Integer> sessions = new HashMap<Long,Integer>();
+
+ fileSnap.deserialize(dataTree, sessions, ia);
Review Comment:
Yes, because it's a standalone tool. You can ignore that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918860894
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
Review Comment:
@anmolnar I'm not sure adding shellcheck is helpful here. The `libexec` folder does not exist at this point so we can not verify if it has the file. I could add `# shellcheck source=/dev/null` but that basically circumvents the shellcheck and at that point I would prefer not to touch the script at all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] symat commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
symat commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r910815039
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotSumFormatter.java:
##########
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotSumFormatter {
+
+ /**
+ * USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
Review Comment:
could you be a bit more "helpful" :) here? like:
```
SnapshotSumFormatter is a tool traversing the whole snapshot and summing up the data in the subtrees.
USAGE:
SnapshotSumFormatter <snapshot_file> <starting_node> <max_depth>
snapshot_file: path to the zookeeper snapshot
starting_node: the path in the zookeeper tree where the traversal should begin
max_depth: the depth where you would like to get the output (data in paths longer than the depth will be still summed up, but won't be printed out)
```
Or something like this...
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotSumFormatter.java:
##########
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotSumFormatter {
+
+ /**
+ * USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
Review Comment:
I just saw your documentation in the code above the class, that might be even better...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918839961
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
+ try (InputStream is = SnapStream.getInputStream(snapshotFile)) {
+ InputArchive ia = BinaryInputArchive.getArchive(is);
+
+ FileSnap fileSnap = new FileSnap(null);
+
+ DataTree dataTree = new DataTree();
+ Map<Long,Integer> sessions = new HashMap<Long,Integer>();
+
+ fileSnap.deserialize(dataTree, sessions, ia);
Review Comment:
`BinaryInputArchive.getArchive(is)` creates a new DataInputStream based on the inputStream. The `try()` only closes the inputStream, the DataInputStream is technically never closed. I see this as a non-issue and this is how SnapshotFormatter is written too, so it should be fine like this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] anmolnar commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
anmolnar commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r918967194
##########
bin/zkSnapshotRecursiveSummaryToolkit.sh:
##########
@@ -0,0 +1,38 @@
+#!/usr/bin/env bash
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# If this scripted is run out of /usr/bin or some other system bin directory
+# it should be linked to and not copied. Things like java jar files are found
+# relative to the canonical path of this script.
+#
+
+# use POSIX interface, symlink is followed automatically
+ZOOBIN="${BASH_SOURCE-$0}"
+ZOOBIN="$(dirname "${ZOOBIN}")"
+ZOOBINDIR="$(cd "${ZOOBIN}"; pwd)"
+
+if [ -e "$ZOOBIN/../libexec/zkEnv.sh" ]; then
+ . "$ZOOBINDIR"/../libexec/zkEnv.sh
Review Comment:
I mean
```
# shellcheck source=/bin/zkEnv.sh
```
Don't waste too much time on this, feel free to ignore the comment.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] BukrosSzabolcs commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
BukrosSzabolcs commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r911902646
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotSumFormatter.java:
##########
@@ -0,0 +1,124 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+
+import java.io.BufferedInputStream;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+import java.util.zip.Adler32;
+import java.util.zip.CheckedInputStream;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotSumFormatter {
+
+ /**
+ * USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println("USAGE: SnapshotSumFormatter snapshot_file starting_node max_depth");
Review Comment:
@symat Thank you for your feedback!
I'm trying to follow the patterns in SnapshotFormatter and will apply or ignore sonatypes based on that. If that is acceptable for you.
I have extended the Tools documentation with a section on the new tool. Renamed it to SnapshotRecursiveSummar / zkSnapshotRecursiveSummaryToolkit.sh and added an extended usage based on your suggestion.
> what about also printing the number of the children ZNodes?
That is already printed. Please check the example output in the jira ticket. Or you would like it presented in some other way?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [zookeeper] sonatype-lift[bot] commented on a diff in pull request #1902: ZOOKEEPER-4566: Create tool for recursive snapshot analysis
Posted by GitBox <gi...@apache.org>.
sonatype-lift[bot] commented on code in PR #1902:
URL: https://github.com/apache/zookeeper/pull/1902#discussion_r911947595
##########
zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotRecursiveSummary.java:
##########
@@ -0,0 +1,138 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ * <p>
+ * http://www.apache.org/licenses/LICENSE-2.0
+ * <p>
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.zookeeper.server;
+
+import org.apache.jute.BinaryInputArchive;
+import org.apache.jute.InputArchive;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.apache.zookeeper.server.persistence.FileSnap;
+import org.apache.zookeeper.server.persistence.SnapStream;
+
+import java.io.File;
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Set;
+
+/**
+ * Recursively processes a snapshot file collecting child node count and summarizes the data size
+ * below each node.
+ * "starting_node" defines the node where the recursion starts
+ * "max_depth" defines the depth where the tool still writes to the output.
+ * 0 means there is no depth limit, every non-leaf node's stats will be displayed, 1 means it will
+ * only contain the starting node's and it's children's stats, 2 ads another level and so on.
+ * This ONLY affects the level of details displayed, NOT the calculation.
+ */
+@InterfaceAudience.Public public class SnapshotRecursiveSummary {
+
+ /**
+ * USAGE: SnapsotRecursiveSummary snapshot_file starting_node max_depth
+ *
+ */
+ public static void main(String[] args) throws Exception {
+ if (args.length != 3) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+ int maxDepth = 0;
+ try {
+ maxDepth = Integer.valueOf(args[2]).intValue();
+ } catch (NumberFormatException e) {
+ System.err.println(getUsage());
+ System.exit(2);
+ }
+
+ new SnapshotRecursiveSummary().run(args[0], args[1], maxDepth);
+ }
+
+ public void run(String snapshotFileName, String startingNode, int maxDepth) throws IOException {
+ File snapshotFile = new File(snapshotFileName);
Review Comment:
I've recorded this as ignored for this pull request. If you change your mind, just comment `@sonatype-lift unignore`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@zookeeper.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org