You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@zookeeper.apache.org by nk...@apache.org on 2020/10/08 18:04:00 UTC
[zookeeper] branch master updated: ZOOKEEPER-3956: Remove
json-simple from ZooKeeper
This is an automated email from the ASF dual-hosted git repository.
nkalmar pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/zookeeper.git
The following commit(s) were added to refs/heads/master by this push:
new 83d79d1 ZOOKEEPER-3956: Remove json-simple from ZooKeeper
83d79d1 is described below
commit 83d79d16d683dbf07a8a1b52ed97558530f92f89
Author: Tamas Penzes <ta...@cloudera.com>
AuthorDate: Thu Oct 8 20:03:51 2020 +0200
ZOOKEEPER-3956: Remove json-simple from ZooKeeper
JSON-Simple was only used in SnapshotFormatter in one place and also in Contrib's LogGraph component.
It's development has discontinued since 2012, so it's time to remove it.
I have replaced it with Jackson2, which we use in ZooKeeper anyway.
Added some unit tests to LogGraph as it didn't have any.
Author: Tamas Penzes <ta...@cloudera.com>
Reviewers: Enrico Olivelli <eo...@apache.org>, Norbert Kalmar <nk...@apache.org>
Closes #1484 from tamaashu/ZOOKEEPER-3956
---
pom.xml | 29 +-
zookeeper-assembly/pom.xml | 4 -
zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml | 4 -
.../{README.txt => README.md} | 46 +--
.../zookeeper-contrib-loggraph/build.xml | 70 -----
.../zookeeper-contrib-loggraph/ivy.xml | 44 ---
.../zookeeper-contrib-loggraph/pom.xml | 48 ++-
.../org/apache/zookeeper/graph/JsonGenerator.java | 325 +++++++++++----------
.../apache/zookeeper/graph/MergedLogSource.java | 280 ++++++++----------
.../zookeeper/graph/servlets/FileLoader.java | 48 ++-
.../org/apache/zookeeper/graph/servlets/Fs.java | 80 ++---
.../apache/zookeeper/graph/servlets/GraphData.java | 16 -
.../zookeeper/graph/servlets/JsonServlet.java | 97 +++---
.../apache/zookeeper/graph/servlets/NumEvents.java | 87 +++---
.../zookeeper/graph/servlets/Throughput.java | 173 ++++++-----
.../src/main/resources/loggraph-dev.sh | 9 +-
.../src/main/resources/loggraph.sh | 9 +-
.../zookeeper/graph/servlets/FileLoaderTest.java | 120 ++++++++
.../apache/zookeeper/graph/servlets/FsTest.java | 46 +++
.../zookeeper/graph/servlets/ThroughputTest.java | 88 ++++++
zookeeper-server/pom.xml | 5 -
.../apache/zookeeper/server/SnapshotFormatter.java | 12 +-
zookeeper-server/src/main/resources/LICENSE.txt | 4 -
.../resources/lib/json-simple-1.1.1.LICENSE.txt | 202 -------------
24 files changed, 884 insertions(+), 962 deletions(-)
diff --git a/pom.xml b/pom.xml
index 5c95183..066b886 100755
--- a/pom.xml
+++ b/pom.xml
@@ -438,7 +438,6 @@
<netty.version>4.1.50.Final</netty.version>
<jetty.version>9.4.24.v20191120</jetty.version>
<jackson.version>2.10.3</jackson.version>
- <json.version>1.1.1</json.version>
<jline.version>2.14.6</jline.version>
<snappy.version>1.1.7</snappy.version>
<kerby.version>2.0.0</kerby.version>
@@ -619,17 +618,6 @@
<version>${jackson.version}</version>
</dependency>
<dependency>
- <groupId>com.googlecode.json-simple</groupId>
- <artifactId>json-simple</artifactId>
- <version>${json.version}</version>
- <exclusions>
- <exclusion>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- </exclusion>
- </exclusions>
- </dependency>
- <dependency>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
<version>${jline.version}</version>
@@ -1080,6 +1068,23 @@
</rules>
</configuration>
</execution>
+ <execution>
+ <id>banned-json-simple</id>
+ <goals>
+ <goal>enforce</goal>
+ </goals>
+ <configuration>
+ <rules>
+ <bannedDependencies>
+ <excludes>
+ <exclude>com.googlecode.json-simple:json-simple</exclude>
+ </excludes>
+ <searchTransitive>false</searchTransitive>
+ <message>We don't use json-simple any more, so do not depend on it directly.</message>
+ </bannedDependencies>
+ </rules>
+ </configuration>
+ </execution>
</executions>
</plugin>
</plugins>
diff --git a/zookeeper-assembly/pom.xml b/zookeeper-assembly/pom.xml
index 8a2967f..1a3998d 100755
--- a/zookeeper-assembly/pom.xml
+++ b/zookeeper-assembly/pom.xml
@@ -100,10 +100,6 @@
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
- <groupId>com.googlecode.json-simple</groupId>
- <artifactId>json-simple</artifactId>
- </dependency>
- <dependency>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
</dependency>
diff --git a/zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml b/zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml
index 9a90dd7..2772b94 100755
--- a/zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml
+++ b/zookeeper-contrib/zookeeper-contrib-fatjar/pom.xml
@@ -79,10 +79,6 @@
<artifactId>jackson-databind</artifactId>
</dependency>
<dependency>
- <groupId>com.googlecode.json-simple</groupId>
- <artifactId>json-simple</artifactId>
- </dependency>
- <dependency>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
</dependency>
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/README.txt b/zookeeper-contrib/zookeeper-contrib-loggraph/README.md
similarity index 72%
rename from zookeeper-contrib/zookeeper-contrib-loggraph/README.txt
rename to zookeeper-contrib/zookeeper-contrib-loggraph/README.md
index 8ccaa1c..a15ea0f 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/README.txt
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/README.md
@@ -1,25 +1,27 @@
-LogGraph README
+# LogGraph README
-1 - About
+## 1 - About
LogGraph is an application for viewing and filtering zookeeper logs. It can handle transaction logs and message logs.
-2 - Compiling
+## 2 - Compiling
-Run "ant jar" in src/contrib/loggraph/. This will download all dependencies and compile all the loggraph code.
+Run `mvn clean install dependency:copy-dependencies` in zookeeper-contrib/zookeeper-contrib-loggraph/.
+This will download all dependencies and compile all the loggraph code.
-Once compilation has finished, you can run it the the loggraph.sh script in zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources.
+Once compilation has finished, you can run it the the `loggraph.sh` script in zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources.
This will start and embedded web server on your machine.
-Navigate to http://localhost:8182/graph/main.html
+Navigate to `http://localhost:8182/graph/main.html`
-3 - Usage
+## 3 - Usage
LogGraph presents the user with 4 views,
a) Simple log view
This view simply displays the log text. This isn't very useful without filters (see "Filtering the logs").
b) Server view
- The server view shows the interactions between the different servers in an ensemble. The X axis represents time.
- * Exceptions show up as red dots. Hovering your mouse over them will give you more details of the exception
+ The server view shows the interactions between the different servers in an ensemble. The X axis represents time.
+
+ * Exceptions show up as red dots. Hovering your mouse over them will give you more details of the exception
* The colour of the line represents the election state of the server.
- orange means LOOKING for leader
- dark green means the server is the leader
@@ -34,37 +36,45 @@ LogGraph presents the user with 4 views,
d) Stats view
There is currently only one statistics view, Transactions/minute. Suggestions for other statistic views are very welcome.
-4 - Filtering the logs
+## 4 - Filtering the logs
The logs can be filtered in 2 ways, by time and by content.
To filter by time simply move the slider to the desired start time. The time window specifies how many milliseconds after and including the start time will be displayed.
Content filtering uses a adhoc filtering language, using prefix notation. The language looks somewhat similar to lisp. A statement in the language takes the form (op arg arg ....). A statement resolves to a boolean value. Statements can be nested.
-4.1 - Filter arguments
+### 4.1 - Filter arguments
An argument can be a number, a string or a symbol. A number is any argument which starts with -, + or 0 to 9. If the number starts with 0x it is interpretted as hexidecimal. Otherwise it is interpretted as decimal. If the argument begins with a double-quote, (") it is interpretted as a string. Anything else is interpretted as a symbol.
-4.2 - Filter symbols
+### 4.2 - Filter symbols
The possible filter symbols are:
client-id : number, the session id of the client who initiated a transaction.
+
cxid : number, the cxid of a transaction
+
zxid : number, the zxid of a transaction
+
operation : string, the operation being performed, for example "setData", "createSession", "closeSession", "error", "create"
-4.3 - Filter operations
+### 4.3 - Filter operations
The possible filter operations are:
or : logical or, takes 1 or more arguments which must be other statements.
+
and : logical and, takes 1 or more arguments which must be other statements.
+
not : logical not, takes 1 argument which must be another statement.
+
xor : exclusive or, takes 1 or more arguments which must be other statements.
-= : equals, takes 1 or more arguments, which must all be equal to each other to return true.
-> : greater than, takes 1 or more arguments, to return true the 1st argument must be greater than the 2nd argument which must be greater than the 3rd argument and so on...
-< : less than, takes 1 or more arguments, to return true the 1st argument must be less than the 2nd argument which must be less than the 3rd argument and so on...
-4.3 - Filter examples
+ = : equals, takes 1 or more arguments, which must all be equal to each other to return true.
+
+> : greater than, takes 1 or more arguments, to return true the 1st argument must be greater than the 2nd argument which must be greater than the 3rd argument and so on...
+
+< : less than, takes 1 or more arguments, to return true the 1st argument must be less than the 2nd argument which must be less than the 3rd argument and so on...
+
+### 4.4 - Filter examples
Give me all the setData operations with session id 0xdeadbeef or 0xcafeb33r but not with zxid 0x12341234 ->
(and (= operation "setData") (or (= client-id 0xdeadbeef) (= client-id 0xcafeb33r)) (not (= zxid 0x12341234)))
-
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/build.xml b/zookeeper-contrib/zookeeper-contrib-loggraph/build.xml
deleted file mode 100644
index 07809ff..0000000
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/build.xml
+++ /dev/null
@@ -1,70 +0,0 @@
-<?xml version="1.0"?>
-
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<project name="loggraph" default="jar">
-
- <import file="../build-contrib.xml"/>
-
- <target name="init" depends="check-contrib,zookeeperbuildcontrib.init" unless="skip.contrib">
- <echo message="contrib: ${name}"/>
- <mkdir dir="${build.dir}"/>
- <antcall target="init-contrib"/>
- </target>
-
- <target name="compile" depends="init,ivy-retrieve,zookeeperbuildcontrib.compile" unless="skip.contrib">
- </target>
-
- <target name="setjarname">
- <property name="jarname" value="${build.dir}/zookeeper-${version}-${name}.jar"/>
- </target>
-
- <target name="jar" depends="setjarname,compile" >
- <jar destfile="${jarname}">
- <fileset file="${zk.root}/LICENSE.txt" />
- <fileset dir="${build.classes}" />
- <fileset dir="src/main/resources/webapp"/>
- <manifest>
- <attribute name="Built-By" value="${user.name}"/>
- <attribute name="Built-At" value="${build.time}"/>
- <attribute name="Built-On" value="${host.name}" />
- <attribute name="Implementation-Title" value="org.apache.zookeeper.graph"/>
- <attribute name="Implementation-Version" value="${revision}"/>
- <attribute name="Implementation-Vendor" value="The Apache Software Foundation"/>
- </manifest>
- </jar>
- </target>
-
- <target name="test">
- <echo message="No test target defined for this package" />
- </target>
-
-
- <target name="package" depends="compile, zookeeperbuildcontrib.package" unless="skip.contrib">
- <echo message="contrib: ${name}"/>
-
- <copy file="${basedir}/build.xml" todir="${dist.dir}/zookeeper-contrib/zookeeper-contrib-${name}"/>
-
- <mkdir dir="${dist.dir}/zookeeper-contrib/zookeeper-contrib-${name}/src"/>
- <copy todir="${dist.dir}/zookeeper-contrib/zookeeper-contrib-${name}/src">
- <fileset dir="${basedir}/src/main"/>
- </copy>
-
- </target>
-
-</project>
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/ivy.xml b/zookeeper-contrib/zookeeper-contrib-loggraph/ivy.xml
deleted file mode 100644
index e3a1b48..0000000
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/ivy.xml
+++ /dev/null
@@ -1,44 +0,0 @@
-<!--
- Licensed to the Apache Software Foundation (ASF) under one or more
- contributor license agreements. See the NOTICE file distributed with
- this work for additional information regarding copyright ownership.
- The ASF licenses this file to You under the Apache License, Version 2.0
- (the "License"); you may not use this file except in compliance with
- the License. You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
--->
-
-<ivy-module version="2.0"
- xmlns:e="http://ant.apache.org/ivy/extra">
-
- <info organisation="org.apache.zookeeper"
- module="${name}" revision="${version}">
- <license name="Apache 2.0"/>
- <ivyauthor name="Apache ZooKeeper" url="http://zookeeper.apache.org"/>
- <description>ZooKeeper Graphing</description>
- </info>
-
- <configurations defaultconfmapping="default">
- <conf name="default"/>
- <conf name="test"/>
- </configurations>
-
- <dependencies>
- <dependency org="org.slf4j" name="slf4j-api" rev="1.7.5"/>
- <dependency org="org.slf4j" name="slf4j-log4j12" rev="1.7.5" transitive="false"/>
-
- <!-- transitive false turns off dependency checking, log4j deps seem borked -->
- <dependency org="log4j" name="log4j" rev="1.2.17" transitive="false"/>
- <dependency org="org.eclipse.jetty" name="jetty-server" rev="9.2.18.v20160721" />
- <dependency org="org.eclipse.jetty" name="jetty-servlet" rev="9.2.18.v20160721" />
- <dependency org="com.googlecode.json-simple" name="json-simple" rev="1.1" />
- </dependencies>
-
-</ivy-module>
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/pom.xml b/zookeeper-contrib/zookeeper-contrib-loggraph/pom.xml
index bf6e673..81e12c8 100755
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/pom.xml
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/pom.xml
@@ -45,6 +45,10 @@
<version>${project.version}</version>
</dependency>
<dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-databind</artifactId>
+ </dependency>
+ <dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</dependency>
@@ -77,8 +81,19 @@
<artifactId>jetty-servlet</artifactId>
</dependency>
<dependency>
- <groupId>com.googlecode.json-simple</groupId>
- <artifactId>json-simple</artifactId>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter-api</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.jupiter</groupId>
+ <artifactId>junit-jupiter-engine</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.mockito</groupId>
+ <artifactId>mockito-core</artifactId>
+ <scope>test</scope>
</dependency>
</dependencies>
@@ -88,6 +103,35 @@
<directory>${project.basedir}/src/main/resources/webapp</directory>
</resource>
</resources>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-dependency-plugin</artifactId>
+ <configuration>
+ <outputDirectory>
+ ${project.basedir}/lib
+ </outputDirectory>
+ </configuration>
+ </plugin>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-surefire-plugin</artifactId>
+ <configuration>
+ <includes>
+ <include>**/*Test.java</include>
+ </includes>
+ <forkCount>${surefire-forkcount}</forkCount>
+ <reuseForks>false</reuseForks>
+ <argLine>-Xmx512m -Dtest.junit.threads=${surefire-forkcount} -Dzookeeper.junit.threadid=${surefire.forkNumber}</argLine>
+ <basedir>${project.basedir}</basedir>
+ <redirectTestOutputToFile>true</redirectTestOutputToFile>
+ <systemPropertyVariables>
+ <build.test.dir>${project.build.directory}/surefire</build.test.dir>
+ <zookeeper.DigestAuthenticationProvider.superDigest>super:D/InIHSb7yEEbrWz8b9l71RjZJU=</zookeeper.DigestAuthenticationProvider.superDigest>
+ </systemPropertyVariables>
+ </configuration>
+ </plugin>
+ </plugins>
</build>
</project>
\ No newline at end of file
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/JsonGenerator.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/JsonGenerator.java
index 8215833..19f919e 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/JsonGenerator.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/JsonGenerator.java
@@ -17,25 +17,22 @@
*/
package org.apache.zookeeper.graph;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
-import java.io.Writer;
-import java.io.OutputStreamWriter;
-import java.io.IOException;
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import java.util.regex.Pattern;
import java.util.regex.Matcher;
import java.util.HashSet;
import java.util.Iterator;
-import java.util.LinkedList;
-import java.util.ListIterator;
import java.util.Set;
public class JsonGenerator {
- private JSONObject root;
- private Set<Integer> servers;
+ private final ObjectMapper mapper = new ObjectMapper();
+ private final JsonNode root;
+ private final Set<Integer> servers;
private class Message {
private int from;
@@ -55,170 +52,178 @@ public class JsonGenerator {
}
};
- public JSONObject txnEntry(TransactionEntry e) {
- JSONObject event = new JSONObject();
+ public JsonNode txnEntry(TransactionEntry e) {
+ JsonNode event = mapper.createObjectNode();
- event.put("time", Long.toString(e.getTimestamp()));
- event.put("client", Long.toHexString(e.getClientId()));
- event.put("cxid", Long.toHexString(e.getCxid()));
- event.put("zxid", Long.toHexString(e.getZxid()));
- event.put("op", e.getOp());
- event.put("extra", e.getExtra());
- event.put("type", "transaction");
+ ((ObjectNode) event).put("time", Long.toString(e.getTimestamp()));
+ ((ObjectNode) event).put("client", Long.toHexString(e.getClientId()));
+ ((ObjectNode) event).put("cxid", Long.toHexString(e.getCxid()));
+ ((ObjectNode) event).put("zxid", Long.toHexString(e.getZxid()));
+ ((ObjectNode) event).put("op", e.getOp());
+ ((ObjectNode) event).put("extra", e.getExtra());
+ ((ObjectNode) event).put("type", "transaction");
- return event;
+ return event;
}
/**
Assumes entries are sorted by timestamp.
*/
public JsonGenerator(LogIterator iter) {
- servers = new HashSet<Integer>();
+ servers = new HashSet<Integer>();
- Pattern stateChangeP = Pattern.compile("- (LOOKING|FOLLOWING|LEADING)");
- Pattern newElectionP = Pattern.compile("New election. My id = (\\d+), Proposed zxid = (\\d+)");
- Pattern receivedProposalP = Pattern.compile("Notification: (\\d+) \\(n.leader\\), (\\d+) \\(n.zxid\\), (\\d+) \\(n.round\\), .+ \\(n.state\\), (\\d+) \\(n.sid\\), .+ \\(my state\\)");
- Pattern exceptionP = Pattern.compile("xception");
-
- root = new JSONObject();
- Matcher m = null;
- JSONArray events = new JSONArray();
- root.put("events", events);
-
- long starttime = Long.MAX_VALUE;
- long endtime = 0;
-
- int leader = 0;
- long curEpoch = 0;
- boolean newEpoch = false;
-
- while (iter.hasNext()) {
- LogEntry ent = iter.next();
-
- if (ent.getTimestamp() < starttime) {
- starttime = ent.getTimestamp();
- }
- if (ent.getTimestamp() > endtime) {
- endtime = ent.getTimestamp();
- }
-
- if (ent.getType() == LogEntry.Type.TXN) {
- events.add(txnEntry((TransactionEntry)ent));
- } else {
- Log4JEntry e = (Log4JEntry)ent;
- servers.add(e.getNode());
-
- if ((m = stateChangeP.matcher(e.getEntry())).find()) {
- JSONObject stateChange = new JSONObject();
- stateChange.put("type", "stateChange");
- stateChange.put("time", e.getTimestamp());
- stateChange.put("server", e.getNode());
- stateChange.put("state", m.group(1));
- events.add(stateChange);
-
- if (m.group(1).equals("LEADING")) {
- leader = e.getNode();
- }
- } else if ((m = newElectionP.matcher(e.getEntry())).find()) {
- Iterator<Integer> iterator = servers.iterator();
- long zxid = Long.valueOf(m.group(2));
- int count = (int)zxid;// & 0xFFFFFFFFL;
- int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
-
- if (leader != 0 && epoch > curEpoch) {
- JSONObject stateChange = new JSONObject();
- stateChange.put("type", "stateChange");
- stateChange.put("time", e.getTimestamp());
- stateChange.put("server", leader);
- stateChange.put("state", "INIT");
- events.add(stateChange);
- leader = 0;
- }
-
- while (iterator.hasNext()) {
- int dst = iterator.next();
- if (dst != e.getNode()) {
- JSONObject msg = new JSONObject();
- msg.put("type", "postmessage");
- msg.put("src", e.getNode());
- msg.put("dst", dst);
- msg.put("time", e.getTimestamp());
- msg.put("zxid", m.group(2));
- msg.put("count", count);
- msg.put("epoch", epoch);
-
- events.add(msg);
+ Pattern stateChangeP = Pattern.compile("- (LOOKING|FOLLOWING|LEADING)");
+ Pattern newElectionP = Pattern.compile("New election. My id = (\\d+), Proposed zxid = (\\d+)");
+ Pattern receivedProposalP = Pattern.compile("Notification: (\\d+) \\(n.leader\\), (\\d+) \\(n.zxid\\), (\\d+) \\(n.round\\), .+ \\(n.state\\), (\\d+) \\(n.sid\\), .+ \\(my state\\)");
+ Pattern exceptionP = Pattern.compile("xception");
+
+ root = mapper.createObjectNode();
+ Matcher m = null;
+ ArrayNode events = mapper.createArrayNode();
+ ((ObjectNode)root).set("events", events);
+
+ long starttime = Long.MAX_VALUE;
+ long endtime = 0;
+
+ int leader = 0;
+ long curEpoch = 0;
+
+ while (iter.hasNext()) {
+ LogEntry ent = iter.next();
+
+ if (ent.getTimestamp() < starttime) {
+ starttime = ent.getTimestamp();
+ }
+ if (ent.getTimestamp() > endtime) {
+ endtime = ent.getTimestamp();
}
- }
- } else if ((m = receivedProposalP.matcher(e.getEntry())).find()) {
- // Pattern.compile("Notification: \\d+, (\\d+), (\\d+), \\d+, [^,]*, [^,]*, (\\d+)");//, LOOKING, LOOKING, 2
- int src = Integer.valueOf(m.group(4));
- long zxid = Long.valueOf(m.group(2));
- int dst = e.getNode();
- long epoch2 = Long.valueOf(m.group(3));
-
- int count = (int)zxid;// & 0xFFFFFFFFL;
- int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
-
- if (leader != 0 && epoch > curEpoch) {
- JSONObject stateChange = new JSONObject();
- stateChange.put("type", "stateChange");
- stateChange.put("time", e.getTimestamp());
- stateChange.put("server", leader);
- stateChange.put("state", "INIT");
- events.add(stateChange);
- leader = 0;
- }
-
- if (src != dst) {
- JSONObject msg = new JSONObject();
- msg.put("type", "delivermessage");
- msg.put("src", src);
- msg.put("dst", dst);
- msg.put("time", e.getTimestamp());
- msg.put("zxid", zxid);
- msg.put("epoch", epoch);
- msg.put("count", count);
- msg.put("epoch2", epoch2);
-
- events.add(msg);
- }
- } else if ((m = exceptionP.matcher(e.getEntry())).find()) {
- JSONObject ex = new JSONObject();
- ex.put("type", "exception");
- ex.put("server", e.getNode());
- ex.put("time", e.getTimestamp());
- ex.put("text", e.getEntry());
- events.add(ex);
- }
- }
- JSONObject ex = new JSONObject();
- ex.put("type", "text");
- ex.put("time", ent.getTimestamp());
- String txt = ent.toString();
- ex.put("text", txt);
- events.add(ex);
- }
- // System.out.println("pending messages: "+pendingMessages.size());
- root.put("starttime", starttime);
- root.put("endtime", endtime);
- JSONArray serversarray = new JSONArray();
- root.put("servers", serversarray);
-
- Iterator<Integer> iterator = servers.iterator();
- while (iterator.hasNext()) {
- serversarray.add(iterator.next());
- }
+ if (ent.getType() == LogEntry.Type.TXN) {
+ events.add(txnEntry((TransactionEntry)ent));
+ }
+ else {
+ Log4JEntry e = (Log4JEntry)ent;
+ servers.add(e.getNode());
+
+ if ((m = stateChangeP.matcher(e.getEntry())).find()) {
+ JsonNode stateChange = add("stateChange", e.getTimestamp(), e.getNode(), m.group(1));
+ events.add(stateChange);
+
+ if (m.group(1).equals("LEADING")) {
+ leader = e.getNode();
+ }
+ }
+ else if ((m = newElectionP.matcher(e.getEntry())).find()) {
+ Iterator<Integer> iterator = servers.iterator();
+ long zxid = Long.valueOf(m.group(2));
+ int count = (int)zxid;// & 0xFFFFFFFFL;
+ int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+
+ if (leader != 0 && epoch > curEpoch) {
+ JsonNode stateChange = add("stateChange", e.getTimestamp(), leader, "INIT");
+ events.add(stateChange);
+
+ leader = 0;
+ }
+
+ while (iterator.hasNext()) {
+ int dst = iterator.next();
+ if (dst != e.getNode()) {
+ JsonNode msg = mapper.createObjectNode();
+ ((ObjectNode)msg).put("type", "postmessage");
+ ((ObjectNode)msg).put("src", e.getNode());
+ ((ObjectNode)msg).put("dst", dst);
+ ((ObjectNode)msg).put("time", e.getTimestamp());
+ ((ObjectNode)msg).put("zxid", m.group(2));
+ ((ObjectNode)msg).put("count", count);
+ ((ObjectNode)msg).put("epoch", epoch);
+
+ events.add(msg);
+ }
+ }
+ }
+ else if ((m = receivedProposalP.matcher(e.getEntry())).find()) {
+ // Pattern.compile("Notification: \\d+, (\\d+), (\\d+), \\d+, [^,]*, [^,]*, (\\d+)");//, LOOKING, LOOKING, 2
+ int src = Integer.valueOf(m.group(4));
+ long zxid = Long.valueOf(m.group(2));
+ int dst = e.getNode();
+ long epoch2 = Long.valueOf(m.group(3));
+
+ int count = (int)zxid;// & 0xFFFFFFFFL;
+ int epoch = (int)Long.rotateRight(zxid, 32);// >> 32;
+
+ if (leader != 0 && epoch > curEpoch) {
+ JsonNode stateChange = add("stateChange", e.getTimestamp(), leader, "INIT");
+ events.add(stateChange);
+
+ leader = 0;
+ }
+
+ if (src != dst) {
+ JsonNode msg = mapper.createObjectNode();
+ ((ObjectNode)msg).put("type", "delivermessage");
+ ((ObjectNode)msg).put("src", src);
+ ((ObjectNode)msg).put("dst", dst);
+ ((ObjectNode)msg).put("time", e.getTimestamp());
+ ((ObjectNode)msg).put("zxid", zxid);
+ ((ObjectNode)msg).put("count", count);
+ ((ObjectNode)msg).put("epoch", epoch);
+ ((ObjectNode)msg).put("epoch2", epoch2);
+
+ events.add(msg);
+ }
+ }
+ else if ((m = exceptionP.matcher(e.getEntry())).find()) {
+ JsonNode ex = mapper.createObjectNode();
+ ((ObjectNode)ex).put("type", "exception");
+ ((ObjectNode)ex).put("time", e.getTimestamp());
+ ((ObjectNode)ex).put("server", e.getNode());
+ ((ObjectNode)ex).put("text", e.getEntry());
+ events.add(ex);
+ }
+ }
+ JsonNode ex = mapper.createObjectNode();
+ ((ObjectNode)ex).put("type", "text");
+ ((ObjectNode)ex).put("time", ent.getTimestamp());
+ String txt = ent.toString();
+ ((ObjectNode)ex).put("text", txt);
+ events.add(ex);
+ }
+ // System.out.println("pending messages: "+pendingMessages.size());
+ ((ObjectNode)root).put("starttime", starttime);
+ ((ObjectNode)root).put("endtime", endtime);
+
+ ArrayNode serversarray = mapper.createArrayNode();
+ ((ObjectNode)root).set("servers", serversarray);
+
+ Iterator<Integer> iterator = servers.iterator();
+ while (iterator.hasNext()) {
+ serversarray.add(iterator.next());
+ }
}
+ private JsonNode add(String type, long timestamp, int node, String entry){
+ JsonNode stateChange = mapper.createObjectNode();
+ ((ObjectNode)stateChange).put("type", type);
+ ((ObjectNode)stateChange).put("time", timestamp);
+ ((ObjectNode)stateChange).put("server", node);
+ ((ObjectNode)stateChange).put("state", entry);
+ return stateChange;
+ }
+
public String toString() {
- return JSONValue.toJSONString(root);
+ String jsonString = null;
+ try {
+ jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(root);
+ } catch (JsonProcessingException e) {
+ jsonString = "{\"ERR\", " + e.getMessage() + "}";
+ }
+ return jsonString;
}
public static void main(String[] args) throws Exception {
- MergedLogSource src = new MergedLogSource(args);
- LogIterator iter = src.iterator();
- System.out.println(new JsonGenerator(iter));
+ MergedLogSource src = new MergedLogSource(args);
+ LogIterator iter = src.iterator();
+ System.out.println(new JsonGenerator(iter));
}
}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/MergedLogSource.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/MergedLogSource.java
index bb789d3..ba405ae 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/MergedLogSource.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/MergedLogSource.java
@@ -17,48 +17,16 @@
*/
package org.apache.zookeeper.graph;
-import java.io.ByteArrayInputStream;
-import java.io.EOFException;
-import java.io.FileInputStream;
import java.io.IOException;
-import java.text.DateFormat;
-import java.util.Date;
-import java.util.zip.Adler32;
-import java.util.zip.Checksum;
-import java.util.HashMap;
-
-import org.apache.jute.BinaryInputArchive;
-import org.apache.jute.InputArchive;
-import org.apache.jute.Record;
-import org.apache.zookeeper.server.TraceFormatter;
-import org.apache.zookeeper.server.persistence.FileHeader;
-import org.apache.zookeeper.server.persistence.FileTxnLog;
-import org.apache.zookeeper.server.util.SerializeUtils;
-import org.apache.zookeeper.txn.TxnHeader;
-
-import org.apache.zookeeper.ZooDefs.OpCode;
-
-import org.apache.zookeeper.txn.CreateSessionTxn;
-import org.apache.zookeeper.txn.CreateTxn;
-import org.apache.zookeeper.txn.DeleteTxn;
-import org.apache.zookeeper.txn.ErrorTxn;
-import org.apache.zookeeper.txn.SetACLTxn;
-import org.apache.zookeeper.txn.SetDataTxn;
-import org.apache.zookeeper.txn.TxnHeader;
-
-import java.io.Closeable;
-import java.io.FileNotFoundException;
-import java.util.Vector;
-import java.util.Iterator;
-import java.util.Collections;
-import java.util.NoSuchElementException;
+import java.util.ArrayList;
+import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class MergedLogSource implements LogSource {
private static final Logger LOG = LoggerFactory.getLogger(MergedLogSource.class);
- private Vector<LogSource> sources = null;
+ protected List<LogSource> sources = new ArrayList<>();
private long starttime = 0;
private long endtime = 0;
private long size = 0;
@@ -72,148 +40,150 @@ public class MergedLogSource implements LogSource {
public long getEndTime() { return endtime; }
private class MergedLogSourceIterator implements LogIterator {
- private LogEntry next = null;
- private long start = 0;
- private long end = 0;
- private MergedLogSource src = null;
- private LogIterator[] sources = null;
- private LogEntry[] nexts = null;
- private FilterOp filter = null;
-
- public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
- Vector<LogIterator> iters = new Vector<LogIterator>();
- for (LogSource s : src.sources) {
- if (s.overlapsRange(starttime, endtime)) {
- iters.add(s.iterator(starttime, endtime, filter));
+ private LogEntry next = null;
+ private long start = 0;
+ private long end = 0;
+ private MergedLogSource src = null;
+ private LogIterator[] sources = null;
+ private LogEntry[] nexts = null;
+ private FilterOp filter = null;
+
+ public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
+ List<LogIterator> iters = new ArrayList<>();
+ for (LogSource s : src.sources) {
+ if (s.overlapsRange(starttime, endtime)) {
+ iters.add(s.iterator(starttime, endtime, filter));
+ }
+ }
+
+ sources = new LogIterator[iters.size()];
+ sources = iters.toArray(sources);
+ nexts = new LogEntry[iters.size()];
+ for (int i = 0; i < sources.length; i++) {
+ if (sources[i].hasNext())
+ nexts[i] = sources[i].next();
+ }
+ this.filter = filter;
}
- }
-
- sources = new LogIterator[iters.size()];
- sources = iters.toArray(sources);
- nexts = new LogEntry[iters.size()];
- for (int i = 0; i < sources.length; i++) {
- if (sources[i].hasNext())
- nexts[i] = sources[i].next();
- }
- this.filter = filter;
- }
-
- public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime) throws IllegalArgumentException, FilterException {
- this(src, starttime, endtime, null);
- }
-
- public long size() throws IOException {
- long size = 0;
- for (LogIterator i : sources) {
- size += i.size();
- }
- return size;
- }
- public boolean hasNext() {
- for (LogEntry n : nexts) {
- if (n != null) return true;
- }
- return false;
- }
-
- public LogEntry next() {
- int min = -1;
- for (int i = 0; i < nexts.length; i++) {
- if (nexts[i] != null) {
- if (min == -1) {
- min = i;
- } else if (nexts[i].getTimestamp() < nexts[min].getTimestamp()) {
- min = i;
- }
+ public MergedLogSourceIterator(MergedLogSource src, long starttime, long endtime) throws IllegalArgumentException, FilterException {
+ this(src, starttime, endtime, null);
}
- }
- if (min == -1) {
- return null;
- } else {
- LogEntry e = nexts[min];
- nexts[min] = sources[min].next();
- return e;
- }
- }
- public void remove() throws UnsupportedOperationException {
- throw new UnsupportedOperationException("remove not supported for Merged logs");
- }
-
- public void close() throws IOException {
- for (LogIterator i : sources) {
- i.close();
- }
- }
+ public long size() throws IOException {
+ long size = 0;
+ for (LogIterator i : sources) {
+ size += i.size();
+ }
+ return size;
+ }
+
+ public boolean hasNext() {
+ for (LogEntry n : nexts) {
+ if (n != null) return true;
+ }
+ return false;
+ }
+
+ public LogEntry next() {
+ int min = -1;
+ for (int i = 0; i < nexts.length; i++) {
+ if (nexts[i] != null) {
+ if (min == -1) {
+ min = i;
+ } else if (nexts[i].getTimestamp() < nexts[min].getTimestamp()) {
+ min = i;
+ }
+ }
+ }
+ if (min == -1) {
+ return null;
+ } else {
+ LogEntry e = nexts[min];
+ nexts[min] = sources[min].next();
+ return e;
+ }
+ }
+
+ public void remove() throws UnsupportedOperationException {
+ throw new UnsupportedOperationException("remove not supported for Merged logs");
+ }
+
+ public void close() throws IOException {
+ for (LogIterator i : sources) {
+ i.close();
+ }
+ }
}
public LogIterator iterator(long starttime, long endtime) throws IllegalArgumentException {
- try {
- return iterator(starttime, endtime, null);
- } catch (FilterException fe) {
- assert(false); // shouldn't happen without filter
- return null;
- }
+ try {
+ return iterator(starttime, endtime, null);
+ }
+ catch (FilterException fe) {
+ assert(false); // shouldn't happen without filter
+ return null;
+ }
}
public LogIterator iterator(long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
- // sanitise start and end times
- if (endtime < starttime) {
- throw new IllegalArgumentException("End time (" + endtime + ") must be greater or equal to starttime (" + starttime + ")");
- }
+ // sanitise start and end times
+ if (endtime < starttime) {
+ throw new IllegalArgumentException("End time (" + endtime + ") must be greater or equal to starttime (" + starttime + ")");
+ }
- return new MergedLogSourceIterator(this, starttime, endtime, filter);
- }
+ return new MergedLogSourceIterator(this, starttime, endtime, filter);
+ }
- public LogIterator iterator() throws IllegalArgumentException {
- return iterator(starttime, endtime+1);
- }
-
- public MergedLogSource(String[] files) throws IOException {
- sources = new Vector<LogSource>();
- for (String f : files) {
- addSource(f);
+ public LogIterator iterator() throws IllegalArgumentException {
+ return iterator(starttime, endtime+1);
}
+
+ public MergedLogSource(String[] files) throws IOException {
+ sources.clear();
+ for (String f : files) {
+ addSource(f);
+ }
}
public void addSource(String f) throws IOException {
- LogSource s = null;
- if (TxnLogSource.isTransactionFile(f)) {
- s = new TxnLogSource(f);
- } else {
- s = new Log4JSource(f);
- }
+ LogSource s = null;
+ if (TxnLogSource.isTransactionFile(f)) {
+ s = new TxnLogSource(f);
+ }
+ else {
+ s = new Log4JSource(f);
+ }
- size += s.size();
- endtime = s.getEndTime() > endtime ? s.getEndTime() : endtime;
- starttime = s.getStartTime() < starttime || starttime == 0 ? s.getStartTime() : starttime;
- sources.add(s);
+ size += s.size();
+ endtime = s.getEndTime() > endtime ? s.getEndTime() : endtime;
+ starttime = s.getStartTime() < starttime || starttime == 0 ? s.getStartTime() : starttime;
+ sources.add(s);
}
public String toString() {
- String s = "MergedLogSource(size=" + size + ", start=" + starttime + ", end=" + endtime +")";
- for (LogSource src : sources) {
- s += "\n\t- " +src;
- }
- return s;
+ String s = "MergedLogSource(size=" + size + ", start=" + starttime + ", end=" + endtime +")";
+ for (LogSource src : sources) {
+ s += "\n\t- " +src;
+ }
+ return s;
}
public static void main(String[] args) throws IOException {
- System.out.println("Time: " + System.currentTimeMillis());
- MergedLogSource s = new MergedLogSource(args);
- System.out.println(s);
-
- LogIterator iter;
-
- iter = s.iterator();
- System.out.println("Time: " + System.currentTimeMillis());
- System.out.println("Iterator Size: " + iter.size());
- System.out.println("Time: " + System.currentTimeMillis());
- /* while (iter.hasNext()) {
- System.out.println(iter.next());
- }*/
- iter.close();
- System.out.println("Time: " + System.currentTimeMillis());
+ System.out.println("Time: " + System.currentTimeMillis());
+ MergedLogSource s = new MergedLogSource(args);
+ System.out.println(s);
+
+ LogIterator iter;
+
+ iter = s.iterator();
+ System.out.println("Time: " + System.currentTimeMillis());
+ System.out.println("Iterator Size: " + iter.size());
+ System.out.println("Time: " + System.currentTimeMillis());
+ /* while (iter.hasNext()) {
+ System.out.println(iter.next());
+ }*/
+ iter.close();
+ System.out.println("Time: " + System.currentTimeMillis());
}
}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/FileLoader.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/FileLoader.java
index 67e8945..8717239 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/FileLoader.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/FileLoader.java
@@ -17,18 +17,10 @@
*/
package org.apache.zookeeper.graph.servlets;
-import java.io.File;
-import java.io.IOException;
-import java.io.FileNotFoundException;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.zookeeper.graph.*;
@@ -37,24 +29,22 @@ public class FileLoader extends JsonServlet
private MergedLogSource source = null;
public FileLoader(MergedLogSource src) throws Exception {
- source = src;
+ source = src;
}
- String handleRequest(JsonRequest request) throws Exception
- {
- String output = "";
-
- String file = request.getString("path", "/");
- JSONObject o = new JSONObject();
- try {
- this.source.addSource(file);
- o.put("status", "OK");
-
- } catch (Exception e) {
- o.put("status", "ERR");
- o.put("error", e.toString());
- }
-
- return JSONValue.toJSONString(o);
+ String handleRequest(JsonRequest request) throws Exception {
+ String file = request.getString("path", "/");
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode rootNode = mapper.createObjectNode();
+ try {
+ this.source.addSource(file);
+ ((ObjectNode) rootNode).put("status", "OK");
+ }
+ catch (Exception e) {
+ ((ObjectNode) rootNode).put("status", "ERR");
+ ((ObjectNode) rootNode).put("error", e.toString());
+ }
+ String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(rootNode);
+ return jsonString;
}
}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Fs.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Fs.java
index e5b1a01..1e0139e 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Fs.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Fs.java
@@ -18,52 +18,56 @@
package org.apache.zookeeper.graph.servlets;
import java.io.File;
-import java.io.IOException;
import java.io.FileNotFoundException;
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
+import java.io.IOException;
import java.util.Arrays;
import java.util.Comparator;
public class Fs extends JsonServlet
{
- String handleRequest(JsonRequest request) throws Exception
- {
- String output = "";
- JSONArray filelist = new JSONArray();
-
- File base = new File(request.getString("path", "/"));
- if (!base.exists() || !base.isDirectory()) {
- throw new FileNotFoundException("Couldn't find [" + request + "]");
- }
- File[] files = base.listFiles();
- Arrays.sort(files, new Comparator<File>() {
- public int compare(File o1, File o2) {
- if (o1.isDirectory() != o2.isDirectory()) {
- if (o1.isDirectory()) {
- return -1;
- } else {
- return 1;
+ String handleRequest(JsonRequest request) throws Exception {
+ File base = new File(request.getString("path", "/"));
+ if (!base.exists() || !base.isDirectory()) {
+ throw new FileNotFoundException("Couldn't find [" + request + "]");
+ }
+ File[] files = base.listFiles();
+ Arrays.sort(files, new Comparator<File>() {
+ public int compare(File o1, File o2) {
+ if (o1.isDirectory() != o2.isDirectory()) {
+ if (o1.isDirectory()) {
+ return -1;
+ } else {
+ return 1;
+ }
+ }
+ return o1.getName().compareToIgnoreCase(o2.getName());
}
- }
- return o1.getName().compareToIgnoreCase(o2.getName());
- }
- });
-
- for (File f : files) {
- JSONObject o = new JSONObject();
- o.put("file", f.getName());
- o.put("type", f.isDirectory() ? "D" : "F");
- o.put("path", f.getCanonicalPath());
- filelist.add(o);
- }
- return JSONValue.toJSONString(filelist);
+ });
+
+ String jsonString = generateJSON(files);
+ return jsonString;
}
+
+ protected static String generateJSON(File[] files) throws IOException {
+ ObjectMapper mapper = new ObjectMapper();
+ ArrayNode fileList = mapper.createArrayNode();
+
+ for (File f : files) {
+ JsonNode node = mapper.createObjectNode().objectNode();
+ ((ObjectNode) node).put("file", f.getName());
+ ((ObjectNode) node).put("type", f.isDirectory() ? "D" : "F");
+ ((ObjectNode) node).put("path", f.getCanonicalPath());
+ fileList.add(node);
+ }
+
+ String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(fileList);
+ return jsonString;
+ }
}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/GraphData.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/GraphData.java
index fc10eb1..7fb3135 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/GraphData.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/GraphData.java
@@ -17,22 +17,6 @@
*/
package org.apache.zookeeper.graph.servlets;
-import java.io.File;
-import java.io.IOException;
-import java.io.FileNotFoundException;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import java.util.regex.Pattern;
-import java.util.regex.Matcher;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
import org.apache.zookeeper.graph.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/JsonServlet.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/JsonServlet.java
index 910d44f..4b3d63c 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/JsonServlet.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/JsonServlet.java
@@ -17,69 +17,78 @@
*/
package org.apache.zookeeper.graph.servlets;
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import java.io.IOException;
-
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
import java.util.Map;
abstract public class JsonServlet extends HttpServlet {
abstract String handleRequest(JsonRequest request) throws Exception;
protected class JsonRequest {
- private Map map;
+ private Map map;
+
+ public JsonRequest(ServletRequest request) {
+ map = request.getParameterMap();
+ }
+
+ public long getNumber(String name, long defaultnum) {
+ String[] vals = (String[])map.get(name);
+ if (vals == null || vals.length == 0) {
+ return defaultnum;
+ }
- public JsonRequest(ServletRequest request) {
- map = request.getParameterMap();
- }
-
- public long getNumber(String name, long defaultnum) {
- String[] vals = (String[])map.get(name);
- if (vals == null || vals.length == 0) {
- return defaultnum;
- }
+ try {
+ return Long.valueOf(vals[0]);
+ }
+ catch (NumberFormatException e) {
+ return defaultnum;
+ }
+ }
- try {
- return Long.valueOf(vals[0]);
- } catch (NumberFormatException e) {
- return defaultnum;
- }
- }
-
- public String getString(String name, String defaultstr) {
- String[] vals = (String[])map.get(name);
- if (vals == null || vals.length == 0) {
- return defaultstr;
- } else {
- return vals[0];
- }
- }
+ public String getString(String name, String defaultstr) {
+ String[] vals = (String[])map.get(name);
+ if (vals == null || vals.length == 0) {
+ return defaultstr;
+ }
+ else {
+ return vals[0];
+ }
+ }
}
- protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException
- {
+ protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
response.setContentType("text/plain;charset=utf-8");
response.setStatus(HttpServletResponse.SC_OK);
-
- try {
- String req = request.getRequestURI().substring(request.getServletPath().length());
- response.getWriter().println(handleRequest(new JsonRequest(request)));
- } catch (Exception e) {
- JSONObject o = new JSONObject();
- o.put("error", e.toString());
- response.getWriter().println(JSONValue.toJSONString(o));
- } catch (java.lang.OutOfMemoryError oom) {
- JSONObject o = new JSONObject();
- o.put("error", "Out of memory. Perhaps you've requested too many logs. Try narrowing you're filter criteria.");
- response.getWriter().println(JSONValue.toJSONString(o));
- }
+ try {
+ String req = request.getRequestURI().substring(request.getServletPath().length());
+
+ response.getWriter().println(handleRequest(new JsonRequest(request)));
+ }
+ catch (Exception e) {
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode rootNode = mapper.createObjectNode();
+ ((ObjectNode) rootNode).put("error", e.toString());
+ String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(rootNode);
+
+ response.getWriter().println(jsonString);
+ }
+ catch (java.lang.OutOfMemoryError oom) {
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode rootNode = mapper.createObjectNode();
+ ((ObjectNode) rootNode).put("error", "Out of memory. Perhaps you've requested too many logs. Try narrowing you're filter criteria.");
+ String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(rootNode);
+
+ response.getWriter().println(jsonString);
+ }
}
}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/NumEvents.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/NumEvents.java
index 5961a12..564f11a 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/NumEvents.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/NumEvents.java
@@ -17,27 +17,14 @@
*/
package org.apache.zookeeper.graph.servlets;
-import java.io.File;
-import java.io.IOException;
-import java.io.FileNotFoundException;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
-import java.util.regex.Pattern;
-import java.util.regex.Matcher;
-
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.zookeeper.graph.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-
public class NumEvents extends JsonServlet
{
private static final Logger LOG = LoggerFactory.getLogger(NumEvents.class);
@@ -46,43 +33,47 @@ public class NumEvents extends JsonServlet
private LogSource source = null;
public NumEvents(LogSource src) throws Exception {
- this.source = src;
+ this.source = src;
}
String handleRequest(JsonRequest request) throws Exception {
- String output = "";
+ String output = "";
+
+ long starttime = 0;
+ long endtime = 0;
+ long period = 0;
+
+ starttime = request.getNumber("start", 0);
+ endtime = request.getNumber("end", 0);
+ period = request.getNumber("period", 0);
+
+ if (starttime == 0) { starttime = source.getStartTime(); }
+ if (endtime == 0) {
+ if (period > 0) {
+ endtime = starttime + period;
+ }
+ else {
+ endtime = source.getEndTime();
+ }
+ }
+
+ long size = 0;
+ LogIterator iter = source.iterator(starttime, endtime);
+ size = iter.size();
- long starttime = 0;
- long endtime = 0;
- long period = 0;
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode data = mapper.createObjectNode();
+ ((ObjectNode) data).put("startTime", starttime);
+ ((ObjectNode) data).put("endTime", endtime);
+ ((ObjectNode) data).put("numEntries", iter.size());
- starttime = request.getNumber("start", 0);
- endtime = request.getNumber("end", 0);
- period = request.getNumber("period", 0);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("handle(start= " + starttime + ", end=" + endtime + ", numEntries=" + size +")");
+ }
+ iter.close();
- if (starttime == 0) { starttime = source.getStartTime(); }
- if (endtime == 0) {
- if (period > 0) {
- endtime = starttime + period;
- } else {
- endtime = source.getEndTime();
- }
- }
-
- LogIterator iter = source.iterator(starttime, endtime);
- JSONObject data = new JSONObject();
- data.put("startTime", starttime);
- data.put("endTime", endtime);
- long size = 0;
-
- size = iter.size();
-
- data.put("numEntries", size);
- if (LOG.isDebugEnabled()) {
- LOG.debug("handle(start= " + starttime + ", end=" + endtime + ", numEntries=" + size +")");
- }
- iter.close();
- return JSONValue.toJSONString(data);
+ String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(data);
+ return jsonString;
}
}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Throughput.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Throughput.java
index 80ed1dc..022c68e 100644
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Throughput.java
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/java/org/apache/zookeeper/graph/servlets/Throughput.java
@@ -18,23 +18,14 @@
package org.apache.zookeeper.graph.servlets;
import java.io.IOException;
-import java.io.BufferedOutputStream;
-import java.io.FileOutputStream;
-import java.io.DataOutputStream;
-import java.io.PrintStream;
-
import java.util.HashSet;
-import java.util.LinkedHashMap;
import java.util.Set;
-
+import com.fasterxml.jackson.core.util.MinimalPrettyPrinter;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.zookeeper.graph.*;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import org.json.simple.JSONArray;
-import org.json.simple.JSONObject;
-import org.json.simple.JSONValue;
-
public class Throughput extends JsonServlet
{
@@ -45,82 +36,88 @@ public class Throughput extends JsonServlet
private LogSource source = null;
public Throughput(LogSource src) throws Exception {
- this.source = src;
+ this.source = src;
}
public String handleRequest(JsonRequest request) throws Exception {
- long starttime = 0;
- long endtime = 0;
- long period = 0;
- long scale = 0;
-
- starttime = request.getNumber("start", 0);
- endtime = request.getNumber("end", 0);
- period = request.getNumber("period", 0);
-
-
- if (starttime == 0) { starttime = source.getStartTime(); }
- if (endtime == 0) {
- if (period > 0) {
- endtime = starttime + period;
- } else {
- endtime = source.getEndTime();
- }
+ long startTime = 0;
+ long endTime = 0;
+ long period = 0;
+ long scale = 0;
+
+ startTime = request.getNumber("start", 0);
+ endTime = request.getNumber("end", 0);
+ period = request.getNumber("period", 0);
+
+
+ if (startTime == 0) { startTime = source.getStartTime(); }
+ if (endTime == 0) {
+ if (period > 0) {
+ endTime = startTime + period;
+ } else {
+ endTime = source.getEndTime();
+ }
+ }
+
+ String scalestr = request.getString("scale", "minutes");
+ if (scalestr.equals("seconds")) {
+ scale = MS_PER_SEC;
+ } else if (scalestr.equals("hours")) {
+ scale = MS_PER_HOUR;
+ } else {
+ scale = MS_PER_MIN;
+ }
+
+ LogIterator iter = source.iterator(startTime, endTime);
+ String jsonString = getJSON(iter, scale);
+ iter.close();
+ return jsonString;
}
-
- String scalestr = request.getString("scale", "minutes");
- if (scalestr.equals("seconds")) {
- scale = MS_PER_SEC;
- } else if (scalestr.equals("hours")) {
- scale = MS_PER_HOUR;
- } else {
- scale = MS_PER_MIN;
- }
-
- LogIterator iter = source.iterator(starttime, endtime);
-
- long current = 0;
- long currentms = 0;
- Set<Long> zxids_ms = new HashSet<Long>();
- long zxidcount = 0;
-
- JSONArray events = new JSONArray();
- while (iter.hasNext()) {
- LogEntry e = iter.next();
- if (e.getType() != LogEntry.Type.TXN) {
- continue;
- }
-
- TransactionEntry cxn = (TransactionEntry)e;
-
- long ms = cxn.getTimestamp();
- long inscale = ms/scale;
-
- if (currentms != ms && currentms != 0) {
- zxidcount += zxids_ms.size();
- zxids_ms.clear();
- }
-
- if (inscale != current && current != 0) {
- JSONObject o = new JSONObject();
- o.put("time", current*scale);
- o.put("count", zxidcount);
- events.add(o);
- zxidcount = 0;
- }
- current = inscale;
- currentms = ms;
-
- zxids_ms.add(cxn.getZxid());
- }
- JSONObject o = new JSONObject();
- o.put("time", current*scale);
- o.put("count", zxidcount);
- events.add(o);
-
- iter.close();
-
- return JSONValue.toJSONString(events);
- }
-};
+ protected String getJSON(final LogIterator iter, final long scale) throws IOException {
+ long current = 0;
+ long currentms = 0;
+ Set<Long> zxids_ms = new HashSet<Long>();
+ long zxidCount = 0;
+
+ ObjectMapper mapper = new ObjectMapper();
+ ArrayNode events = mapper.createArrayNode();
+
+ while (iter.hasNext()) {
+ LogEntry e = iter.next();
+ if (e.getType() != LogEntry.Type.TXN) {
+ continue;
+ }
+
+ TransactionEntry cxn = (TransactionEntry)e;
+
+ long ms = cxn.getTimestamp();
+ long inscale = ms/ scale;
+
+ if (currentms != ms && currentms != 0) {
+ zxidCount += zxids_ms.size();
+ zxids_ms.clear();
+ }
+
+ if (inscale != current && current != 0) {
+ JsonNode node = mapper.createObjectNode();
+ ((ObjectNode) node).put("time", current * scale);
+ ((ObjectNode) node).put("count", zxidCount);
+ events.add(node);
+ zxidCount = 0;
+ }
+ current = inscale;
+ currentms = ms;
+
+ zxids_ms.add(cxn.getZxid());
+ }
+
+ JsonNode node = mapper.createObjectNode();
+ ((ObjectNode) node).put("time", current * scale);
+ ((ObjectNode) node).put("count", zxidCount);
+ events.add(node);
+
+ String jsonString = mapper.writer(new MinimalPrettyPrinter()).writeValueAsString(events);
+ return jsonString;
+ }
+}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph-dev.sh b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph-dev.sh
index e04434e..616ac66 100755
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph-dev.sh
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph-dev.sh
@@ -20,10 +20,9 @@ make_canonical () {
}
SCRIPTDIR=`dirname $0`
-BUILDDIR=`make_canonical $SCRIPTDIR/../../../../../build/contrib/loggraph`
-LIBDIR=`make_canonical $BUILDDIR/lib`
+BUILDDIR=`make_canonical $SCRIPTDIR/../../../target/`
+LIBDIR=`make_canonical $BUILDDIR/../lib`
WEBDIR=`make_canonical $SCRIPTDIR/../web`
-ZKDIR=`make_canonical $SCRIPTDIR/../../../../../build/`
if [ ! -x $BUILDDIR ]; then
echo "\n\n*** You need to build loggraph before running it ***\n\n";
@@ -34,10 +33,6 @@ for i in `ls $LIBDIR`; do
CLASSPATH=$LIBDIR/$i:$CLASSPATH
done
-for i in $ZKDIR/zookeeper-*.jar; do
- CLASSPATH="$i:$CLASSPATH"
-done
-
CLASSPATH=$BUILDDIR/classes:$WEBDIR:$CLASSPATH
echo $CLASSPATH
java -Dlog4j.configuration=org/apache/zookeeper/graph/log4j.properties -Xdebug -Xrunjdwp:transport=dt_socket,address=4444,server=y,suspend=n -cp $CLASSPATH org.apache.zookeeper.graph.LogServer $*
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph.sh b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph.sh
index 0259dc6..9b32743 100755
--- a/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph.sh
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/main/resources/loggraph.sh
@@ -20,9 +20,8 @@ make_canonical () {
}
SCRIPTDIR=`dirname $0`
-BUILDDIR=`make_canonical $SCRIPTDIR/../../../../../build/contrib/loggraph`
-LIBDIR=`make_canonical $BUILDDIR/lib`
-ZKDIR=`make_canonical $SCRIPTDIR/../../../../../build/`
+BUILDDIR=`make_canonical $SCRIPTDIR/../../../target/`
+LIBDIR=`make_canonical $BUILDDIR/../lib`
if [ ! -x $BUILDDIR ]; then
echo "\n\n*** You need to build loggraph before running it ***\n\n";
@@ -37,10 +36,6 @@ for i in `ls $BUILDDIR/*.jar`; do
CLASSPATH=$i:$CLASSPATH
done
-for i in $ZKDIR/zookeeper-*.jar; do
- CLASSPATH="$i:$CLASSPATH"
-done
-
java -cp $CLASSPATH org.apache.zookeeper.graph.LogServer $*
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FileLoaderTest.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FileLoaderTest.java
new file mode 100644
index 0000000..0eb688c
--- /dev/null
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FileLoaderTest.java
@@ -0,0 +1,120 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.graph.servlets;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+import org.apache.zookeeper.graph.FilterException;
+import org.apache.zookeeper.graph.FilterOp;
+import org.apache.zookeeper.graph.LogIterator;
+import org.apache.zookeeper.graph.LogSource;
+import org.apache.zookeeper.graph.MergedLogSource;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.Assertions;
+import java.io.IOException;
+import java.util.List;
+
+public class FileLoaderTest {
+
+ @Test
+ public void testHandleRequestOK() throws Exception {
+ String[] files = {""};
+ MyMergedLogSource mls = new MyMergedLogSource(files);
+ final JsonServlet.JsonRequest jsonRequest = mock(JsonServlet.JsonRequest.class);
+ when(jsonRequest.getString("path", "/")).thenReturn("/tmp");
+ FileLoader fl = new FileLoader(mls);
+ String s = fl.handleRequest(jsonRequest);
+ Assertions.assertEquals("{\"status\":\"OK\"}", s);
+ Assertions.assertTrue(mls.getSources().contains(new MySource("/tmp")));
+ Assertions.assertFalse(mls.getSources().contains(new MySource("/tmp2")));
+ }
+
+ @Test
+ public void testHandleRequestERR() throws Exception {
+ String[] files = {""};
+ MyMergedLogSource mls = new MyMergedLogSource(files);
+ final JsonServlet.JsonRequest jsonRequest = mock(JsonServlet.JsonRequest.class);
+ when(jsonRequest.getString("path", "/")).thenReturn("/tmp3");
+ FileLoader fl = new FileLoader(mls);
+ String s = fl.handleRequest(jsonRequest);
+ Assertions.assertEquals("{\"status\":\"ERR\",\"error\":\"java.io.IOException: Message\"}", s);
+ Assertions.assertFalse(mls.getSources().contains(new MySource("/tmp")));
+ Assertions.assertFalse(mls.getSources().contains(new MySource("/tmp2")));
+ }
+
+ private class MyMergedLogSource extends MergedLogSource {
+ public MyMergedLogSource(String[] files) throws IOException {
+ super(files);
+ }
+ public void addSource(String f) throws IOException {
+ if ("/tmp3".equals(f)) throw new IOException("Message");
+ sources.add(new MySource(f));
+ }
+ public List<LogSource> getSources(){
+ return sources;
+ }
+ }
+
+ private class MySource implements LogSource{
+ private String file = null;
+ public MySource(String file) throws IOException {
+ this.file=file;
+ }
+
+ public boolean equals(Object o){
+ if(!(o instanceof MySource)) return false;
+ if(((MySource)o).file == null) return this.file==null;
+ return ((MySource)o).file.equals(this.file);
+ }
+
+ @Override
+ public LogIterator iterator(long starttime, long endtime, FilterOp filter) throws IllegalArgumentException, FilterException {
+ return null;
+ }
+
+ @Override
+ public LogIterator iterator(long starttime, long endtime) throws IllegalArgumentException {
+ return null;
+ }
+
+ @Override
+ public LogIterator iterator() throws IllegalArgumentException {
+ return null;
+ }
+
+ @Override
+ public boolean overlapsRange(long starttime, long endtime) {
+ return false;
+ }
+
+ @Override
+ public long size() {
+ return 0;
+ }
+
+ @Override
+ public long getStartTime() {
+ return 0;
+ }
+
+ @Override
+ public long getEndTime() {
+ return 0;
+ }
+ }
+}
\ No newline at end of file
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FsTest.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FsTest.java
new file mode 100644
index 0000000..007c291
--- /dev/null
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/FsTest.java
@@ -0,0 +1,46 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.graph.servlets;
+
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import java.io.File;
+import java.io.IOException;
+
+public class FsTest {
+ @Test
+ public void testGenerateJSON() throws IOException {
+ File[] files = new File[2];
+ final File file1 = mock(File.class);
+ when(file1.getName()).thenReturn("testDir");
+ when(file1.isDirectory()).thenReturn(true);
+ when(file1.getCanonicalPath()).thenReturn("/tmp/testDir");
+ final File file2 = mock(File.class);
+ when(file2.getName()).thenReturn("test");
+ when(file2.isDirectory()).thenReturn(false);
+ when(file2.getCanonicalPath()).thenReturn("/tmp/test");
+ files[0]=file1;
+ files[1]=file2;
+ String output = Fs.generateJSON(files);
+ String expectedOutput = "[{\"file\":\"testDir\",\"type\":\"D\",\"path\":\"/tmp/testDir\"}," +
+ "{\"file\":\"test\",\"type\":\"F\",\"path\":\"/tmp/test\"}]";
+ Assertions.assertEquals(expectedOutput, output);
+ }
+}
diff --git a/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/ThroughputTest.java b/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/ThroughputTest.java
new file mode 100644
index 0000000..f496868
--- /dev/null
+++ b/zookeeper-contrib/zookeeper-contrib-loggraph/src/test/java/org/apache/zookeeper/graph/servlets/ThroughputTest.java
@@ -0,0 +1,88 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.zookeeper.graph.servlets;
+
+import static org.mockito.Mockito.mock;
+import org.apache.zookeeper.graph.FilterException;
+import org.apache.zookeeper.graph.Log4JEntry;
+import org.apache.zookeeper.graph.Log4JSource;
+import org.apache.zookeeper.graph.LogEntry;
+import org.apache.zookeeper.graph.LogIterator;
+import org.apache.zookeeper.graph.TransactionEntry;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.NoSuchElementException;
+
+public class ThroughputTest {
+
+ @Test
+ public void testGetJSON() throws Exception {
+ long scale = 1;
+ Log4JSource source = mock(Log4JSource.class);
+ Throughput tp = new Throughput(source);
+ LogIterator iter = new MyIterator();
+ String jsonString = tp.getJSON(iter, scale);
+ String expected = "[{\"time\":3,\"count\":1},{\"time\":4,\"count\":1},{\"time\":5,\"count\":0}]";
+ Assertions.assertEquals(expected, jsonString);
+ }
+
+ private class MyIterator implements LogIterator {
+ int index = 0;
+ List<LogEntry> list = new ArrayList<>();
+
+ public MyIterator() throws IllegalArgumentException, FilterException {
+ for(int i=1; i<3; i++){
+ long timestamp = i;
+ int node = i;
+ String entry = Integer.toString(i);
+ Log4JEntry le = new Log4JEntry(timestamp, node, entry);
+ list.add(le);
+ }
+ for(int i=3; i<7; i++){
+ long timestamp = i;
+ long clientId = i;
+ long Cxid = i;
+ long Zxid = i;
+ String op = Integer.toString(i);
+ TransactionEntry te = new TransactionEntry(timestamp, clientId, Cxid, Zxid, op);
+ list.add(te);
+ }
+ }
+
+ synchronized public long size() throws IOException {
+ return list.size();
+ }
+
+ public boolean hasNext() {
+ return index<list.size()-1;
+ }
+
+ public LogEntry next() throws NoSuchElementException {
+ return list.get(index++);
+ }
+
+ public void remove() throws UnsupportedOperationException {
+ throw new UnsupportedOperationException("remove not supported for L4J logs");
+ }
+
+ public void close(){}
+ }
+}
diff --git a/zookeeper-server/pom.xml b/zookeeper-server/pom.xml
index 5dbc028..296083f 100755
--- a/zookeeper-server/pom.xml
+++ b/zookeeper-server/pom.xml
@@ -94,11 +94,6 @@
<scope>provided</scope>
</dependency>
<dependency>
- <groupId>com.googlecode.json-simple</groupId>
- <artifactId>json-simple</artifactId>
- <scope>provided</scope>
- </dependency>
- <dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcprov-jdk15on</artifactId>
<scope>test</scope>
diff --git a/zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java b/zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
index 5fce206..0e5ec86 100644
--- a/zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
+++ b/zookeeper-server/src/main/java/org/apache/zookeeper/server/SnapshotFormatter.java
@@ -19,6 +19,7 @@
package org.apache.zookeeper.server;
import static org.apache.zookeeper.server.persistence.FileSnap.SNAPSHOT_FILE_PREFIX;
+import com.fasterxml.jackson.core.io.JsonStringEncoder;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
@@ -36,7 +37,6 @@ import org.apache.zookeeper.server.persistence.FileSnap;
import org.apache.zookeeper.server.persistence.SnapStream;
import org.apache.zookeeper.server.persistence.Util;
import org.apache.zookeeper.util.ServiceUtils;
-import org.json.simple.JSONValue;
/**
* Dump a snapshot file to stdout.
@@ -181,14 +181,16 @@ public class SnapshotFormatter {
}
private void printSnapshotJson(final DataTree dataTree) {
+ JsonStringEncoder encoder = JsonStringEncoder.getInstance();
System.out.printf(
"[1,0,{\"progname\":\"SnapshotFormatter.java\",\"progver\":\"0.01\",\"timestamp\":%d}",
System.currentTimeMillis());
- printZnodeJson(dataTree, "/");
+ printZnodeJson(dataTree, "/", encoder);
System.out.print("]");
}
- private void printZnodeJson(final DataTree dataTree, final String fullPath) {
+ private void printZnodeJson(final DataTree dataTree, final String fullPath, JsonStringEncoder encoder) {
+
final DataNode n = dataTree.getNode(fullPath);
@@ -209,7 +211,7 @@ public class SnapshotFormatter {
}
StringBuilder nodeSB = new StringBuilder();
nodeSB.append("{");
- nodeSB.append("\"name\":\"").append(JSONValue.escape(name)).append("\"").append(",");
+ nodeSB.append("\"name\":\"").append(encoder.quoteAsString(name)).append("\"").append(",");
nodeSB.append("\"asize\":").append(dataLen).append(",");
nodeSB.append("\"dsize\":").append(dataLen).append(",");
nodeSB.append("\"dev\":").append(0).append(",");
@@ -223,7 +225,7 @@ public class SnapshotFormatter {
if (children != null && children.size() > 0) {
System.out.print("[" + nodeSB);
for (String child : children) {
- printZnodeJson(dataTree, fullPath + (fullPath.equals("/") ? "" : "/") + child);
+ printZnodeJson(dataTree, fullPath + (fullPath.equals("/") ? "" : "/") + child, encoder);
}
System.out.print("]");
} else {
diff --git a/zookeeper-server/src/main/resources/LICENSE.txt b/zookeeper-server/src/main/resources/LICENSE.txt
index b1264c0..efa786f 100644
--- a/zookeeper-server/src/main/resources/LICENSE.txt
+++ b/zookeeper-server/src/main/resources/LICENSE.txt
@@ -214,10 +214,6 @@ This distribution bundles SLF4J 1.7.5, which is available under the MIT
License. For details, see a copy of the license in
lib/slf4j-1.7.5.LICENSE.txt
-This distribution bundles json-simple v1.1.1, which is available under the
-Apache Software License, Version 2.0. For details, see a copy of the license in
-lib/json-simple-1.1.1.LICENSE.txt
-
This distribution bundles a modified version of 'JZLib' as part of
Netty-3.7.0, which is available under the 3-clause BSD licence. For
details, see a copy of the licence in META-INF/license/LICENSE-jzlib.txt
diff --git a/zookeeper-server/src/main/resources/lib/json-simple-1.1.1.LICENSE.txt b/zookeeper-server/src/main/resources/lib/json-simple-1.1.1.LICENSE.txt
deleted file mode 100644
index 6279e52..0000000
--- a/zookeeper-server/src/main/resources/lib/json-simple-1.1.1.LICENSE.txt
+++ /dev/null
@@ -1,202 +0,0 @@
-
- Apache License
- Version 2.0, January 2004
- http://www.apache.org/licenses/
-
- TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
-
- 1. Definitions.
-
- "License" shall mean the terms and conditions for use, reproduction,
- and distribution as defined by Sections 1 through 9 of this document.
-
- "Licensor" shall mean the copyright owner or entity authorized by
- the copyright owner that is granting the License.
-
- "Legal Entity" shall mean the union of the acting entity and all
- other entities that control, are controlled by, or are under common
- control with that entity. For the purposes of this definition,
- "control" means (i) the power, direct or indirect, to cause the
- direction or management of such entity, whether by contract or
- otherwise, or (ii) ownership of fifty percent (50%) or more of the
- outstanding shares, or (iii) beneficial ownership of such entity.
-
- "You" (or "Your") shall mean an individual or Legal Entity
- exercising permissions granted by this License.
-
- "Source" form shall mean the preferred form for making modifications,
- including but not limited to software source code, documentation
- source, and configuration files.
-
- "Object" form shall mean any form resulting from mechanical
- transformation or translation of a Source form, including but
- not limited to compiled object code, generated documentation,
- and conversions to other media types.
-
- "Work" shall mean the work of authorship, whether in Source or
- Object form, made available under the License, as indicated by a
- copyright notice that is included in or attached to the work
- (an example is provided in the Appendix below).
-
- "Derivative Works" shall mean any work, whether in Source or Object
- form, that is based on (or derived from) the Work and for which the
- editorial revisions, annotations, elaborations, or other modifications
- represent, as a whole, an original work of authorship. For the purposes
- of this License, Derivative Works shall not include works that remain
- separable from, or merely link (or bind by name) to the interfaces of,
- the Work and Derivative Works thereof.
-
- "Contribution" shall mean any work of authorship, including
- the original version of the Work and any modifications or additions
- to that Work or Derivative Works thereof, that is intentionally
- submitted to Licensor for inclusion in the Work by the copyright owner
- or by an individual or Legal Entity authorized to submit on behalf of
- the copyright owner. For the purposes of this definition, "submitted"
- means any form of electronic, verbal, or written communication sent
- to the Licensor or its representatives, including but not limited to
- communication on electronic mailing lists, source code control systems,
- and issue tracking systems that are managed by, or on behalf of, the
- Licensor for the purpose of discussing and improving the Work, but
- excluding communication that is conspicuously marked or otherwise
- designated in writing by the copyright owner as "Not a Contribution."
-
- "Contributor" shall mean Licensor and any individual or Legal Entity
- on behalf of whom a Contribution has been received by Licensor and
- subsequently incorporated within the Work.
-
- 2. Grant of Copyright License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- copyright license to reproduce, prepare Derivative Works of,
- publicly display, publicly perform, sublicense, and distribute the
- Work and such Derivative Works in Source or Object form.
-
- 3. Grant of Patent License. Subject to the terms and conditions of
- this License, each Contributor hereby grants to You a perpetual,
- worldwide, non-exclusive, no-charge, royalty-free, irrevocable
- (except as stated in this section) patent license to make, have made,
- use, offer to sell, sell, import, and otherwise transfer the Work,
- where such license applies only to those patent claims licensable
- by such Contributor that are necessarily infringed by their
- Contribution(s) alone or by combination of their Contribution(s)
- with the Work to which such Contribution(s) was submitted. If You
- institute patent litigation against any entity (including a
- cross-claim or counterclaim in a lawsuit) alleging that the Work
- or a Contribution incorporated within the Work constitutes direct
- or contributory patent infringement, then any patent licenses
- granted to You under this License for that Work shall terminate
- as of the date such litigation is filed.
-
- 4. Redistribution. You may reproduce and distribute copies of the
- Work or Derivative Works thereof in any medium, with or without
- modifications, and in Source or Object form, provided that You
- meet the following conditions:
-
- (a) You must give any other recipients of the Work or
- Derivative Works a copy of this License; and
-
- (b) You must cause any modified files to carry prominent notices
- stating that You changed the files; and
-
- (c) You must retain, in the Source form of any Derivative Works
- that You distribute, all copyright, patent, trademark, and
- attribution notices from the Source form of the Work,
- excluding those notices that do not pertain to any part of
- the Derivative Works; and
-
- (d) If the Work includes a "NOTICE" text file as part of its
- distribution, then any Derivative Works that You distribute must
- include a readable copy of the attribution notices contained
- within such NOTICE file, excluding those notices that do not
- pertain to any part of the Derivative Works, in at least one
- of the following places: within a NOTICE text file distributed
- as part of the Derivative Works; within the Source form or
- documentation, if provided along with the Derivative Works; or,
- within a display generated by the Derivative Works, if and
- wherever such third-party notices normally appear. The contents
- of the NOTICE file are for informational purposes only and
- do not modify the License. You may add Your own attribution
- notices within Derivative Works that You distribute, alongside
- or as an addendum to the NOTICE text from the Work, provided
- that such additional attribution notices cannot be construed
- as modifying the License.
-
- You may add Your own copyright statement to Your modifications and
- may provide additional or different license terms and conditions
- for use, reproduction, or distribution of Your modifications, or
- for any such Derivative Works as a whole, provided Your use,
- reproduction, and distribution of the Work otherwise complies with
- the conditions stated in this License.
-
- 5. Submission of Contributions. Unless You explicitly state otherwise,
- any Contribution intentionally submitted for inclusion in the Work
- by You to the Licensor shall be under the terms and conditions of
- this License, without any additional terms or conditions.
- Notwithstanding the above, nothing herein shall supersede or modify
- the terms of any separate license agreement you may have executed
- with Licensor regarding such Contributions.
-
- 6. Trademarks. This License does not grant permission to use the trade
- names, trademarks, service marks, or product names of the Licensor,
- except as required for reasonable and customary use in describing the
- origin of the Work and reproducing the content of the NOTICE file.
-
- 7. Disclaimer of Warranty. Unless required by applicable law or
- agreed to in writing, Licensor provides the Work (and each
- Contributor provides its Contributions) on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
- implied, including, without limitation, any warranties or conditions
- of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
- PARTICULAR PURPOSE. You are solely responsible for determining the
- appropriateness of using or redistributing the Work and assume any
- risks associated with Your exercise of permissions under this License.
-
- 8. Limitation of Liability. In no event and under no legal theory,
- whether in tort (including negligence), contract, or otherwise,
- unless required by applicable law (such as deliberate and grossly
- negligent acts) or agreed to in writing, shall any Contributor be
- liable to You for damages, including any direct, indirect, special,
- incidental, or consequential damages of any character arising as a
- result of this License or out of the use or inability to use the
- Work (including but not limited to damages for loss of goodwill,
- work stoppage, computer failure or malfunction, or any and all
- other commercial damages or losses), even if such Contributor
- has been advised of the possibility of such damages.
-
- 9. Accepting Warranty or Additional Liability. While redistributing
- the Work or Derivative Works thereof, You may choose to offer,
- and charge a fee for, acceptance of support, warranty, indemnity,
- or other liability obligations and/or rights consistent with this
- License. However, in accepting such obligations, You may act only
- on Your own behalf and on Your sole responsibility, not on behalf
- of any other Contributor, and only if You agree to indemnify,
- defend, and hold each Contributor harmless for any liability
- incurred by, or claims asserted against, such Contributor by reason
- of your accepting any such warranty or additional liability.
-
- END OF TERMS AND CONDITIONS
-
- APPENDIX: How to apply the Apache License to your work.
-
- To apply the Apache License to your work, attach the following
- boilerplate notice, with the fields enclosed by brackets "[]"
- replaced with your own identifying information. (Don't include
- the brackets!) The text should be enclosed in the appropriate
- comment syntax for the file format. We also recommend that a
- file or class name and description of purpose be included on the
- same "printed page" as the copyright notice for easier
- identification within third-party archives.
-
- Copyright 1999-2005 The Apache Software Foundation
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.