You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@accumulo.apache.org by lstav <gi...@git.apache.org> on 2017/04/05 17:10:59 UTC
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
GitHub user lstav opened a pull request:
https://github.com/apache/accumulo/pull/242
ACCUMULO-2181/3005 REST API and new Monitor UI
Based on the original code from @joshelser, added a REST API to the Monitor to access all the information from the current Monitor, modernized the Monitor with a new UI using Bootstrap, moved all server functionality to the client (such as sorting tables), and other improvements.
Most of the code to serve the REST server and resources server was done by @ctubbsii.
Some of the back end stuff still need some polishing such as the JS and CSS includes which are using CDNs, which @ctubbsii will look into adding it to the packaging process, and Maven related nuances (@ctubbsii will also help with that).
New features/old features that have been modernized include:
1. A new notification functionality for the navbar. The user will see a summary of all logs/problems as well as level of logs from the navbar. The user will also see a summary of the server status in the navbar (Green, Yellow, Red).
2. The Instance name is now part of the navbar and redirects to the "Overview page".
3. Instance information has been moved to an "About" modal accessible from the navbar.
4. New Namespace Filter (improved from earlier 2.0.0) using the Select2 library.
5. New calls to delete logs and problems without redirecting the page.
6. New Auto-Refresh implementation that uses Ajax to refresh the information.
Any feedback on the code and the new UI will be appreciated.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/lstav/accumulo ACCUMULO-MONITOR
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/accumulo/pull/242.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #242
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110466768
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+
+ /*
+ * Add the tables that have the selected namespace Asterisk = All namespaces Hyphen = Default namespace
+ */
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
+ tableNamespace.addTable(new TableNamespace(key));
+ }
+ }
+
+ return generateTables(tableNamespace);
+ }
+
+ /**
+ * Generates a table list based on the list of namespaces
+ *
+ * @param tableNamespace
+ * Namespace list
+ * @return Table list
+ */
+ private TablesList generateTables(TablesList tableNamespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add tables to the list
+ for (Entry<String,String> entry : Tables.getNameToIdMap(HdfsZooInstance.getInstance()).entrySet()) {
+ String tableName = entry.getKey(), tableId = entry.getValue();
+ TableInfo tableInfo = tableStats.get(tableName);
+ if (null != tableInfo) {
+ Double holdTime = compactingByTable.get(tableId);
+ if (holdTime == null)
+ holdTime = Double.valueOf(0.);
+
+ for (TableNamespace name : tableNamespace.tables) {
+ // Check if table has the default namespace
+ if (!tableName.contains(".") && name.namespace.equals("")) {
+ name.addTable(new TableInformation(tableName, tableId, tableInfo, holdTime, tableManager.getTableState(tableId).name()));
+ } else if (tableName.startsWith(name.namespace + ".")) {
+ name.addTable(new TableInformation(tableName, tableId, tableInfo, holdTime, tableManager.getTableState(tableId).name()));
+ }
+ }
+ tables.add(new TableInformation(tableName, tableId, tableInfo, holdTime, tableManager.getTableState(tableId).name()));
+ } else {
+ for (TableNamespace name : tableNamespace.tables) {
+ if (!tableName.contains(".") && name.namespace.equals("")) {
+ name.addTable(new TableInformation(tableName, tableId, tableManager.getTableState(tableId).name()));
+ } else if (tableName.startsWith(name.namespace + ".")) {
+ name.addTable(new TableInformation(tableName, tableId, tableManager.getTableState(tableId).name()));
+ }
+ }
+ tables.add(new TableInformation(tableName, tableId, tableManager.getTableState(tableId).name()));
+ }
+ }
+
+ return tableNamespace;
+ }
+
+ /**
+ * Generates a list of all the tables
+ *
+ * @return list with all tables
+ */
+ @GET
+ public TablesList getTables() {
+ return generateTables("*");
+ }
+
+ /**
+ * Generates a list with the selected namespace
+ *
+ * @param namespace
+ * Namespace to filter tables
+ * @return list with selected tables
+ */
+ @GET
+ @Path("namespace/{namespace}")
+ public TablesList getTable(@PathParam("namespace") String namespace) {
+ return generateTables(namespace);
+ }
+
+ /**
+ * Generates a list with the list of namespaces
+ *
+ * @param namespaceList
+ * List of namespaces separated by a comma
+ * @return list with selected tables
+ */
+ @GET
+ @Path("namespaces/{namespaces}")
+ public TablesList getTableWithNamespace(@PathParam("namespaces") String namespaceList) {
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ /*
+ * Add the tables that have the selected namespace Asterisk = All namespaces Hyphen = Default namespace
+ */
+ for (String namespace : namespaceList.split(",")) {
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
+ tableNamespace.addTable(new TableNamespace(key));
+ }
+ }
+ }
+
+ return generateTables(tableNamespace);
+ }
+
+ /**
+ * Generates a list of participating tservers for a table
+ *
+ * @param tableId
+ * Table ID to find participating tservers
+ * @return List of participating tservers
+ */
+ @Path("{tableId}")
+ @GET
+ public TabletServers getParticipatingTabletServers(@PathParam("tableId") String tableId) throws Exception {
+ Instance instance = Monitor.getContext().getInstance();
+
+ TreeSet<String> locs = new TreeSet<>();
+ if (RootTable.ID.equals(tableId)) {
+ locs.add(instance.getRootTabletLocation());
+ } else {
+ String systemTableName = MetadataTable.ID.equals(tableId) ? RootTable.NAME : MetadataTable.NAME;
+ MetaDataTableScanner scanner = new MetaDataTableScanner(Monitor.getContext(), new Range(KeyExtent.getMetadataEntry(tableId, new Text()),
+ KeyExtent.getMetadataEntry(tableId, null)), systemTableName);
+
+ while (scanner.hasNext()) {
+ TabletLocationState state = scanner.next();
+ if (state.current != null) {
+ try {
+ locs.add(state.current.hostPort());
+ } catch (Exception ex) {}
--- End diff --
If you don't have any exception handling on the server, it would be better to at least throw exception back to the client. It is ugly when it happens but at least it won't be hidden.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110454857
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --
The variables from the DedupedLogEvent are different than the ones I have on LogEvents (which I need to make the JSON for the REST call).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
>> I changed to use the public variables at the suggestion of @ctubbsii, but we could discuss the best approach and change it to that.
>>
> Ok. Mostly I was thinking that if we are moving these towards being publicly consumable, it would be better if we encapsulate the state and just expose it via methods (that gives us more flexibility).
>
I understand, I'll discuss with him the reason he suggested otherwise (could have been the initial implementation).
>>I tested on both FF and Chrome, I couldn't get the proxy to work for IE so I haven't been able to test it.
>>
>Ok, this would be good to keep in mind before merging. IE still has a bit of prevalence in "the enterprise". We should perform some diligence before merge to make sure that we aren't shipping something entirely busted on the major IE versions.
>
OK, I'll try to test it on IE, I believe that most of the libraries I used can be used since IE8.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110050871
--- Diff: server/monitor/src/main/resources/templates/overview.ftl ---
@@ -0,0 +1,90 @@
+<#--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <div><h3>${title}</h3></div>
+ <br>
+ <div class="center-block">
+ <table class="overview-table">
--- End diff --
Nice suggestion! I'll look into it tomorrow and see if I can implement it with no major problems.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110162027
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
+
+ <link rel="stylesheet" type="text/css" href="/resources/screen.css" media="screen" />
+ <script>
+ /**
+ * Sets up autorefresh on initial load
+ */
+ $(document).ready(function() {
+ setupAutoRefresh();
+ });
+ </script>
+ <#if js??>
--- End diff --
It checks if the JavaScript file name is included in the template, since at least one of the pages (vis.ftl) require the JavaScript file to be included after the rest of the template.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110470413
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/ServerShuttingDownInformation.java ---
@@ -14,25 +14,32 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-package org.apache.accumulo.monitor.util.celltypes;
+package org.apache.accumulo.monitor.rest.tservers;
-import java.io.Serializable;
-import java.util.Comparator;
+import javax.xml.bind.annotation.XmlAttribute;
-public abstract class CellType<T> implements Comparator<T>, Serializable {
-
- private static final long serialVersionUID = 1L;
- private boolean sortable = true;
-
- abstract public String alignment();
+/**
+ *
+ * Generates a tserver shutting down
+ *
+ * @since 2.0.0
+ *
+ */
+public class ServerShuttingDownInformation {
--- End diff --
+1 Much better name
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111509157
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
No problem (I should have started with it I think :P)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110242657
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java ---
@@ -118,19 +98,17 @@ public void start() {
}
}
- public void stop() {
+ private void stop() {
--- End diff --
I was trying to clean up the embedded web server, and simplify its abstraction around Jetty. We never call stop outside the method. I marked it private to verify and left it that way. This is an internal class, so there's no reason it should be public if it doesn't need to be. I would like to do further refactoring and simplification of the Java portions of the monitor service... but that can happen in a future task.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111158861
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/zk/ZKInformation.java ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.zk;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ *
+ * Generates a list of Zookeeper server information
+ *
+ * @since 2.0.0
+ *
+ */
+public class ZKInformation {
--- End diff --
I have the class to keep consistency for all the calls. We could replace pretty much everything with Lists (or Maps of Lists), but it can be confusing.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110476591
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+ // Variable names become JSON keys
+ private TabletStats total, historical;
+
+ /**
+ * Generates tserver summary
+ *
+ * @return tserver summary
+ */
+ @GET
+ public TabletServers getTserverSummary() {
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ tserverInfo.addTablet(new TabletServer(status));
+ }
+
+ tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+ return tserverInfo;
+ }
+
+ /**
+ * REST call to clear dead servers from list
+ *
+ * @param server
+ * Dead server to clear
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ public void clearDeadServer(@QueryParam("server") String server) throws Exception {
+ DeadServerList obit = new DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + Constants.ZDEADTSERVERS);
+ obit.delete(server);
+ }
+
+ /**
+ * Generates a recovery tserver list
+ *
+ * @return Recovery tserver list
+ */
+ @Path("recovery")
+ @GET
+ public Map<String,List<Map<String,String>>> getTserverRecovery() {
--- End diff --
Yes, it has to return a maps of maps (in this case), since the first map uses the Key for JSON (this is another way of writing what I did with classes, I might change this to have another class, but I used this instead since it was faster to code (but can get confusing)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110050764
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
+
+ <link rel="stylesheet" type="text/css" href="/resources/screen.css" media="screen" />
+ <script>
+ /**
+ * Sets up autorefresh on initial load
+ */
+ $(document).ready(function() {
+ setupAutoRefresh();
+ });
+ </script>
+ <#if js??>
+ <script src="/resources/${js}"></script>
+ </#if>
+ <script src="/resources/sidebar.js"></script>
+ </head>
+
+ <body>
+ <#include "/templates/modals.ftl">
+ <div id="content-wrapper">
+ <div id="content">
+ <div id="sidebar" class="navbar navbar-inverse navbar-fixed-top">
+ <!--<#include "/templates/header.ftl">-->
+ <#include "/templates/sidebar.ftl">
--- End diff --
Thanks, initially it was a sidebar that turned into a navbar and I forgot to rename it. Yes, the comment and the file can be removed. Will do this tomorrow.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110254199
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+ // Variable names become JSON keys
+ private TabletStats total, historical;
+
+ /**
+ * Generates tserver summary
+ *
+ * @return tserver summary
+ */
+ @GET
+ public TabletServers getTserverSummary() {
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ tserverInfo.addTablet(new TabletServer(status));
+ }
+
+ tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+ return tserverInfo;
+ }
+
+ /**
+ * REST call to clear dead servers from list
+ *
+ * @param server
+ * Dead server to clear
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ public void clearDeadServer(@QueryParam("server") String server) throws Exception {
+ DeadServerList obit = new DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + Constants.ZDEADTSERVERS);
+ obit.delete(server);
+ }
+
+ /**
+ * Generates a recovery tserver list
+ *
+ * @return Recovery tserver list
+ */
+ @Path("recovery")
+ @GET
+ public Map<String,List<Map<String,String>>> getTserverRecovery() {
+
+ Map<String,List<Map<String,String>>> jsonObj = new HashMap<String,List<Map<String,String>>>();
+ List<Map<String,String>> recoveryList = new ArrayList<>();
+ Map<String,String> recoveryObj = new HashMap<String,String>();
+
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ for (TabletServerStatus server : mmi.tServerInfo) {
+ if (server.logSorts != null) {
+ for (RecoveryStatus recovery : server.logSorts) {
+ recoveryObj.put("server", AddressUtil.parseAddress(server.name, false).getHostText());
+ recoveryObj.put("log", recovery.name);
+ recoveryObj.put("time", Long.toString(recovery.runtime));
+ recoveryObj.put("copySort", Double.toString(recovery.progress));
+
+ recoveryList.add(recoveryObj);
+ }
+ }
+ }
+
+ jsonObj.put("recoveryList", recoveryList);
+
+ return jsonObj;
+ }
+
+ /**
+ * Generates details for the selected tserver
+ *
+ * @param tserverAddr
+ * TServer name
+ * @return TServer details
+ */
+ @Path("{address}")
+ @GET
+ public TabletServerSummary getTserverDetails(@PathParam("address") String tserverAddr) throws Exception {
+
+ String tserverAddress = tserverAddr;
+
+ boolean tserverExists = false;
+ if (tserverAddress != null && tserverAddress.isEmpty() == false) {
+ for (TabletServerStatus ts : Monitor.getMmi().getTServerInfo()) {
+ if (tserverAddress.equals(ts.getName())) {
+ tserverExists = true;
+ break;
+ }
+ }
+ }
+
+ if (tserverAddress == null || tserverAddress.isEmpty() || tserverExists == false) {
+
+ return null;
+ }
+
+ double totalElapsedForAll = 0;
+ double splitStdDev = 0;
+ double minorStdDev = 0;
+ double minorQueueStdDev = 0;
+ double majorStdDev = 0;
+ double majorQueueStdDev = 0;
+ double currentMinorAvg = 0;
+ double currentMajorAvg = 0;
+ double currentMinorStdDev = 0;
+ double currentMajorStdDev = 0;
+ total = new TabletStats(null, new ActionStats(), new ActionStats(), new ActionStats(), 0, 0, 0, 0);
+ HostAndPort address = HostAndPort.fromString(tserverAddress);
+ historical = new TabletStats(null, new ActionStats(), new ActionStats(), new ActionStats(), 0, 0, 0, 0);
+ List<TabletStats> tsStats = new ArrayList<>();
+
+ try {
+ ClientContext context = Monitor.getContext();
+ TabletClientService.Client client = ThriftUtil.getClient(new TabletClientService.Client.Factory(), address, context);
+ try {
+ for (String tableId : Monitor.getMmi().tableMap.keySet()) {
+ tsStats.addAll(client.getTabletStats(Tracer.traceInfo(), context.rpcCreds(), tableId));
+ }
+ historical = client.getHistoricalStats(Tracer.traceInfo(), context.rpcCreds());
+ } finally {
+ ThriftUtil.returnClient(client);
+ }
+ } catch (Exception e) {
+ return null;
+ }
+
+ List<CurrentOperations> currentOps = doCurrentOperations(tsStats);
+
+ if (total.minors.num != 0)
+ currentMinorAvg = (long) (total.minors.elapsed / total.minors.num);
+ if (total.minors.elapsed != 0 && total.minors.num != 0)
+ currentMinorStdDev = stddev(total.minors.elapsed, total.minors.num, total.minors.sumDev);
+ if (total.majors.num != 0)
+ currentMajorAvg = total.majors.elapsed / total.majors.num;
+ if (total.majors.elapsed != 0 && total.majors.num != 0 && total.majors.elapsed > total.majors.num)
+ currentMajorStdDev = stddev(total.majors.elapsed, total.majors.num, total.majors.sumDev);
+
+ ActionStatsUpdator.update(total.minors, historical.minors);
+ ActionStatsUpdator.update(total.majors, historical.majors);
+ totalElapsedForAll += total.majors.elapsed + historical.splits.elapsed + total.minors.elapsed;
+
+ minorStdDev = stddev(total.minors.elapsed, total.minors.num, total.minors.sumDev);
+ minorQueueStdDev = stddev(total.minors.queueTime, total.minors.num, total.minors.queueSumDev);
+ majorStdDev = stddev(total.majors.elapsed, total.majors.num, total.majors.sumDev);
+ majorQueueStdDev = stddev(total.majors.queueTime, total.majors.num, total.majors.queueSumDev);
+ splitStdDev = stddev(historical.splits.num, historical.splits.elapsed, historical.splits.sumDev);
+
+ TabletServerDetailInformation details = doDetails(address, tsStats.size());
+
+ List<AllTimeTabletResults> allTime = doAllTimeResults(majorQueueStdDev, minorQueueStdDev, totalElapsedForAll, splitStdDev, majorStdDev, minorStdDev);
+
+ CurrentTabletResults currentRes = doCurrentTabletResults(currentMinorAvg, currentMinorStdDev, currentMajorAvg, currentMajorStdDev);
+
+ TabletServerSummary tserverDetails = new TabletServerSummary(details, allTime, currentRes, currentOps);
+
+ return tserverDetails;
+ }
+
+ private static final int concurrentScans = Monitor.getContext().getConfiguration().getCount(Property.TSERV_READ_AHEAD_MAXCONCURRENT);
--- End diff --
Ah, yes this is static presently. Ignore...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110077224
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
+
+ <link rel="stylesheet" type="text/css" href="/resources/screen.css" media="screen" />
+ <script>
+ /**
+ * Sets up autorefresh on initial load
+ */
+ $(document).ready(function() {
+ setupAutoRefresh();
+ });
+ </script>
+ <#if js??>
--- End diff --
what's this do?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110253445
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
+ }
+
+ // Obtains the hours, if time is 0, return hours, minutes, and units
+ hr = time % 24;
+ time = Math.floor(time / 24);
+ if (time == 0) {
+ return hr + 'h' + ' ' + min + 'm';
+ }
+
+ // Obtains the days, if time is 0, return days, hours, and units
+ day = time % 365;
+ time = Math.floor(time / 365);
+ if (time == 0) {
+ return day + 'd' + ' ' + hr + 'h';
+ }
+
+ // Obtains the years, if time is 0, return years, days, and units
+ yr = Math.floor(time);
+ return yr + 'y' + ' ' + day + 'd';
+}
+
+/**
+ * Sorts the selected table by column in the direction chosen
+ *
+ * @param {string} tableID Table to sort
+ * @param {string} direction Direction to sort table, asc or desc
+ * @param {number} n Column to sort
+ */
+function sortTables(tableID, direction, n) {
+ var table, rows, switching, i, x, y, h, shouldSwitch, dir, xFinal, yFinal;
+ table = document.getElementById(tableID);
+ switching = true;
+
+ dir = direction;
+ sessionStorage.direction = dir;
+
+ // Select the rows of the table
+ rows = table.getElementsByTagName('tr');
+
+ // Clears the sortable class from the table columns
+ var count = 0;
+ while (rows[0].getElementsByTagName('th').length > count) {
+ var tmpH = rows[0].getElementsByTagName('th')[count];
+ tmpH.classList.remove('sortable');
+ if (rows.length > 2) {
+ tmpH.classList.add('sortable');
+ }
+ $(tmpH.getElementsByTagName('span')).remove();
+ count += 1;
+ }
+
+ // If there are more than 2 rows, add arrow to the selected column
+ if (rows.length <= 2) {
+ switching = false;
+ } else {
+ h = rows[0].getElementsByTagName('th')[n];
+ if (dir == 'asc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-up"' +
+ ' width="10px" height="10px" />');
+ } else if (dir == 'desc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-down"' +
+ ' width="10px" height="10px" />');
+ }
+ }
+
+ /*
+ * Make a loop that will continue until
+ * no switching has been done:
+ */
+ while (switching) {
+ switching = false;
+ rows = table.getElementsByTagName('tr');
+
+ /*
+ * Loop through all table rows (except the
+ * first, which contains table headers):
+ */
+ for (i = 1; i < (rows.length - 1); i++) {
+ shouldSwitch = false;
+ /*
+ * Get two elements to compare,
+ * one from current row and one from the next:
+ * If the element is a dash, convert to null, otherwise,
+ * if it is a string, convert to number
+ */
+ x = rows[i].getElementsByTagName('td')[n].getAttribute('data-value');
+ xFinal = (x === '-' || x === '—' ?
+ null : (Number(x) == x ? Number(x) : x));
+
+ y = rows[i + 1].getElementsByTagName('td')[n].getAttribute('data-value');
+ yFinal = (y === '-' || y === '—' ?
+ null : (Number(y) == y ? Number(y) : y));
+
+ /*
+ * Check if the two rows should switch place,
+ * based on the direction, asc or desc:
+ */
+ if (dir == 'asc') {
+ if (xFinal > yFinal || (xFinal !== null && yFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ } else if (dir == 'desc') {
+ if (xFinal < yFinal || (yFinal !== null && xFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ }
+ }
+ if (shouldSwitch) {
+ /*
+ * If a switch has been marked, make the switch
+ * and mark that a switch has been done:
+ */
+ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
+ switching = true;
+ }
+ }
+}
+
+/**
+ * Clears the selected table while leaving the headers
+ *
+ * @param {string} tableID Table to clear
+ */
+function clearTable(tableID) {
+ // JQuery selector to select all rows except for the first row (header)
+ $('#' + tableID).find('tr:not(:first)').remove();
+}
+
+///// REST Calls /////////////
+
+/**
+ * REST GET call for the master information,
+ * stores it on a sessionStorage variable
+ */
+function getMaster() {
+ $.getJSON('/rest/master', function(data) {
+ sessionStorage.master = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the zookeeper information,
+ * stores it on a sessionStorage variable
+ */
+function getZK() {
+ $.getJSON('/rest/zk', function(data) {
+ sessionStorage.zk = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the namespaces, stores it on a global variable
+ */
+function getNamespaces() {
+ $.getJSON('/rest/tables/namespaces', function(data) {
+ NAMESPACES = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tables on each namespace,
+ * stores it on a sessionStorage variable
+ *
+ * @param {array} namespaces Array holding the selected namespaces
+ */
+function getNamespaceTables(namespaces) {
+
+ // Creates a JSON object to store the tables
+ var jsonObj = {};
+ jsonObj.tables = [];
+
+ /* If the namespace array include *, get all tables, otherwise,
+ * get tables from specific namespaces
+ */
+ if (namespaces.indexOf('*') != -1) {
+ getTables();
+ } else {
+ $.each(namespaces, function(key, val) {
+ /* Makes the rest call for each of the namespaces in the array,
+ * stores them on the JSON object
+ */
+ if (val !== '*') {
+ var call = '/rest/tables/namespace/' + val;
+ $.getJSON(call, function(data) {
+ $.each(data.tables, function(key2, val2) {
+ jsonObj.tables.push(val2);
+ });
+ });
+ }
+ });
+ sessionStorage.tables = JSON.stringify(jsonObj);
+ }
+}
+
+/**
+ * REST GET call for the tables, stores it on a sessionStorage variable
+ */
+function getTables() {
+ $.getJSON('/rest/tables', function(data) {
+ sessionStorage.tables = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear a specific dead server
+ *
+ * @param {string} server Dead Server ID
+ */
+function clearDeadServers(server) {
+ var call = '/rest/tservers?server=' + server;
+ $.post(call);
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ */
+function getTServers() {
+ $.getJSON('/rest/tservers', function(data) {
+ sessionStorage.tservers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ *
+ * @param {string} server Server ID
+ */
+function getTServer(server) {
+ var call = '/rest/tservers/' + server;
+ $.getJSON(call, function(data) {
+ sessionStorage.server = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the scans, stores it on a sessionStorage variable
+ */
+function getScans() {
+ $.getJSON('/rest/scans', function(data) {
+ sessionStorage.scans = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the bulk imports, stores it on a sessionStorage variable
+ */
+function getBulkImports() {
+ $.getJSON('/rest/bulkImports', function(data) {
+ sessionStorage.bulkImports = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the garbage collector,
+ * stores it on a sessionStorage variable
+ */
+function getGarbageCollector() {
+ $.getJSON('/rest/gc', function(data) {
+ sessionStorage.gc = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the server stats, stores it on a sessionStorage variable
+ */
+function getServerStats() {
+ $.getJSON('/rest/tservers/serverStats', function(data) {
+ sessionStorage.serverStats = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the recovery list, stores it on a sessionStorage variable
+ */
+function getRecoveryList() {
+ $.getJSON('/rest/tservers/recovery', function(data) {
+ sessionStorage.recoveryList = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the participating tablet servers,
+ * stores it on a sessionStorage variable
+ *
+ * @param {string} table Table ID
+ */
+function getTableServers(table) {
+ var call = '/rest/tables/' + table;
+ $.getJSON(call, function(data) {
+ sessionStorage.tableServers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace summary, stores it on a sessionStorage variable
+ *
+ * @param {string} minutes Number of minutes to display trace summary
+ */
+function getTraceSummary(minutes) {
+ var call = '/rest/trace/summary/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceSummary = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace type, stores it on a sessionStorage variable
+ *
+ * @param {string} type Type of the trace
+ * @param {string} minutes Number of minutes to display trace
+ */
+function getTraceOfType(type, minutes) {
+ var call = '/rest/trace/listType/' + type + '/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceType = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace id, stores it on a sessionStorage variable
+ *
+ * @param {string} id Trace ID
+ */
+function getTraceShow(id) {
+ var call = '/rest/trace/show/' + id;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceShow = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the logs, stores it on a sessionStorage variable
+ */
+function getLogs() {
+ $.getJSON('/rest/logs', function(data) {
+ sessionStorage.logs = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear logs
+ */
+function clearLogs() {
+ $.post('/rest/logs');
+}
+
+/**
+ * REST GET call for the problems
+ */
+function getProblems() {
+ getProblemSummary();
+ getProblemDetails();
+}
+
+/**
+ * REST POST call to clear all table problems
+ *
+ * @param {string} tableID Table ID
+ */
+function clearTableProblems(tableID) {
+ var call = '/rest/problems/summary?s=' + tableID;
+ // Change plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
+ $.post(call);
+}
+
+/**
+ * REST POST call to clear detail problems
+ *
+ * @param {string} table Table ID
+ * @param {string} resource Resource for problem
+ * @param {string} type Type of problem
+ */
+function clearDetailsProblems(table, resource, type) {
+ var call = '/rest/problems/details?table=' + table + '&resource=' +
+ resource + '&ptype=' + type;
+ // Changes plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
--- End diff --
Also, this substitution is to sanitize the table id, not the URL, could sanitize the params individually before concatenation into the URL.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110997560
--- Diff: server/monitor/src/main/resources/templates/overview.ftl ---
@@ -0,0 +1,90 @@
+<#--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <div><h3>${title}</h3></div>
+ <br>
+ <div class="center-block">
+ <table class="overview-table">
--- End diff --
Really simple change! Worked great! I'll be pushing the changes soon.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075373
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogResource.java ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.server.monitor.DedupedLogEvent;
+import org.apache.accumulo.server.monitor.LogService;
+import org.apache.log4j.spi.LoggingEvent;
+
+/**
+ *
+ * Responsible for generating a new log JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/logs")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class LogResource {
+
+ /**
+ * Generates log event list as a JSON object
+ *
+ * @return log event array
+ */
+ @GET
+ public List<LogEvent> getRecentLogs() {
+ List<DedupedLogEvent> dedupedLogEvents = LogService.getInstance().getEvents();
+ ArrayList<LogEvent> logEvents = new ArrayList<>(dedupedLogEvents.size());
+
+ final StringBuilder msg = new StringBuilder(64);
+ for (DedupedLogEvent dev : dedupedLogEvents) {
+ msg.setLength(0);
+ final LoggingEvent ev = dev.getEvent();
+ Object application = ev.getMDC("application");
+ if (application == null)
+ application = "";
+
+ msg.append(ev.getMessage().toString());
+ if (ev.getThrowableStrRep() != null)
+ for (String line : ev.getThrowableStrRep())
+ msg.append("\n\t").append(line);
+
+ // Add a new log event to the list
+ logEvents.add(new LogEvent(ev.getTimeStamp(), application, dev.getCount(), ev.getLevel().toString(), msg.toString().trim()));
--- End diff --
the `trim()` seems unnecessary at a glance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075995
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
--- End diff --
Make the asterisk a static-final constant if it has special meaning.
What special meaning does the hyphen have? (comment)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110193783
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
@lstav this is one example that I was referring to earlier (bare XmlAttribute). Could you make a pass through the rest of the POJOs to make sure that we only have the attribute set as necessary, please?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110253880
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
--- End diff --
The licenses for these are okay (flot hasn't changed, and select2 is also MIT). We can continue bundling these, but it'd be nice to add configuration to be able to override with updated versions in a CDN.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> The active monitor registers itself as the log receiver in ZK
> its expecting a standby monitor
Reminder: there are two advertisements for the monitor in ZooKeeper. One is the HTTP address and the other is the log-forwarding address. The former is what users can access to get the hostname to put in their browser, the latter is how the TabletServers know where to send their log messages (that match the configured level).
The test is verifying that the HTTP address is being advertised and the HTTP server is listening for requests before the ZK lock is acquired. I wouldn't know why this functionality has changed. It is a bug if it has without justification (as it will break things downstream...).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075219
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/gc/GarbageCollectorCycle.java ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.gc;
+
+import org.apache.accumulo.core.gc.thrift.GcCycleStats;
+
+/**
+ *
+ * Metrics about a single cycle of the garbage collector
+ *
+ * @since 2.0.0
+ *
+ */
+public class GarbageCollectorCycle {
+
+ public static final GarbageCollectorCycle EMPTY = new GarbageCollectorCycle();
+
+ // Variable names become JSON key
+ public long started, finished, candidates, inUse, deleted, errors;
+
+ public GarbageCollectorCycle() {
+ started = finished = candidates = inUse = deleted = errors = 0l;
--- End diff --
Each on their own line, or do this when you declare the variables and remove the default constructor. This doesn't match existing style.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110050790
--- Diff: server/monitor/src/main/resources/resources/vis.js ---
@@ -0,0 +1,506 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
--- End diff --
Thanks, forgot to remove it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110192525
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
+
+ <link rel="stylesheet" type="text/css" href="/resources/screen.css" media="screen" />
+ <script>
+ /**
+ * Sets up autorefresh on initial load
+ */
+ $(document).ready(function() {
+ setupAutoRefresh();
+ });
+ </script>
+ <#if js??>
--- End diff --
Oh neat. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110162252
--- Diff: test/src/main/java/org/apache/accumulo/test/ThriftServerBindsBeforeZooKeeperLockIT.java ---
@@ -94,7 +94,7 @@ public void testMonitorService() throws Exception {
final int responseCode = cnxn.getResponseCode();
final String errorText = FunctionalTestUtils.readAll(cnxn.getErrorStream());
// This is our "assertion", but we want to re-check it if it's not what we expect
- if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains(BasicServlet.STANDBY_MONITOR_MESSAGE)) {
+ if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains("This is not the active Monitor")) {
--- End diff --
@ctubbsii worked on this better than I can, but he made the change since we no longer have a BasicServlet class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
To be clear, the registration that @joshelser describes has not changed. What has changed is that the standby serves the REST API and views instead of displaying a message that it does not have the lock, while it is waiting for the lock.
The sole purpose of the lock for the monitor is to support automatic forwarding to one, and only one, monitor with the out-of-box log configuration. Otherwise, the service registration would be a list instead of a singleton lock.
@lstav and I discussed the possibility of adding a message to the log view only (instead of all pages) similar to the old message, so that a user won't be confused why no logs are showing up when they visit a standby monitor. That would only matter for the log view, because that's the only functionality that would differ between the active and standby monitors. However, I'm not sure whether or not that's the right thing to do, because a user may wish to configure a log appender to forward logs to all monitors, and such a message would only apply to our custom monitor appender, which discriminates in favor of the active monitor for log forwarding.
To summarize: the test should still check that HTTP is serving before acquiring the lock. However, the text of what it is looking for in the standby monitor may have changed. We could add a simple REST endpoint for the monitor to report its own status... and that could contain things like: active or standby, listening ports, listening interfaces, last master contact time, etc. That would make it easier to verify that the HTTP service is working, even when the lock has not yet been acquired.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
I didn't get a chance to try to merge this in today. I should have more time next week. In the meantime, I'll set up my Jenkins instance to run the ITs for me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110476674
--- Diff: server/monitor/src/main/resources/resources/master.js ---
@@ -0,0 +1,261 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates master initial table
+ */
+$(document).ready(function() {
+ createHeader();
+ doBanner('masterBanner', 'danger', 'Master Server Not Running');
+ refreshMaster();
+
+ // Create tooltip for table column information
+ $(document).tooltip();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshMaster() {
+ $.ajaxSetup({
+ async: false
+ });
+ getMaster();
+ getRecoveryList();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshMasterTable();
+ recoveryList();
+}
+
+
+/*
+ * The tables refresh function will do this functionality
+ * If tables are removed from Master, uncomment this function
+ */
+/**
+ * Used to redraw the page
+ */
+/*function refresh() {
+ refreshMaster();
+}*/
+
+/**
+ * Creates recovery list table
+ */
+function recoveryList() {
+ /*
+ * Get the recovery value obtained earlier,
+ * if it doesn't exists, create an empty array
+ */
+ var data = sessionStorage.recoveryList === undefined ?
+ [] : JSON.parse(sessionStorage.recoveryList);
+
+ $('#recoveryList tr').remove();
+
+ // If there is no recovery list data, hide the table
+ if (data.length === 0 || data.recoveryList.length === 0) {
+ $('#recoveryList').hide();
+ } else {
+ $('#recoveryList').show();
+
+ var caption = [];
+
+ caption.push('<span class="table-caption">Log Recovery</span><br />');
+ caption.push('<span class="table-subcaption">Some tablets were unloaded' +
+ ' in an unsafe manner. Write-ahead logs are being' +
+ ' recovered.</span><br />');
+
+ $('<caption/>', {
+ html: caption.join('')
+ }).appendTo('#recoveryList');
+
+ var items = [];
+
+ /*
+ * Create the header for the recovery list table
+ * Adds the columns, add sortTable function on click,
+ * if the column has a description, add title taken from the global.js
+ */
+ items.push('<th class="firstcell" onclick="sortTable(0)">' +
+ 'Server </th>');
+ items.push('<th onclick="sortTable(1)">Log </th>');
+ items.push('<th onclick="sortTable(2)">Time </th>');
+ items.push('<th onclick="sortTable(3)">Copy/Sort </th>');
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#recoveryList');
+
+ // Creates the table for the recovery list
+ $.each(data.recoveryList, function(key, val) {
+ var items = [];
+ items.push('<td class="firstcell left" data-value="' + val.server + '">' +
+ val.server + '</td>');
+ items.push('<td class="right" data-value="' + val.log + '">' + val.log +
+ '</td>');
+ var date = new Date(parseInt(val.time));
+ date = date.toLocaleString().split(' ').join(' ');
+ items.push('<td class="right" data-value="' + val.time + '">' + date +
+ '</td>');
+ items.push('<td class="right" data-value="' + val.copySort + '">' +
+ val.copySort + '</td>');
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#recoveryList');
+ });
+ }
+}
+
+/**
+ * Generates the master table
+ */
+function refreshMasterTable() {
+ // Gets the master status
+ var status = JSON.parse(sessionStorage.status).masterStatus;
+
+ // Hide the banner and the master table
+ $('#masterBanner').hide();
+ $('#masterStatus tr:gt(0)').remove();
+ $('#masterStatus').hide();
+
+ // If master status is error, show banner, otherwise, create master table
+ if (status === 'ERROR') {
+ $('#masterBanner').show();
+ } else {
+ $('#masterStatus').show();
+ var data = JSON.parse(sessionStorage.master);
+ var items = [];
+ items.push('<td class="firstcell left" data-value="' + data.master +
+ '">' + data.master + '</td>');
+
+ items.push('<td class="right" data-value="' + data.onlineTabletServers +
+ '">' + data.onlineTabletServers + '</td>');
+
+ items.push('<td class="right" data-value="' + data.totalTabletServers +
+ '">' + data.totalTabletServers + '</td>');
+
+ var date = new Date(parseInt(data.lastGC));
+ date = date.toLocaleString().split(' ').join(' ');
+ items.push('<td class="left" data-value="' + data.lasGC +
+ '"><a href="/gc">' + date + '</a></td>');
+
+ items.push('<td class="right" data-value="' + data.tablets +
+ '">' + bigNumberForQuantity(data.tablets) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.unassignedTablets +
+ '">' + bigNumberForQuantity(data.unassignedTablets) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.numentries +
+ '">' + bigNumberForQuantity(data.numentries) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.ingestrate +
+ '">' + bigNumberForQuantity(Math.round(data.ingestrate)) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.entriesRead +
+ '">' + bigNumberForQuantity(Math.round(data.entriesRead)) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.queryrate +
+ '">' + bigNumberForQuantity(Math.round(data.queryrate)) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.holdTime +
+ '">' + timeDuration(data.holdTime) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.osload +
+ '">' + bigNumberForQuantity(data.osload) + '</td>');
--- End diff --
Sounds like a good idea, I'll check it out!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110473784
--- Diff: server/monitor/src/main/resources/resources/overview.js ---
@@ -0,0 +1,293 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates overview initial tables
+ */
+$(document).ready(function() {
+ createMasterTable();
+ createZKTable();
+ refreshOverview();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshOverview() {
+ $.ajaxSetup({
+ async: false
+ });
+ getMaster();
+ getZK();
+ getIngestRate();
+ getScanEntries();
+ getIngestByteRate();
+ getQueryByteRate();
+ getLoadAverage();
+ getLookups();
+ getMinorCompactions();
+ getMajorCompactions();
+ getIndexCacheHitRate();
+ getDataCacheHitRate();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshMasterTable();
+ refreshZKTable();
+ makePlots();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshOverview();
+}
+
+/**
+ * Refreshes the master table
+ */
+function refreshMasterTable() {
+ var data = sessionStorage.master === undefined ?
+ [] : JSON.parse(sessionStorage.master);
+
+ $('#master tr td:first').hide();
+ $('#master tr td').hide();
+
+ // If the master is down, show the first row, otherwise refresh old values
+ if (data.length === 0 || data.master === 'No Masters running') {
+ $('#master tr td:first').show();
+ } else {
+ $('#master tr td:not(:first)').show();
+ var table = $('#master td.right');
+
+ table.eq(0).html(bigNumberForQuantity(data.tables));
+ table.eq(1).html(bigNumberForQuantity(data.totalTabletServers));
+ table.eq(2).html(bigNumberForQuantity(data.deadTabletServersCount));
+ table.eq(3).html(bigNumberForQuantity(data.tablets));
+ table.eq(4).html(bigNumberForQuantity(data.numentries));
+ table.eq(5).html(bigNumberForQuantity(data.lookups));
+ table.eq(6).html(timeDuration(data.uptime));
+ }
+}
+
+/**
+ * Generates the master table
+ */
+function createMasterTable() {
+ var items = [];
+ items.push('<tr><th colspan="2"><a href="/master">Accumulo' +
+ ' Master</a></th></tr>');
+
+ items.push('<tr><td colspan="2" class="center">' +
+ '<span class="label label-danger">Master is Down</span></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tables">Tables</a></td>' +
+ '<td class="right"></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tservers">Tablet' +
+ ' Servers</a></td><td class="right"></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tservers">Dead ' +
+ 'Tablet Servers</a></td><td class="right"></td></tr>');
+
+ items.push('<tr><td class="left">Tablets</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Entries</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Lookups</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Uptime</td><td class="right"></td></tr>');
+
+ $('<table/>', {
+ html: items.join(''),
+ class: 'table table-bordered table-striped table-condensed'
+ }).appendTo('#master');
+}
+
+/**
+ * Refresh the zookeeper table
+ */
+function refreshZKTable() {
+ var data = sessionStorage.zk === undefined ?
+ [] : JSON.parse(sessionStorage.zk);
+
+ $('#zookeeper tr td:first').hide();
+ $('#zookeeper tr:gt(2)').remove();
+
+ if (data.length === 0 || data.zkServers.length === 0) {
+ $('#zookeeper tr td:first').show();
+ } else {
+ var items = [];
+ $.each(data.zkServers, function(key, val) {
+ if (val.clients >= 0) {
+ items.push('<td class="left">' + val.server + '</td>');
+ items.push('<td class="left">' + val.mode + '</td>');
+ items.push('<td class="right">' + val.clients + '</td></tr>');
+ } else {
+ items.push('<tr><td class="left">' + val.server + '</td>');
+ items.push('<td class="left"><span class="error">Down</span></td>');
+ items.push('<td class="right"></td>');
+ }
+ });
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#zookeeper table');
+ }
+}
+
+/**
+ * Generates the zookeeper table
+ */
+function createZKTable() {
+ var items = [];
+ items.push('<tr><th colspan="3">Zookeeper</th></tr>');
+ items.push('<tr><th>Server</th><th>Mode</th><th>Clients</th></tr>');
+ items.push('<td class="center" colspan="3"><i>No Zookeepers</i></td>');
+ $('<table/>', {
+ html: items.join(''),
+ class: 'table table-bordered table-striped table-condensed'
+ }).appendTo('#zookeeper');
+}
+
+//// Overview plot creation
+
+/**
+ * Create the plots for the overview page
+ */
+function makePlots() {
+ var d = new Date();
+ var n = d.getTimezoneOffset() * 60000; // Converts offset to milliseconds
+ var tz = new Date().toLocaleTimeString('en-us',
+ {timeZoneName: 'short'}).split(' ')[2]; // Short version of timezone
+ var tzFormat = '%H:%M<br />' + tz;
+
+ // Create Ingest Rate plot
+ var ingestRate = [];
+ var data = sessionStorage.ingestRate === undefined ?
+ [] : JSON.parse(sessionStorage.ingestRate);
+ $.each(data, function(key, val) {
+
+ ingestRate.push([val.first - n, val.second]);
+ });
+ $.plot($('#ingest_entries'), [{ data: ingestRate,
+ lines: { show: true }, color: '#d9534f' }],
+ {yaxis: {}, xaxis: {mode: 'time', minTickSize: [1, 'minute'],
+ timeformat: tzFormat, ticks: 3}});
+
+ // Create Scan Entries plot
+ var scanEntries = {'Read': [], 'Returned': []};
+ var data = sessionStorage.scanEntries === undefined ?
+ [] : JSON.parse(sessionStorage.scanEntries);
+ $.each(data, function(key, val) {
+ $.each(val.second, function(key2, val2) {
+ scanEntries[val.first].push([val2.first - n, val2.second]);
+ });
+ });
+ $.plot($('#scan_entries'), [{ label: 'Read',
+ data: scanEntries.Read, lines: { show: true }, color: '#d9534f' },
+ { label: 'Returned', data: scanEntries.Returned, lines: { show: true },
+ color: '#337ab7' }], {yaxis: {}, xaxis: {mode: 'time',
+ minTickSize: [1, 'minute'], timeformat: tzFormat, ticks: 3}});
--- End diff --
You should break up these calls to plot into multiple lines. Or if you can't, create a function to return an object.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on the issue:
https://github.com/apache/accumulo/pull/242
One problem with ThriftServerBindsBeforeZooKeeperLockIT (independent of these changes) is that a failure or error will result in the test running indefinitely. Only when things go as expected does it exit out of the infinite loops.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111437778
--- Diff: assemble/pom.xml ---
@@ -195,6 +235,86 @@
<optional>true</optional>
</dependency>
<dependency>
+ <groupId>org.freemarker</groupId>
+ <artifactId>freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-utils</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>osgi-resource-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>aopalliance-repackaged</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>javax.inject</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.bundles.repackaged</groupId>
+ <artifactId>jersey-guava</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-jetty-http</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-client</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-common</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-server</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-entity-filtering</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-mvc</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-mvc-freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.media</groupId>
+ <artifactId>jersey-media-jaxb</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.media</groupId>
+ <artifactId>jersey-media-json-jackson</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.javassist</groupId>
+ <artifactId>javassist</artifactId>
--- End diff --
We (I) will clean up dependencies during the merge. @mikewalch 's recent changes to enforce dep analysis will help with that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110154632
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java ---
@@ -118,19 +98,17 @@ public void start() {
}
}
- public void stop() {
+ private void stop() {
--- End diff --
This is part of the work @ctubbsii did, so I am not sure what the reason behind this is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075716
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
Not specific to this class (just happened to think about it): isn't the `@Produces` annotation inherited? e.g. we could make a base `Resource` class which has the `@Produces` marking and not have to worry about getting it right on every other resource we create?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110468167
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLInformation.java ---
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.xml.bind.annotation.XmlRootElement;
+
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerList;
+import org.apache.accumulo.monitor.rest.master.MasterInformation;
+import org.apache.accumulo.monitor.rest.tables.TableInformation;
+import org.apache.accumulo.monitor.rest.tables.TableInformationList;
+import org.apache.accumulo.monitor.rest.tables.TableNamespace;
+import org.apache.accumulo.monitor.rest.tables.TablesList;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServers;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerList;
+import org.apache.accumulo.monitor.rest.tservers.ServersShuttingDown;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Generate XML summary of Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@XmlRootElement(name = "stats")
+public class XMLInformation {
+
+ // Variable names become JSON keys
+ public List<TabletServer> servers;
+
+ public String masterGoalState, masterState;
+
+ public BadTabletServers badTabletServers;
+ public ServersShuttingDown tabletServersShuttingDown;
+ public Integer unassignedTablets;
+ public DeadServerList deadTabletServers;
+
+ public DeadLoggerList deadLoggers;
+
+ public TableInformationList tables;
+
+ public Totals totals;
+
+ public XMLInformation() {
+ servers = new ArrayList<>();
+ }
+
+ /**
+ * Stores Monitor information as XML
+ *
+ * @param size
+ * Number of tservers
+ * @param info
+ * Master information
+ * @param tablesList
+ * Table list
+ */
+ public XMLInformation(int size, MasterInformation info, TablesList tablesList) {
+ this.servers = new ArrayList<>(size);
+
+ this.masterGoalState = info.masterGoalState;
+ this.masterState = info.masterState;
+
+ this.badTabletServers = info.badTabletServers;
+ this.tabletServersShuttingDown = info.tabletServersShuttingDown;
+ this.unassignedTablets = info.unassignedTablets;
+ this.deadTabletServers = info.deadTabletServers;
+ this.deadLoggers = info.deadLoggers;
+
+ getTableInformationList(tablesList.tables);
+
+ this.totals = new Totals(info.ingestrate, info.queryrate, info.numentries);
+ }
+
+ /**
+ * Adds a new tablet to the XML
+ *
+ * @param tablet
+ * Tablet to add
+ */
+ public void addTablet(TabletServer tablet) {
--- End diff --
@joshelser not sure what you mean by this, you mean changing addTablet to addTabletServer?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110468835
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+ /**
+ * Generates a trace summary
+ *
+ * @param minutes
+ * Range of minutes to filter traces
+ * @return Trace summary in specified range
+ */
+ @Path("summary/{minutes}")
+ @GET
+ public RecentTracesList getTraces(@DefaultValue("10") @PathParam("minutes") int minutes) throws Exception {
+
+ RecentTracesList recentTraces = new RecentTracesList();
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return recentTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+ scanner.setRange(range);
+
+ final Map<String,RecentTracesInformation> summary = new TreeMap<>();
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
--- End diff --
Why do you need privilege escalation in this class?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110464603
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/status/StatusResource.java ---
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.status;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.master.thrift.DeadServer;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.monitor.DedupedLogEvent;
+import org.apache.accumulo.server.monitor.LogService;
+import org.apache.log4j.Level;
+
+/**
+ *
+ * Generates the status for master, gc, and tservers as well as log and problem reports
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/status")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class StatusResource {
+
+ /**
+ * Generates the JSON object with the status
+ *
+ * @return Status report
+ */
+ @GET
+ public StatusInformation getTables() {
+
+ StatusInformation status;
+ String masterStatus;
+ String gcStatus;
+ String tServerStatus = "ERROR";
--- End diff --
You could create an enum for the Status values.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110250805
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
It may still be possible to declare these on a base resource. However, I removed that in my PR to @lstav because it created a StackOverflowError while processing sub-resources on the parent class. It's possible it can be inherited, but I'm not sure if it inherits from a parent class/interface to a child class or if it inherits from a parent resource to sub-resources. Doing both causes problems, and I didn't try them individually.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on the issue:
https://github.com/apache/accumulo/pull/242
While looking into the failing IT, ThriftServerBindsBeforeZooKeeperLockIT, I noticed BasicServlet (which was deleted) had functionality of a standby Monitor as part of HighlyAvailableService. @lstav @ctubbsii Do you know if this standby ability has been removed or just relocated?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110074364
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java ---
@@ -118,19 +98,17 @@ public void start() {
}
}
- public void stop() {
+ private void stop() {
--- End diff --
why private?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110012789
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java ---
@@ -461,25 +452,9 @@ public void run(String hostname) {
try {
log.debug("Creating monitor on port " + port);
server = new EmbeddedWebServer(hostname, port);
- server.addServlet(DefaultServlet.class, "/");
- server.addServlet(OperationServlet.class, "/op");
- server.addServlet(MasterServlet.class, "/master");
- server.addServlet(TablesServlet.class, "/tables");
- server.addServlet(TServersServlet.class, "/tservers");
- server.addServlet(ProblemServlet.class, "/problems");
- server.addServlet(GcStatusServlet.class, "/gc");
- server.addServlet(LogServlet.class, "/log");
- server.addServlet(XMLServlet.class, "/xml");
- server.addServlet(JSONServlet.class, "/json");
- server.addServlet(VisServlet.class, "/vis");
- server.addServlet(ScanServlet.class, "/scans");
- server.addServlet(BulkImportServlet.class, "/bulkImports");
- server.addServlet(Summary.class, "/trace/summary");
- server.addServlet(ListType.class, "/trace/listType");
- server.addServlet(ShowTrace.class, "/trace/show");
- server.addServlet(ReplicationServlet.class, "/replication");
- if (server.isUsingSsl())
- server.addServlet(ShellServlet.class, "/shell");
+ server.addServlet(getDefaultServlet(), "/resources/*");
+ server.addServlet(getRestServlet(), "/rest/*");
--- End diff --
Nice reduction in servlets
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111495815
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
Oh interesting. I didn't realize the annotation was dual-purpose. I figured both the serialization for REST and explicit XML would be implicit (having the annotation would be unnecessary). Am I interpreting this right that you're saying this is required for the XML summary to work?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110161691
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
--- End diff --
Bundling this with the packaging is part of what @ctubbsii is working/will work on. He checked the licenses for the included libraries and they are permitted (I believe all of them use the MIT license)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110074737
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/Totals.java ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+/**
+ *
+ * Generates the totals for XML summary
+ *
+ * @since 2.0.0
+ *
+ */
+public class Totals {
+
+ // Variable names become JSON keys
+ public double ingestrate, queryrate, diskrate = 0.0;
--- End diff --
The normal style is to define these each on their own line and not perform any inline assignment (do it in the constructor). Camel-case'ing the name would also be good.
I assume this affects much of the other code in here. I won't comment everywhere..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on the issue:
https://github.com/apache/accumulo/pull/242
Ok the reason I was asking was because of this check in ThriftServerBindsBeforeZooKeeperLockIT
`// This is our "assertion", but we want to re-check it if it's not what we expect`
`if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains("This is not the active Monitor")) {
return;
}`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110156589
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
--- End diff --
Thanks, I didn't think of this, just for reference the asterisk means all tables, and the hyphen means the default namespace (hopefully we can change the default in the future to be an actual character instead of an empty character. I'll make the necessary comments on the code to explain what it is.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110215877
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
--- End diff --
Ok, thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on the issue:
https://github.com/apache/accumulo/pull/242
I started reviewing this. I was able to build a tarball and run Accumulo locally with the new monitor. However, it looks like the integration tests are failing due to missing/extra dependencies in the monitor pom.xml. I will create a PR to your branch to help you fix up those dependencies and get the tests working again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
@ctubbsii thanks, should I delete the ACCUMULO-MONITOR branch? or would you rather I wait?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
> Apparently we remember things differently. ... I thought that's why I built that 503-response in the first place.
I was always uncomfortable with the 503 on the whole thing when it only really mattered for the log forwarding. Even then, while I understood the convenience of configure-less detection of monitor location for log forwarding, I wasn't happy with the restriction. We can still do the 503, but I think we can make it more narrowly focused, to just the feature which require exclusivity (more specifically, when the user wants it to be exclusive). We now have a configure-less detection of monitor location for log forwarding which does not require this exclusivity behavior.
Personally, I would prefer to support multiple monitors, and I have a use case where log forwarding is disabled. I don't want to launch unusable secondary monitors which is only unusable to support a feature which I'm not using. For this reason, I was disappointed when Eric first introduced the "active/standby" to the monitor. I never thought this was a good idea, or a necessary one. Not enforcing this restriction actually makes the code simpler and supports more use cases.
> But they still only go to one place, don't they?
With the default log4j configuration, yes. However, the improvements we've made make it easier for the user to configure log forwarding in a way that works for them. That includes forwarding to a single monitor, multiple monitors, or none.
> I think the simple solution would just be to return something that isn't `"[]" with HTTP/200`
Agreed. That's basically what I thought I was describing, but with some informative message in the view when the AJAX receives this other "something".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110991710
--- Diff: server/monitor/src/main/resources/resources/overview.js ---
@@ -0,0 +1,293 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates overview initial tables
+ */
+$(document).ready(function() {
+ createMasterTable();
+ createZKTable();
+ refreshOverview();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshOverview() {
+ $.ajaxSetup({
+ async: false
+ });
+ getMaster();
+ getZK();
+ getIngestRate();
+ getScanEntries();
+ getIngestByteRate();
+ getQueryByteRate();
+ getLoadAverage();
+ getLookups();
+ getMinorCompactions();
+ getMajorCompactions();
+ getIndexCacheHitRate();
+ getDataCacheHitRate();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshMasterTable();
+ refreshZKTable();
+ makePlots();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshOverview();
+}
+
+/**
+ * Refreshes the master table
+ */
+function refreshMasterTable() {
+ var data = sessionStorage.master === undefined ?
+ [] : JSON.parse(sessionStorage.master);
+
+ $('#master tr td:first').hide();
+ $('#master tr td').hide();
+
+ // If the master is down, show the first row, otherwise refresh old values
+ if (data.length === 0 || data.master === 'No Masters running') {
+ $('#master tr td:first').show();
+ } else {
+ $('#master tr td:not(:first)').show();
+ var table = $('#master td.right');
+
+ table.eq(0).html(bigNumberForQuantity(data.tables));
+ table.eq(1).html(bigNumberForQuantity(data.totalTabletServers));
+ table.eq(2).html(bigNumberForQuantity(data.deadTabletServersCount));
+ table.eq(3).html(bigNumberForQuantity(data.tablets));
+ table.eq(4).html(bigNumberForQuantity(data.numentries));
+ table.eq(5).html(bigNumberForQuantity(data.lookups));
+ table.eq(6).html(timeDuration(data.uptime));
+ }
+}
+
+/**
+ * Generates the master table
+ */
+function createMasterTable() {
+ var items = [];
+ items.push('<tr><th colspan="2"><a href="/master">Accumulo' +
+ ' Master</a></th></tr>');
+
+ items.push('<tr><td colspan="2" class="center">' +
+ '<span class="label label-danger">Master is Down</span></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tables">Tables</a></td>' +
+ '<td class="right"></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tservers">Tablet' +
+ ' Servers</a></td><td class="right"></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tservers">Dead ' +
+ 'Tablet Servers</a></td><td class="right"></td></tr>');
+
+ items.push('<tr><td class="left">Tablets</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Entries</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Lookups</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Uptime</td><td class="right"></td></tr>');
+
+ $('<table/>', {
+ html: items.join(''),
+ class: 'table table-bordered table-striped table-condensed'
+ }).appendTo('#master');
+}
+
+/**
+ * Refresh the zookeeper table
+ */
+function refreshZKTable() {
+ var data = sessionStorage.zk === undefined ?
+ [] : JSON.parse(sessionStorage.zk);
+
+ $('#zookeeper tr td:first').hide();
+ $('#zookeeper tr:gt(2)').remove();
+
+ if (data.length === 0 || data.zkServers.length === 0) {
+ $('#zookeeper tr td:first').show();
+ } else {
+ var items = [];
+ $.each(data.zkServers, function(key, val) {
+ if (val.clients >= 0) {
+ items.push('<td class="left">' + val.server + '</td>');
+ items.push('<td class="left">' + val.mode + '</td>');
+ items.push('<td class="right">' + val.clients + '</td></tr>');
+ } else {
+ items.push('<tr><td class="left">' + val.server + '</td>');
+ items.push('<td class="left"><span class="error">Down</span></td>');
+ items.push('<td class="right"></td>');
+ }
+ });
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#zookeeper table');
+ }
+}
+
+/**
+ * Generates the zookeeper table
+ */
+function createZKTable() {
+ var items = [];
+ items.push('<tr><th colspan="3">Zookeeper</th></tr>');
+ items.push('<tr><th>Server</th><th>Mode</th><th>Clients</th></tr>');
+ items.push('<td class="center" colspan="3"><i>No Zookeepers</i></td>');
+ $('<table/>', {
+ html: items.join(''),
+ class: 'table table-bordered table-striped table-condensed'
+ }).appendTo('#zookeeper');
+}
+
+//// Overview plot creation
+
+/**
+ * Create the plots for the overview page
+ */
+function makePlots() {
+ var d = new Date();
+ var n = d.getTimezoneOffset() * 60000; // Converts offset to milliseconds
+ var tz = new Date().toLocaleTimeString('en-us',
+ {timeZoneName: 'short'}).split(' ')[2]; // Short version of timezone
+ var tzFormat = '%H:%M<br />' + tz;
+
+ // Create Ingest Rate plot
+ var ingestRate = [];
+ var data = sessionStorage.ingestRate === undefined ?
+ [] : JSON.parse(sessionStorage.ingestRate);
+ $.each(data, function(key, val) {
+
+ ingestRate.push([val.first - n, val.second]);
+ });
+ $.plot($('#ingest_entries'), [{ data: ingestRate,
+ lines: { show: true }, color: '#d9534f' }],
+ {yaxis: {}, xaxis: {mode: 'time', minTickSize: [1, 'minute'],
+ timeformat: tzFormat, ticks: 3}});
+
+ // Create Scan Entries plot
+ var scanEntries = {'Read': [], 'Returned': []};
+ var data = sessionStorage.scanEntries === undefined ?
+ [] : JSON.parse(sessionStorage.scanEntries);
+ $.each(data, function(key, val) {
+ $.each(val.second, function(key2, val2) {
+ scanEntries[val.first].push([val2.first - n, val2.second]);
+ });
+ });
+ $.plot($('#scan_entries'), [{ label: 'Read',
+ data: scanEntries.Read, lines: { show: true }, color: '#d9534f' },
+ { label: 'Returned', data: scanEntries.Returned, lines: { show: true },
+ color: '#337ab7' }], {yaxis: {}, xaxis: {mode: 'time',
+ minTickSize: [1, 'minute'], timeformat: tzFormat, ticks: 3}});
--- End diff --
Managed to get this pretty simplified :), now it is just a simple function call that has all of this in it (no more repeating the same code :)), I'll push once I finish another change on the HTML/JavaScript code.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110192139
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
--- End diff --
Ah, don't worry about it then.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110470726
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLResource.java ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.monitor.rest.tables.TablesResource;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Responsible for generating an XML summary of the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class XMLResource {
+
+ /**
+ * Generates an XML summary of the Monitor
+ *
+ * @return XML summary
+ */
+ @GET
+ public XMLInformation getXMLInformation() {
+
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ // Add Monitor information
+ XMLInformation xml = new XMLInformation(mmi.tServerInfo.size(), new MasterResource().getTables(), new TablesResource().getTables());
+
+ // Add tserver information
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ xml.addTablet(new TabletServer(status));
--- End diff --
@joshelser I'm not sure I understand what is the confusion, is it just for the XMLResource class? Or is it anywhere where TabletServer exists? If it is the second one, keep in mind that none of the objects of the REST call are the actual instances, I created new classes specifically for the REST (in order to store variables that will serve as JSON keys). If it is something else, could you elaborate?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076953
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
+ }
+
+ // Obtains the hours, if time is 0, return hours, minutes, and units
+ hr = time % 24;
+ time = Math.floor(time / 24);
+ if (time == 0) {
+ return hr + 'h' + ' ' + min + 'm';
+ }
+
+ // Obtains the days, if time is 0, return days, hours, and units
+ day = time % 365;
+ time = Math.floor(time / 365);
+ if (time == 0) {
+ return day + 'd' + ' ' + hr + 'h';
+ }
+
+ // Obtains the years, if time is 0, return years, days, and units
+ yr = Math.floor(time);
+ return yr + 'y' + ' ' + day + 'd';
+}
+
+/**
+ * Sorts the selected table by column in the direction chosen
+ *
+ * @param {string} tableID Table to sort
+ * @param {string} direction Direction to sort table, asc or desc
+ * @param {number} n Column to sort
+ */
+function sortTables(tableID, direction, n) {
+ var table, rows, switching, i, x, y, h, shouldSwitch, dir, xFinal, yFinal;
+ table = document.getElementById(tableID);
+ switching = true;
+
+ dir = direction;
+ sessionStorage.direction = dir;
+
+ // Select the rows of the table
+ rows = table.getElementsByTagName('tr');
+
+ // Clears the sortable class from the table columns
+ var count = 0;
+ while (rows[0].getElementsByTagName('th').length > count) {
+ var tmpH = rows[0].getElementsByTagName('th')[count];
+ tmpH.classList.remove('sortable');
+ if (rows.length > 2) {
+ tmpH.classList.add('sortable');
+ }
+ $(tmpH.getElementsByTagName('span')).remove();
+ count += 1;
+ }
+
+ // If there are more than 2 rows, add arrow to the selected column
+ if (rows.length <= 2) {
+ switching = false;
+ } else {
+ h = rows[0].getElementsByTagName('th')[n];
+ if (dir == 'asc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-up"' +
+ ' width="10px" height="10px" />');
+ } else if (dir == 'desc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-down"' +
+ ' width="10px" height="10px" />');
+ }
+ }
+
+ /*
+ * Make a loop that will continue until
+ * no switching has been done:
+ */
+ while (switching) {
+ switching = false;
+ rows = table.getElementsByTagName('tr');
+
+ /*
+ * Loop through all table rows (except the
+ * first, which contains table headers):
+ */
+ for (i = 1; i < (rows.length - 1); i++) {
+ shouldSwitch = false;
+ /*
+ * Get two elements to compare,
+ * one from current row and one from the next:
+ * If the element is a dash, convert to null, otherwise,
+ * if it is a string, convert to number
+ */
+ x = rows[i].getElementsByTagName('td')[n].getAttribute('data-value');
+ xFinal = (x === '-' || x === '—' ?
+ null : (Number(x) == x ? Number(x) : x));
+
+ y = rows[i + 1].getElementsByTagName('td')[n].getAttribute('data-value');
+ yFinal = (y === '-' || y === '—' ?
+ null : (Number(y) == y ? Number(y) : y));
+
+ /*
+ * Check if the two rows should switch place,
+ * based on the direction, asc or desc:
+ */
+ if (dir == 'asc') {
+ if (xFinal > yFinal || (xFinal !== null && yFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ } else if (dir == 'desc') {
+ if (xFinal < yFinal || (yFinal !== null && xFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ }
+ }
+ if (shouldSwitch) {
+ /*
+ * If a switch has been marked, make the switch
+ * and mark that a switch has been done:
+ */
+ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
+ switching = true;
+ }
+ }
+}
+
+/**
+ * Clears the selected table while leaving the headers
+ *
+ * @param {string} tableID Table to clear
+ */
+function clearTable(tableID) {
+ // JQuery selector to select all rows except for the first row (header)
+ $('#' + tableID).find('tr:not(:first)').remove();
+}
+
+///// REST Calls /////////////
+
+/**
+ * REST GET call for the master information,
+ * stores it on a sessionStorage variable
+ */
+function getMaster() {
+ $.getJSON('/rest/master', function(data) {
+ sessionStorage.master = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the zookeeper information,
+ * stores it on a sessionStorage variable
+ */
+function getZK() {
+ $.getJSON('/rest/zk', function(data) {
+ sessionStorage.zk = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the namespaces, stores it on a global variable
+ */
+function getNamespaces() {
+ $.getJSON('/rest/tables/namespaces', function(data) {
+ NAMESPACES = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tables on each namespace,
+ * stores it on a sessionStorage variable
+ *
+ * @param {array} namespaces Array holding the selected namespaces
+ */
+function getNamespaceTables(namespaces) {
+
+ // Creates a JSON object to store the tables
+ var jsonObj = {};
+ jsonObj.tables = [];
+
+ /* If the namespace array include *, get all tables, otherwise,
+ * get tables from specific namespaces
+ */
+ if (namespaces.indexOf('*') != -1) {
+ getTables();
+ } else {
+ $.each(namespaces, function(key, val) {
+ /* Makes the rest call for each of the namespaces in the array,
+ * stores them on the JSON object
+ */
+ if (val !== '*') {
+ var call = '/rest/tables/namespace/' + val;
+ $.getJSON(call, function(data) {
+ $.each(data.tables, function(key2, val2) {
+ jsonObj.tables.push(val2);
+ });
+ });
+ }
+ });
+ sessionStorage.tables = JSON.stringify(jsonObj);
+ }
+}
+
+/**
+ * REST GET call for the tables, stores it on a sessionStorage variable
+ */
+function getTables() {
+ $.getJSON('/rest/tables', function(data) {
+ sessionStorage.tables = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear a specific dead server
+ *
+ * @param {string} server Dead Server ID
+ */
+function clearDeadServers(server) {
+ var call = '/rest/tservers?server=' + server;
+ $.post(call);
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ */
+function getTServers() {
+ $.getJSON('/rest/tservers', function(data) {
+ sessionStorage.tservers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ *
+ * @param {string} server Server ID
+ */
+function getTServer(server) {
+ var call = '/rest/tservers/' + server;
+ $.getJSON(call, function(data) {
+ sessionStorage.server = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the scans, stores it on a sessionStorage variable
+ */
+function getScans() {
+ $.getJSON('/rest/scans', function(data) {
+ sessionStorage.scans = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the bulk imports, stores it on a sessionStorage variable
+ */
+function getBulkImports() {
+ $.getJSON('/rest/bulkImports', function(data) {
+ sessionStorage.bulkImports = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the garbage collector,
+ * stores it on a sessionStorage variable
+ */
+function getGarbageCollector() {
+ $.getJSON('/rest/gc', function(data) {
+ sessionStorage.gc = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the server stats, stores it on a sessionStorage variable
+ */
+function getServerStats() {
+ $.getJSON('/rest/tservers/serverStats', function(data) {
+ sessionStorage.serverStats = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the recovery list, stores it on a sessionStorage variable
+ */
+function getRecoveryList() {
+ $.getJSON('/rest/tservers/recovery', function(data) {
+ sessionStorage.recoveryList = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the participating tablet servers,
+ * stores it on a sessionStorage variable
+ *
+ * @param {string} table Table ID
+ */
+function getTableServers(table) {
+ var call = '/rest/tables/' + table;
+ $.getJSON(call, function(data) {
+ sessionStorage.tableServers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace summary, stores it on a sessionStorage variable
+ *
+ * @param {string} minutes Number of minutes to display trace summary
+ */
+function getTraceSummary(minutes) {
+ var call = '/rest/trace/summary/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceSummary = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace type, stores it on a sessionStorage variable
+ *
+ * @param {string} type Type of the trace
+ * @param {string} minutes Number of minutes to display trace
+ */
+function getTraceOfType(type, minutes) {
+ var call = '/rest/trace/listType/' + type + '/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceType = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace id, stores it on a sessionStorage variable
+ *
+ * @param {string} id Trace ID
+ */
+function getTraceShow(id) {
+ var call = '/rest/trace/show/' + id;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceShow = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the logs, stores it on a sessionStorage variable
+ */
+function getLogs() {
+ $.getJSON('/rest/logs', function(data) {
+ sessionStorage.logs = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear logs
+ */
+function clearLogs() {
+ $.post('/rest/logs');
+}
+
+/**
+ * REST GET call for the problems
+ */
+function getProblems() {
+ getProblemSummary();
+ getProblemDetails();
+}
+
+/**
+ * REST POST call to clear all table problems
+ *
+ * @param {string} tableID Table ID
+ */
+function clearTableProblems(tableID) {
+ var call = '/rest/problems/summary?s=' + tableID;
+ // Change plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
+ $.post(call);
+}
+
+/**
+ * REST POST call to clear detail problems
+ *
+ * @param {string} table Table ID
+ * @param {string} resource Resource for problem
+ * @param {string} type Type of problem
+ */
+function clearDetailsProblems(table, resource, type) {
+ var call = '/rest/problems/details?table=' + table + '&resource=' +
+ resource + '&ptype=' + type;
+ // Changes plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
--- End diff --
why not have a method to replace '+' with '%2B' when constructing the URL path in the first place?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110289116
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
Specifically, in the automatic WADL processing of annotations when querying a REST endpoint with the OPTIONS http verb.
I can experiment a bit more in the future, to help simplify and reduce the repetitiveness, but I think it could be done after this is merged in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111509043
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
Ahh, thanks for the example. I understand now (was getting confused with "data included in XML" and "how data is included in XML")
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110474249
--- Diff: server/monitor/src/main/resources/resources/tables.js ---
@@ -0,0 +1,348 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshTables() {
+ $.ajaxSetup({
+ async: false
+ });
+ getNamespaces();
+ $.ajaxSetup({
+ async: true
+ });
+ createNamespacesDropdown();
+ // If the namespaces in sessionStorage is undefined, select all namespaces
+ if (sessionStorage.namespaces === undefined) {
+ sessionStorage.namespaces = '[]';
+ populateTable('*');
+ }
+ populateTable(undefined);
+ sortTable(sessionStorage.tableColumnSort === undefined ?
+ 0 : sessionStorage.tableColumnSort);
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ // If tables are in master page, refresh master and tables
+ if (!hasMaster) {
+ refreshTables();
+ } else {
+ refreshMaster();
+ refreshTables();
+ }
+}
+
+var hasMaster = false;
+function toggleMaster(master) {
+ hasMaster = master
+}
+
+/**
+ * Creates listeners for when the namespaces are selected or unselected
+ */
+function namespaceChanged() {
+ var $namespaceSelect = $('#namespaces');
+
+ $namespaceSelect.off();
+
+ $namespaceSelect.on('select2:select', function(e) {
+ var id = e.params === null ? undefined : e.params['data']['id'];
+ populateTable(id);
+ });
+
+ $namespaceSelect.on('select2:unselect', function(e) {
+ var id = e.params === null ? undefined : e.params['data']['id'];
+ populateTable(id);
+ });
+}
+
+/**
+ * Creates the namespaces dropdown menu
+ */
+function createNamespacesDropdown() {
+ var data = JSON.parse(NAMESPACES).namespaces;
+ var caption = [];
+
+ caption.push('<span class="table-caption">Table List</span><br />');
+
+ $('<caption/>', {
+ html: caption.join('')
+ }).appendTo('#filters');
+
+ var data2 = [{ id: '*', text: '* (All Tables)'}];
+ $.each(data, function(key, val) {
+ var namespace = val === '' ? '- (DEFAULT)' : val;
+ data2.push({id: val === '' ? '-' : val, text: namespace});
+ });
+
+ $('#namespaces').select2({
+ data: data2,
+ allowClear: true
+ });
+ namespaceChanged();
+}
+
+/**
+ * Creates the tables table with the selected namespace
+ *
+ * @param {string} ns Selected namespace
+ */
+function populateTable(ns) {
+ var tmpArr = sessionStorage.namespaces === undefined ?
+ [] : JSON.parse(sessionStorage.namespaces);
+ sessionStorage.namespaceChanged = true;
+ var namespaces = JSON.parse(NAMESPACES).namespaces;
+
+ // If there is a selected namespace, change the displayed tables
+ if (ns !== undefined) {
+ /*
+ * If the namespace has not been selected,
+ * add it to the namespace array, otherwise remove it
+ */
+ if (tmpArr.indexOf(ns) == -1) {
+ /* If the namespace is *, add all namespaces to the array,
+ * otherwise just add the selected namespace
+ */
+ if (ns === '*') {
+ tmpArr = [];
+
+ tmpArr.push('*');
+ $.each(namespaces, function(key, val) {
+ tmpArr.push(val === '' ? '-' : val);
+ });
+ } else {
+ tmpArr.push(ns);
+ /*
+ * If the namespace array is the same size
+ * as the total number of namespaces, add *
+ */
+ if (tmpArr.length == namespaces.length) {
+ tmpArr.push('*');
+ }
+ }
+ } else {
+ /*
+ * If * is in the array, and the selected
+ * namespace is not *, remove * from array
+ */
+ if (tmpArr.indexOf('*') !== -1 && ns !== '*') {
+ tmpArr.splice(tmpArr.indexOf('*'), 1);
+ }
+ /*
+ * If the selected namespace is not *,
+ * remove it from array, otherwise, remove all
+ */
+ if (ns !== '*') {
+ tmpArr.splice(tmpArr.indexOf(ns), 1);
+ } else {
+ tmpArr = [];
+ }
+ }
+ }
+
+ $('#namespaces').select2().val(tmpArr).trigger('change'); // TODO Fix this, causes null dataAdapter
+
+ sessionStorage.namespaces = JSON.stringify(tmpArr);
+
+ $.ajaxSetup({
+ async: false
+ });
+ getNamespaceTables(tmpArr);
+ $.ajaxSetup({
+ async: true
+ });
+
+ var data = sessionStorage.tables === undefined ?
+ [] : JSON.parse(sessionStorage.tables);
+ clearTable('tableList');
+
+ var numTables = 0;
+
+ $.each(data.tables, function(keyT, tab) {
+ // Only add tables that are part of the array, or all if * is in the array
+ if (tmpArr.indexOf(tab.namespace === '' ? '-' : tab.namespace) !== -1 ||
+ tmpArr.indexOf('*') !== -1) {
+ $.each(tab.table, function(key, val) {
+
+ var row = [];
+ row.push('<td class="firstcell left" data-value="' + val.tablename +
+ '"><a href="/tables/' + val.tableId +
+ '">' + val.tablename + '</a></td>');
+ row.push('<td class="center" data-value="' + val.tableState +
+ '"><span>' + val.tableState + '</span></td>');
+
+ if (val.tableState === 'ONLINE') {
+ row.push('<td class="right" data-value="' + val.tablets +
+ '">' + bigNumberForQuantity(val.tablets) + '</td>');
+
+ row.push('<td class="right" data-value="' + val.offlineTablets +
+ '">' + bigNumberForQuantity(val.offlineTablets) + '</td>');
+
+ row.push('<td class="right" data-value="' + val.recs + '">' +
+ bigNumberForQuantity(val.recs) + '</td>');
+
+ row.push('<td class="right" data-value="' + val.recsInMemory +
+ '">' + bigNumberForQuantity(val.recsInMemory) + '</td>');
+
+ row.push('<td class="right" data-value="' + val.ingest +
+ '">' + bigNumberForQuantity(Math.floor(val.ingest)) + '</td>');
+
+ row.push('<td class="right" data-value="' + val.entriesRead +
+ '">' + bigNumberForQuantity(Math.floor(val.entriesRead)) +
+ '</td>');
+
+ row.push('<td class="right" data-value="' + val.entriesReturned +
+ '">' + bigNumberForQuantity(Math.floor(val.entriesReturned)) +
+ '</td>');
+
+ row.push('<td class="right" data-value="' + val.holdTime + '">' +
--- End diff --
Another simple function would be nice here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
I'm going to try to merge this in tomorrow, unless there are any further issues which cannot be resolved in a follow-up issue. @lstav @joshelser any objection to squashing things down (retaining @lstav as author)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110288818
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
--- End diff --
Yes, instead of simply empty string. I was thinking something with reserved chars, like `_default_`. The benefit here is that we could add a connection namespace context like `use <namespace>;` in traditional RDBMS for a more intuitive API. If using the `_default_` namespace, a smarter client could interpret the absence of a fully qualified table name as relative to the current namespace context, rather than always the default namespace. This would work in both the current API (whose context is always the default namespace) and a new API where you can switch namespace contexts.
If I end up working on that API some more, I can elaborate a bit more. For these purposes, I think it just makes sense to have a better name for UI menus. A dash just doesn't cut it, I think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110252576
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+ // Variable names become JSON keys
+ private TabletStats total, historical;
+
+ /**
+ * Generates tserver summary
+ *
+ * @return tserver summary
+ */
+ @GET
+ public TabletServers getTserverSummary() {
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ tserverInfo.addTablet(new TabletServer(status));
+ }
+
+ tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+ return tserverInfo;
+ }
+
+ /**
+ * REST call to clear dead servers from list
+ *
+ * @param server
+ * Dead server to clear
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ public void clearDeadServer(@QueryParam("server") String server) throws Exception {
+ DeadServerList obit = new DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + Constants.ZDEADTSERVERS);
+ obit.delete(server);
+ }
+
+ /**
+ * Generates a recovery tserver list
+ *
+ * @return Recovery tserver list
+ */
+ @Path("recovery")
+ @GET
+ public Map<String,List<Map<String,String>>> getTserverRecovery() {
+
+ Map<String,List<Map<String,String>>> jsonObj = new HashMap<String,List<Map<String,String>>>();
+ List<Map<String,String>> recoveryList = new ArrayList<>();
+ Map<String,String> recoveryObj = new HashMap<String,String>();
+
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ for (TabletServerStatus server : mmi.tServerInfo) {
+ if (server.logSorts != null) {
+ for (RecoveryStatus recovery : server.logSorts) {
+ recoveryObj.put("server", AddressUtil.parseAddress(server.name, false).getHostText());
+ recoveryObj.put("log", recovery.name);
+ recoveryObj.put("time", Long.toString(recovery.runtime));
+ recoveryObj.put("copySort", Double.toString(recovery.progress));
+
+ recoveryList.add(recoveryObj);
+ }
+ }
+ }
+
+ jsonObj.put("recoveryList", recoveryList);
+
+ return jsonObj;
+ }
+
+ /**
+ * Generates details for the selected tserver
+ *
+ * @param tserverAddr
+ * TServer name
+ * @return TServer details
+ */
+ @Path("{address}")
+ @GET
+ public TabletServerSummary getTserverDetails(@PathParam("address") String tserverAddr) throws Exception {
+
+ String tserverAddress = tserverAddr;
+
+ boolean tserverExists = false;
+ if (tserverAddress != null && tserverAddress.isEmpty() == false) {
+ for (TabletServerStatus ts : Monitor.getMmi().getTServerInfo()) {
+ if (tserverAddress.equals(ts.getName())) {
+ tserverExists = true;
+ break;
+ }
+ }
+ }
+
+ if (tserverAddress == null || tserverAddress.isEmpty() || tserverExists == false) {
+
+ return null;
+ }
+
+ double totalElapsedForAll = 0;
+ double splitStdDev = 0;
+ double minorStdDev = 0;
+ double minorQueueStdDev = 0;
+ double majorStdDev = 0;
+ double majorQueueStdDev = 0;
+ double currentMinorAvg = 0;
+ double currentMajorAvg = 0;
+ double currentMinorStdDev = 0;
+ double currentMajorStdDev = 0;
+ total = new TabletStats(null, new ActionStats(), new ActionStats(), new ActionStats(), 0, 0, 0, 0);
+ HostAndPort address = HostAndPort.fromString(tserverAddress);
+ historical = new TabletStats(null, new ActionStats(), new ActionStats(), new ActionStats(), 0, 0, 0, 0);
+ List<TabletStats> tsStats = new ArrayList<>();
+
+ try {
+ ClientContext context = Monitor.getContext();
+ TabletClientService.Client client = ThriftUtil.getClient(new TabletClientService.Client.Factory(), address, context);
+ try {
+ for (String tableId : Monitor.getMmi().tableMap.keySet()) {
+ tsStats.addAll(client.getTabletStats(Tracer.traceInfo(), context.rpcCreds(), tableId));
+ }
+ historical = client.getHistoricalStats(Tracer.traceInfo(), context.rpcCreds());
+ } finally {
+ ThriftUtil.returnClient(client);
+ }
+ } catch (Exception e) {
+ return null;
+ }
+
+ List<CurrentOperations> currentOps = doCurrentOperations(tsStats);
+
+ if (total.minors.num != 0)
+ currentMinorAvg = (long) (total.minors.elapsed / total.minors.num);
+ if (total.minors.elapsed != 0 && total.minors.num != 0)
+ currentMinorStdDev = stddev(total.minors.elapsed, total.minors.num, total.minors.sumDev);
+ if (total.majors.num != 0)
+ currentMajorAvg = total.majors.elapsed / total.majors.num;
+ if (total.majors.elapsed != 0 && total.majors.num != 0 && total.majors.elapsed > total.majors.num)
+ currentMajorStdDev = stddev(total.majors.elapsed, total.majors.num, total.majors.sumDev);
+
+ ActionStatsUpdator.update(total.minors, historical.minors);
+ ActionStatsUpdator.update(total.majors, historical.majors);
+ totalElapsedForAll += total.majors.elapsed + historical.splits.elapsed + total.minors.elapsed;
+
+ minorStdDev = stddev(total.minors.elapsed, total.minors.num, total.minors.sumDev);
+ minorQueueStdDev = stddev(total.minors.queueTime, total.minors.num, total.minors.queueSumDev);
+ majorStdDev = stddev(total.majors.elapsed, total.majors.num, total.majors.sumDev);
+ majorQueueStdDev = stddev(total.majors.queueTime, total.majors.num, total.majors.queueSumDev);
+ splitStdDev = stddev(historical.splits.num, historical.splits.elapsed, historical.splits.sumDev);
+
+ TabletServerDetailInformation details = doDetails(address, tsStats.size());
+
+ List<AllTimeTabletResults> allTime = doAllTimeResults(majorQueueStdDev, minorQueueStdDev, totalElapsedForAll, splitStdDev, majorStdDev, minorStdDev);
+
+ CurrentTabletResults currentRes = doCurrentTabletResults(currentMinorAvg, currentMinorStdDev, currentMajorAvg, currentMajorStdDev);
+
+ TabletServerSummary tserverDetails = new TabletServerSummary(details, allTime, currentRes, currentOps);
+
+ return tserverDetails;
+ }
+
+ private static final int concurrentScans = Monitor.getContext().getConfiguration().getCount(Property.TSERV_READ_AHEAD_MAXCONCURRENT);
--- End diff --
Dynamic reaction to properties seems out of scope for this issue. Couldn't this be done in a future task? I doubt the current monitor does this (well).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110258980
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/view/Indexes.java ---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.view;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.glassfish.jersey.server.mvc.Template;
+
+/**
+ *
+ * Index is responsible of specifying Monitor paths and setting the templates for the HTML code
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces(MediaType.TEXT_HTML)
+public class Indexes {
+
+ private Map<String,Object> getModel() {
+
+ Map<String,Object> model = new HashMap<>();
+ model.put("version", Constants.VERSION);
+ model.put("instance_name", Monitor.cachedInstanceName.get());
+ model.put("instance_id", Monitor.getContext().getInstance().getInstanceID());
+ return model;
+ }
+
+ /**
+ * Returns the overview template
+ *
+ * @return Overview model
+ */
+ @GET
+ @Template(name = "/index.ftl")
--- End diff --
Could rename `index.ftl` to `base.ftl` or `default.ftl` as it is the base template that is used to build all views.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on the issue:
https://github.com/apache/accumulo/pull/242
I am not sure what the second while loop is checking for in testMonitorService but its expecting a standby monitor. If its just testing for a standby monitor think its safe to remove it?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110252042
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
--- End diff --
I've previously tossed around the idea of creating a proper name for the default namespace, and leaving empty string as an alias. Cleaning up the namespace utility classes to do this is probably a bit tedious. If it helps with this interface, it may help prioritize giving it a real name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111496086
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/EmbeddedWebServer.java ---
@@ -118,19 +98,17 @@ public void start() {
}
}
- public void stop() {
+ private void stop() {
--- End diff --
sgtm. Long-term we should be moving to treating the objects that can be started/stopped with some sort of Service interface abstraction (e.g. TabletServer, Monitor, GC), but doesn't need to happen here and such a refactoring could invalidate anyways.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075046
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLResource.java ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.monitor.rest.tables.TablesResource;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Responsible for generating an XML summary of the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class XMLResource {
+
+ /**
+ * Generates an XML summary of the Monitor
+ *
+ * @return XML summary
+ */
+ @GET
+ public XMLInformation getXMLInformation() {
+
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ // Add Monitor information
+ XMLInformation xml = new XMLInformation(mmi.tServerInfo.size(), new MasterResource().getTables(), new TablesResource().getTables());
--- End diff --
Make `getTables()` on `MasterResources` and `TablesResources` static methods. They look stateless, so the static method could just invoke the existing non-static method on a static final singleton instance of the respective classes.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> That includes forwarding to a single monitor, multiple monitors, or none.
ACK. I didn't realize such an improvement had actually landed (instead of being a hypothetical as it has been). Nice to hear.
> > I think the simple solution would just be to return something that isn't "[]" with HTTP/200
> Agreed. That's basically what I thought I was describing, but with some informative message in the view when the AJAX receives this other "something".
Cool, I think that's the only concern I have on the matter. Makes sense to update that IT then as the implementation has changed. Thanks for bringing me up to speed. What was being presented didn't jive with my current level of understanding -- turns out that was just a bit stale.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110074283
--- Diff: assemble/pom.xml ---
@@ -195,6 +235,86 @@
<optional>true</optional>
</dependency>
<dependency>
+ <groupId>org.freemarker</groupId>
+ <artifactId>freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-utils</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>osgi-resource-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>aopalliance-repackaged</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>javax.inject</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.bundles.repackaged</groupId>
+ <artifactId>jersey-guava</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-jetty-http</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-client</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-common</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-server</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-entity-filtering</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-mvc</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-mvc-freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.media</groupId>
+ <artifactId>jersey-media-jaxb</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.media</groupId>
+ <artifactId>jersey-media-json-jackson</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.javassist</groupId>
+ <artifactId>javassist</artifactId>
--- End diff --
Do we have an explicit dependency on this? I would have expected to see this transitively brought in.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110074670
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java ---
@@ -778,77 +777,51 @@ public static long getStartTime() {
}
public static List<Pair<Long,Double>> getLoadOverTime() {
- synchronized (loadOverTime) {
- return new ArrayList<>(loadOverTime);
- }
+ return new ArrayList<>(loadOverTime);
--- End diff --
Nice simplification on these.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110074838
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLInformation.java ---
@@ -0,0 +1,115 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.xml.bind.annotation.XmlRootElement;
+
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerList;
+import org.apache.accumulo.monitor.rest.master.MasterInformation;
+import org.apache.accumulo.monitor.rest.tables.TableInformation;
+import org.apache.accumulo.monitor.rest.tables.TableInformationList;
+import org.apache.accumulo.monitor.rest.tables.TableNamespace;
+import org.apache.accumulo.monitor.rest.tables.TablesList;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServers;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerList;
+import org.apache.accumulo.monitor.rest.tservers.ServersShuttingDown;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Generate XML summary of Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@XmlRootElement(name = "stats")
+public class XMLInformation {
+
+ // Variable names become JSON keys
+ public List<TabletServer> servers;
+
+ public String masterGoalState, masterState;
+
+ public BadTabletServers badTabletServers;
+ public ServersShuttingDown tabletServersShuttingDown;
+ public Integer unassignedTablets;
+ public DeadServerList deadTabletServers;
+
+ public DeadLoggerList deadLoggers;
+
+ public TableInformationList tables;
+
+ public Totals totals;
+
+ public XMLInformation() {
+ servers = new ArrayList<>();
+ }
+
+ /**
+ * Stores Monitor information as XML
+ *
+ * @param size
+ * Number of tservers
+ * @param info
+ * Master information
+ * @param tablesList
+ * Table list
+ */
+ public XMLInformation(int size, MasterInformation info, TablesList tablesList) {
+ this.servers = new ArrayList<>(size);
+
+ this.masterGoalState = info.masterGoalState;
+ this.masterState = info.masterState;
+
+ this.badTabletServers = info.badTabletServers;
+ this.tabletServersShuttingDown = info.tabletServersShuttingDown;
+ this.unassignedTablets = info.unassignedTablets;
+ this.deadTabletServers = info.deadTabletServers;
+ this.deadLoggers = info.deadLoggers;
+
+ getTableInformationList(tablesList.tables);
+
+ this.totals = new Totals(info.ingestrate, info.queryrate, info.numentries);
+ }
+
+ /**
+ * Adds a new tablet to the XML
+ *
+ * @param tablet
+ * Tablet to add
+ */
+ public void addTablet(TabletServer tablet) {
--- End diff --
s/addTablet/addTabletServer/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075324
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/gc/GarbageCollectorStatus.java ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.gc;
+
+import org.apache.accumulo.core.gc.thrift.GCStatus;
+
+/**
+ *
+ * Responsible for grouping files and wals into a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+public class GarbageCollectorStatus {
+
+ public static final GarbageCollectorStatus EMPTY = new GarbageCollectorStatus();
+
+ // variable names become JSON key
+ public GarbageCollection files = new GarbageCollection();
--- End diff --
When invoking the constructor which takes a `GCStatus` , you're instantiating two extra GarbageCollection objects unnecessarily. Move these into the no-args constructor if you don't want `files` and `wals` to be null.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
Thanks for the feedback @joshelser, I'll start working on it ASAP. Here is an answer for the general comments you left outside the code:
- **I see some pojo classes have @XmlAttribute on the member variables, and others don't? Why the inconsistency (when the name isn't being overridden)?** The @XmlAttribute only shows up on some of the XML tags which have a different name than the attribute, that's why not all of the variables have it.
- **Might just be me, but I'd prefer to see consistency on the variable declaration (one per line) and only set an initial value when it doesn't involve instantiating a new object. I don't remember what Accumulo's "style" actually says.** I'll look into what the style says, but I agree with you, I think I started doing it for some reason and then kept doing it out of habit, but I can change it to your suggestion if the Accumulo "style" agrees.
- **All of those pojos also leave the member variables public. Can we not encapsulate them and expose them with getters? I am pretty sure that Jackson supports this.** It does, I changed to use the public variables at the suggestion of @ctubbsii, but we could discuss the best approach and change it to that.
- **Some minor whitespace things. Extra newlines at the start of a method body which are just bloat.
Any cross-browser testing? e.g. FF, Chrome, IE?** I tried using the code templates for Accumulo and running verify, so that might have escaped me. I tested on both FF and Chrome, I couldn't get the proxy to work for IE so I haven't been able to test it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110473250
--- Diff: server/monitor/src/main/resources/resources/master.js ---
@@ -0,0 +1,261 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates master initial table
+ */
+$(document).ready(function() {
+ createHeader();
+ doBanner('masterBanner', 'danger', 'Master Server Not Running');
+ refreshMaster();
+
+ // Create tooltip for table column information
+ $(document).tooltip();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshMaster() {
+ $.ajaxSetup({
+ async: false
+ });
+ getMaster();
+ getRecoveryList();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshMasterTable();
+ recoveryList();
+}
+
+
+/*
+ * The tables refresh function will do this functionality
+ * If tables are removed from Master, uncomment this function
+ */
+/**
+ * Used to redraw the page
+ */
+/*function refresh() {
+ refreshMaster();
+}*/
+
+/**
+ * Creates recovery list table
+ */
+function recoveryList() {
+ /*
+ * Get the recovery value obtained earlier,
+ * if it doesn't exists, create an empty array
+ */
+ var data = sessionStorage.recoveryList === undefined ?
+ [] : JSON.parse(sessionStorage.recoveryList);
+
+ $('#recoveryList tr').remove();
+
+ // If there is no recovery list data, hide the table
+ if (data.length === 0 || data.recoveryList.length === 0) {
+ $('#recoveryList').hide();
+ } else {
+ $('#recoveryList').show();
+
+ var caption = [];
+
+ caption.push('<span class="table-caption">Log Recovery</span><br />');
+ caption.push('<span class="table-subcaption">Some tablets were unloaded' +
+ ' in an unsafe manner. Write-ahead logs are being' +
+ ' recovered.</span><br />');
+
+ $('<caption/>', {
+ html: caption.join('')
+ }).appendTo('#recoveryList');
+
+ var items = [];
+
+ /*
+ * Create the header for the recovery list table
+ * Adds the columns, add sortTable function on click,
+ * if the column has a description, add title taken from the global.js
+ */
+ items.push('<th class="firstcell" onclick="sortTable(0)">' +
+ 'Server </th>');
+ items.push('<th onclick="sortTable(1)">Log </th>');
+ items.push('<th onclick="sortTable(2)">Time </th>');
+ items.push('<th onclick="sortTable(3)">Copy/Sort </th>');
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#recoveryList');
+
+ // Creates the table for the recovery list
+ $.each(data.recoveryList, function(key, val) {
+ var items = [];
+ items.push('<td class="firstcell left" data-value="' + val.server + '">' +
+ val.server + '</td>');
+ items.push('<td class="right" data-value="' + val.log + '">' + val.log +
+ '</td>');
+ var date = new Date(parseInt(val.time));
+ date = date.toLocaleString().split(' ').join(' ');
+ items.push('<td class="right" data-value="' + val.time + '">' + date +
+ '</td>');
+ items.push('<td class="right" data-value="' + val.copySort + '">' +
+ val.copySort + '</td>');
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#recoveryList');
+ });
+ }
+}
+
+/**
+ * Generates the master table
+ */
+function refreshMasterTable() {
+ // Gets the master status
+ var status = JSON.parse(sessionStorage.status).masterStatus;
+
+ // Hide the banner and the master table
+ $('#masterBanner').hide();
+ $('#masterStatus tr:gt(0)').remove();
+ $('#masterStatus').hide();
+
+ // If master status is error, show banner, otherwise, create master table
+ if (status === 'ERROR') {
+ $('#masterBanner').show();
+ } else {
+ $('#masterStatus').show();
+ var data = JSON.parse(sessionStorage.master);
+ var items = [];
+ items.push('<td class="firstcell left" data-value="' + data.master +
+ '">' + data.master + '</td>');
+
+ items.push('<td class="right" data-value="' + data.onlineTabletServers +
+ '">' + data.onlineTabletServers + '</td>');
+
+ items.push('<td class="right" data-value="' + data.totalTabletServers +
+ '">' + data.totalTabletServers + '</td>');
+
+ var date = new Date(parseInt(data.lastGC));
+ date = date.toLocaleString().split(' ').join(' ');
+ items.push('<td class="left" data-value="' + data.lasGC +
+ '"><a href="/gc">' + date + '</a></td>');
+
+ items.push('<td class="right" data-value="' + data.tablets +
+ '">' + bigNumberForQuantity(data.tablets) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.unassignedTablets +
+ '">' + bigNumberForQuantity(data.unassignedTablets) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.numentries +
+ '">' + bigNumberForQuantity(data.numentries) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.ingestrate +
+ '">' + bigNumberForQuantity(Math.round(data.ingestrate)) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.entriesRead +
+ '">' + bigNumberForQuantity(Math.round(data.entriesRead)) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.queryrate +
+ '">' + bigNumberForQuantity(Math.round(data.queryrate)) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.holdTime +
+ '">' + timeDuration(data.holdTime) + '</td>');
+
+ items.push('<td class="right" data-value="' + data.osload +
+ '">' + bigNumberForQuantity(data.osload) + '</td>');
--- End diff --
You should create a function to clean this up. Something simple like createTableCell(dataValue, prettyValue) would help a lot.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
Haha alright, thanks for all the help!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110467050
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/RecentTracesInformation.java ---
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+
+/**
+ *
+ * Generates a recent trace
+ *
+ * @since 2.0.0
+ *
+ */
+public class RecentTracesInformation {
+
+ // Variable names become JSON keys
+ public String type;
+ public Long avg;
+
+ public int total = 0;
+
+ public long min = Long.MAX_VALUE, max = Long.MIN_VALUE;
+ private long totalMS = 0l;
+ public long histogram[] = new long[] {0l, 0l, 0l, 0l, 0l, 0l};
--- End diff --
Nit: Capitol L is easier to read
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110439061
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/Totals.java ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+/**
+ *
+ * Generates the totals for XML summary
+ *
+ * @since 2.0.0
+ *
+ */
+public class Totals {
+
+ // Variable names become JSON keys
+ public double ingestrate, queryrate, diskrate = 0.0;
--- End diff --
I am looking at the code and I believe that the reason these are not Camel-cased is that on the original Monitor's XML response they are not Camel-cased, and as such I kept it that way for backwards compatibility.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110466878
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,269 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+
+ /*
+ * Add the tables that have the selected namespace Asterisk = All namespaces Hyphen = Default namespace
+ */
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
+ tableNamespace.addTable(new TableNamespace(key));
+ }
+ }
+
+ return generateTables(tableNamespace);
+ }
+
+ /**
+ * Generates a table list based on the list of namespaces
+ *
+ * @param tableNamespace
+ * Namespace list
+ * @return Table list
+ */
+ private TablesList generateTables(TablesList tableNamespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add tables to the list
+ for (Entry<String,String> entry : Tables.getNameToIdMap(HdfsZooInstance.getInstance()).entrySet()) {
+ String tableName = entry.getKey(), tableId = entry.getValue();
+ TableInfo tableInfo = tableStats.get(tableName);
+ if (null != tableInfo) {
+ Double holdTime = compactingByTable.get(tableId);
+ if (holdTime == null)
+ holdTime = Double.valueOf(0.);
+
+ for (TableNamespace name : tableNamespace.tables) {
+ // Check if table has the default namespace
+ if (!tableName.contains(".") && name.namespace.equals("")) {
+ name.addTable(new TableInformation(tableName, tableId, tableInfo, holdTime, tableManager.getTableState(tableId).name()));
+ } else if (tableName.startsWith(name.namespace + ".")) {
+ name.addTable(new TableInformation(tableName, tableId, tableInfo, holdTime, tableManager.getTableState(tableId).name()));
+ }
+ }
+ tables.add(new TableInformation(tableName, tableId, tableInfo, holdTime, tableManager.getTableState(tableId).name()));
+ } else {
+ for (TableNamespace name : tableNamespace.tables) {
+ if (!tableName.contains(".") && name.namespace.equals("")) {
+ name.addTable(new TableInformation(tableName, tableId, tableManager.getTableState(tableId).name()));
+ } else if (tableName.startsWith(name.namespace + ".")) {
+ name.addTable(new TableInformation(tableName, tableId, tableManager.getTableState(tableId).name()));
+ }
+ }
+ tables.add(new TableInformation(tableName, tableId, tableManager.getTableState(tableId).name()));
+ }
+ }
+
+ return tableNamespace;
+ }
+
+ /**
+ * Generates a list of all the tables
+ *
+ * @return list with all tables
+ */
+ @GET
+ public TablesList getTables() {
+ return generateTables("*");
+ }
+
+ /**
+ * Generates a list with the selected namespace
+ *
+ * @param namespace
+ * Namespace to filter tables
+ * @return list with selected tables
+ */
+ @GET
+ @Path("namespace/{namespace}")
+ public TablesList getTable(@PathParam("namespace") String namespace) {
+ return generateTables(namespace);
+ }
+
+ /**
+ * Generates a list with the list of namespaces
+ *
+ * @param namespaceList
+ * List of namespaces separated by a comma
+ * @return list with selected tables
+ */
+ @GET
+ @Path("namespaces/{namespaces}")
+ public TablesList getTableWithNamespace(@PathParam("namespaces") String namespaceList) {
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ /*
+ * Add the tables that have the selected namespace Asterisk = All namespaces Hyphen = Default namespace
+ */
+ for (String namespace : namespaceList.split(",")) {
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
+ tableNamespace.addTable(new TableNamespace(key));
+ }
+ }
+ }
+
+ return generateTables(tableNamespace);
+ }
+
+ /**
+ * Generates a list of participating tservers for a table
+ *
+ * @param tableId
+ * Table ID to find participating tservers
+ * @return List of participating tservers
+ */
+ @Path("{tableId}")
+ @GET
+ public TabletServers getParticipatingTabletServers(@PathParam("tableId") String tableId) throws Exception {
+ Instance instance = Monitor.getContext().getInstance();
+
+ TreeSet<String> locs = new TreeSet<>();
+ if (RootTable.ID.equals(tableId)) {
+ locs.add(instance.getRootTabletLocation());
+ } else {
+ String systemTableName = MetadataTable.ID.equals(tableId) ? RootTable.NAME : MetadataTable.NAME;
+ MetaDataTableScanner scanner = new MetaDataTableScanner(Monitor.getContext(), new Range(KeyExtent.getMetadataEntry(tableId, new Text()),
+ KeyExtent.getMetadataEntry(tableId, null)), systemTableName);
+
+ while (scanner.hasNext()) {
+ TabletLocationState state = scanner.next();
+ if (state.current != null) {
+ try {
+ locs.add(state.current.hostPort());
+ } catch (Exception ex) {}
+ }
+ }
+ scanner.close();
+ }
+
+ TabletServers tabletServers = new TabletServers(Monitor.getMmi().tServerInfo.size());
+
+ List<TabletServerStatus> tservers = new ArrayList<>();
+ if (Monitor.getMmi() != null) {
+ for (TabletServerStatus tss : Monitor.getMmi().tServerInfo) {
+ try {
+ if (tss.name != null && locs.contains(tss.name))
+ tservers.add(tss);
+ } catch (Exception ex) {
--- End diff --
Do something or throw
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111168269
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java ---
@@ -1,285 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.accumulo.monitor.servlets;
-
-import static java.nio.charset.StandardCharsets.UTF_8;
-
-import java.io.IOException;
-import java.io.PrintWriter;
-import java.io.StringWriter;
-import java.io.UnsupportedEncodingException;
-import java.net.HttpURLConnection;
-import java.net.URLDecoder;
-import java.net.URLEncoder;
-import java.util.Date;
-import java.util.List;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.Cookie;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import org.apache.accumulo.core.Constants;
-import org.apache.accumulo.core.conf.Property;
-import org.apache.accumulo.monitor.Monitor;
-import org.apache.accumulo.server.monitor.DedupedLogEvent;
-import org.apache.accumulo.server.monitor.LogService;
-import org.apache.log4j.Level;
-import org.apache.log4j.Logger;
-
-abstract public class BasicServlet extends HttpServlet {
- public static final String STANDBY_MONITOR_MESSAGE = "This is not the active Monitor";
-
- private static final long serialVersionUID = 1L;
- protected static final Logger log = Logger.getLogger(BasicServlet.class);
- private String bannerText;
- private String bannerColor;
- private String bannerBackground;
-
- abstract protected String getTitle(HttpServletRequest req);
-
- public boolean isActiveMonitor() {
- // If the HighlyAvailableService is not initialized or it's not the active service, throw an exception
- // to prevent processing of the servlet.
- if (null == Monitor.HA_SERVICE_INSTANCE || !Monitor.HA_SERVICE_INSTANCE.isActiveService()) {
- return false;
- }
- return true;
- }
-
- @Override
- public void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
- // Verify that this is the active Monitor instance
- if (!isActiveMonitor()) {
- resp.sendError(HttpURLConnection.HTTP_UNAVAILABLE, STANDBY_MONITOR_MESSAGE);
- return;
- }
- StringBuilder sb = new StringBuilder();
- try {
- Monitor.fetchData();
- bannerText = sanitize(Monitor.getContext().getConfiguration().get(Property.MONITOR_BANNER_TEXT));
--- End diff --
Thanks for pointing this out.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110254019
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
--- End diff --
> it may help prioritize giving it a real name.
Other than "" (empty string), you mean? IMO, I actually kind of like that. HBase uses "default", but then it comes with this baggage of when is it "default" and when is it "".
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111439164
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
+ }
+
+ // Obtains the hours, if time is 0, return hours, minutes, and units
+ hr = time % 24;
+ time = Math.floor(time / 24);
+ if (time == 0) {
+ return hr + 'h' + ' ' + min + 'm';
+ }
+
+ // Obtains the days, if time is 0, return days, hours, and units
+ day = time % 365;
+ time = Math.floor(time / 365);
+ if (time == 0) {
+ return day + 'd' + ' ' + hr + 'h';
+ }
+
+ // Obtains the years, if time is 0, return years, days, and units
+ yr = Math.floor(time);
+ return yr + 'y' + ' ' + day + 'd';
+}
+
+/**
+ * Sorts the selected table by column in the direction chosen
+ *
+ * @param {string} tableID Table to sort
+ * @param {string} direction Direction to sort table, asc or desc
+ * @param {number} n Column to sort
+ */
+function sortTables(tableID, direction, n) {
+ var table, rows, switching, i, x, y, h, shouldSwitch, dir, xFinal, yFinal;
+ table = document.getElementById(tableID);
+ switching = true;
+
+ dir = direction;
+ sessionStorage.direction = dir;
+
+ // Select the rows of the table
+ rows = table.getElementsByTagName('tr');
+
+ // Clears the sortable class from the table columns
+ var count = 0;
+ while (rows[0].getElementsByTagName('th').length > count) {
+ var tmpH = rows[0].getElementsByTagName('th')[count];
+ tmpH.classList.remove('sortable');
+ if (rows.length > 2) {
+ tmpH.classList.add('sortable');
+ }
+ $(tmpH.getElementsByTagName('span')).remove();
+ count += 1;
+ }
+
+ // If there are more than 2 rows, add arrow to the selected column
+ if (rows.length <= 2) {
+ switching = false;
+ } else {
+ h = rows[0].getElementsByTagName('th')[n];
+ if (dir == 'asc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-up"' +
+ ' width="10px" height="10px" />');
+ } else if (dir == 'desc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-down"' +
+ ' width="10px" height="10px" />');
+ }
+ }
+
+ /*
+ * Make a loop that will continue until
+ * no switching has been done:
+ */
+ while (switching) {
+ switching = false;
+ rows = table.getElementsByTagName('tr');
+
+ /*
+ * Loop through all table rows (except the
+ * first, which contains table headers):
+ */
+ for (i = 1; i < (rows.length - 1); i++) {
+ shouldSwitch = false;
+ /*
+ * Get two elements to compare,
+ * one from current row and one from the next:
+ * If the element is a dash, convert to null, otherwise,
+ * if it is a string, convert to number
+ */
+ x = rows[i].getElementsByTagName('td')[n].getAttribute('data-value');
+ xFinal = (x === '-' || x === '—' ?
+ null : (Number(x) == x ? Number(x) : x));
+
+ y = rows[i + 1].getElementsByTagName('td')[n].getAttribute('data-value');
+ yFinal = (y === '-' || y === '—' ?
+ null : (Number(y) == y ? Number(y) : y));
+
+ /*
+ * Check if the two rows should switch place,
+ * based on the direction, asc or desc:
+ */
+ if (dir == 'asc') {
+ if (xFinal > yFinal || (xFinal !== null && yFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ } else if (dir == 'desc') {
+ if (xFinal < yFinal || (yFinal !== null && xFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ }
+ }
+ if (shouldSwitch) {
+ /*
+ * If a switch has been marked, make the switch
+ * and mark that a switch has been done:
+ */
+ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
+ switching = true;
+ }
+ }
+}
+
+/**
+ * Clears the selected table while leaving the headers
+ *
+ * @param {string} tableID Table to clear
+ */
+function clearTable(tableID) {
+ // JQuery selector to select all rows except for the first row (header)
+ $('#' + tableID).find('tr:not(:first)').remove();
+}
+
+///// REST Calls /////////////
+
+/**
+ * REST GET call for the master information,
+ * stores it on a sessionStorage variable
+ */
+function getMaster() {
+ $.getJSON('/rest/master', function(data) {
+ sessionStorage.master = JSON.stringify(data);
--- End diff --
@lstav and I tested some failure cases, and I can't remember the details, but I believe we don't get an error message, but the function is not called, and the data structures are not updated. I'm pretty sure the existing views handle this situation well... but we can push it through some harsher edge case testing later, I think.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110465276
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/status/StatusResource.java ---
@@ -0,0 +1,116 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.status;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.master.thrift.DeadServer;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.monitor.DedupedLogEvent;
+import org.apache.accumulo.server.monitor.LogService;
+import org.apache.log4j.Level;
+
+/**
+ *
+ * Generates the status for master, gc, and tservers as well as log and problem reports
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/status")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class StatusResource {
+
+ /**
+ * Generates the JSON object with the status
+ *
+ * @return Status report
+ */
+ @GET
+ public StatusInformation getTables() {
+
+ StatusInformation status;
+ String masterStatus;
+ String gcStatus;
+ String tServerStatus = "ERROR";
--- End diff --
Didn't think of that, thanks for the suggestion!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075081
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/XMLResource.java ---
@@ -0,0 +1,67 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.monitor.rest.tables.TablesResource;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+
+/**
+ *
+ * Responsible for generating an XML summary of the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class XMLResource {
+
+ /**
+ * Generates an XML summary of the Monitor
+ *
+ * @return XML summary
+ */
+ @GET
+ public XMLInformation getXMLInformation() {
+
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ // Add Monitor information
+ XMLInformation xml = new XMLInformation(mmi.tServerInfo.size(), new MasterResource().getTables(), new TablesResource().getTables());
+
+ // Add tserver information
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ xml.addTablet(new TabletServer(status));
--- End diff --
Calling the metrics object TabletServer is a bit confusing to me. It's metrics/information, not the actual TabletServer instance (which already exists).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110161076
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
+ }
+
+ // Obtains the hours, if time is 0, return hours, minutes, and units
+ hr = time % 24;
+ time = Math.floor(time / 24);
+ if (time == 0) {
+ return hr + 'h' + ' ' + min + 'm';
+ }
+
+ // Obtains the days, if time is 0, return days, hours, and units
+ day = time % 365;
+ time = Math.floor(time / 365);
+ if (time == 0) {
+ return day + 'd' + ' ' + hr + 'h';
+ }
+
+ // Obtains the years, if time is 0, return years, days, and units
+ yr = Math.floor(time);
+ return yr + 'y' + ' ' + day + 'd';
+}
+
+/**
+ * Sorts the selected table by column in the direction chosen
+ *
+ * @param {string} tableID Table to sort
+ * @param {string} direction Direction to sort table, asc or desc
+ * @param {number} n Column to sort
+ */
+function sortTables(tableID, direction, n) {
+ var table, rows, switching, i, x, y, h, shouldSwitch, dir, xFinal, yFinal;
+ table = document.getElementById(tableID);
+ switching = true;
+
+ dir = direction;
+ sessionStorage.direction = dir;
+
+ // Select the rows of the table
+ rows = table.getElementsByTagName('tr');
+
+ // Clears the sortable class from the table columns
+ var count = 0;
+ while (rows[0].getElementsByTagName('th').length > count) {
+ var tmpH = rows[0].getElementsByTagName('th')[count];
+ tmpH.classList.remove('sortable');
+ if (rows.length > 2) {
+ tmpH.classList.add('sortable');
+ }
+ $(tmpH.getElementsByTagName('span')).remove();
+ count += 1;
+ }
+
+ // If there are more than 2 rows, add arrow to the selected column
+ if (rows.length <= 2) {
+ switching = false;
+ } else {
+ h = rows[0].getElementsByTagName('th')[n];
+ if (dir == 'asc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-up"' +
+ ' width="10px" height="10px" />');
+ } else if (dir == 'desc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-down"' +
+ ' width="10px" height="10px" />');
+ }
+ }
+
+ /*
+ * Make a loop that will continue until
+ * no switching has been done:
+ */
+ while (switching) {
+ switching = false;
+ rows = table.getElementsByTagName('tr');
+
+ /*
+ * Loop through all table rows (except the
+ * first, which contains table headers):
+ */
+ for (i = 1; i < (rows.length - 1); i++) {
+ shouldSwitch = false;
+ /*
+ * Get two elements to compare,
+ * one from current row and one from the next:
+ * If the element is a dash, convert to null, otherwise,
+ * if it is a string, convert to number
+ */
+ x = rows[i].getElementsByTagName('td')[n].getAttribute('data-value');
+ xFinal = (x === '-' || x === '—' ?
+ null : (Number(x) == x ? Number(x) : x));
+
+ y = rows[i + 1].getElementsByTagName('td')[n].getAttribute('data-value');
+ yFinal = (y === '-' || y === '—' ?
+ null : (Number(y) == y ? Number(y) : y));
+
+ /*
+ * Check if the two rows should switch place,
+ * based on the direction, asc or desc:
+ */
+ if (dir == 'asc') {
+ if (xFinal > yFinal || (xFinal !== null && yFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ } else if (dir == 'desc') {
+ if (xFinal < yFinal || (yFinal !== null && xFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ }
+ }
+ if (shouldSwitch) {
+ /*
+ * If a switch has been marked, make the switch
+ * and mark that a switch has been done:
+ */
+ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
+ switching = true;
+ }
+ }
+}
+
+/**
+ * Clears the selected table while leaving the headers
+ *
+ * @param {string} tableID Table to clear
+ */
+function clearTable(tableID) {
+ // JQuery selector to select all rows except for the first row (header)
+ $('#' + tableID).find('tr:not(:first)').remove();
+}
+
+///// REST Calls /////////////
+
+/**
+ * REST GET call for the master information,
+ * stores it on a sessionStorage variable
+ */
+function getMaster() {
+ $.getJSON('/rest/master', function(data) {
+ sessionStorage.master = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the zookeeper information,
+ * stores it on a sessionStorage variable
+ */
+function getZK() {
+ $.getJSON('/rest/zk', function(data) {
+ sessionStorage.zk = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the namespaces, stores it on a global variable
+ */
+function getNamespaces() {
+ $.getJSON('/rest/tables/namespaces', function(data) {
+ NAMESPACES = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tables on each namespace,
+ * stores it on a sessionStorage variable
+ *
+ * @param {array} namespaces Array holding the selected namespaces
+ */
+function getNamespaceTables(namespaces) {
+
+ // Creates a JSON object to store the tables
+ var jsonObj = {};
+ jsonObj.tables = [];
+
+ /* If the namespace array include *, get all tables, otherwise,
+ * get tables from specific namespaces
+ */
+ if (namespaces.indexOf('*') != -1) {
+ getTables();
+ } else {
+ $.each(namespaces, function(key, val) {
+ /* Makes the rest call for each of the namespaces in the array,
+ * stores them on the JSON object
+ */
+ if (val !== '*') {
+ var call = '/rest/tables/namespace/' + val;
+ $.getJSON(call, function(data) {
+ $.each(data.tables, function(key2, val2) {
+ jsonObj.tables.push(val2);
+ });
+ });
+ }
+ });
+ sessionStorage.tables = JSON.stringify(jsonObj);
+ }
+}
+
+/**
+ * REST GET call for the tables, stores it on a sessionStorage variable
+ */
+function getTables() {
+ $.getJSON('/rest/tables', function(data) {
+ sessionStorage.tables = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear a specific dead server
+ *
+ * @param {string} server Dead Server ID
+ */
+function clearDeadServers(server) {
+ var call = '/rest/tservers?server=' + server;
+ $.post(call);
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ */
+function getTServers() {
+ $.getJSON('/rest/tservers', function(data) {
+ sessionStorage.tservers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ *
+ * @param {string} server Server ID
+ */
+function getTServer(server) {
+ var call = '/rest/tservers/' + server;
+ $.getJSON(call, function(data) {
+ sessionStorage.server = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the scans, stores it on a sessionStorage variable
+ */
+function getScans() {
+ $.getJSON('/rest/scans', function(data) {
+ sessionStorage.scans = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the bulk imports, stores it on a sessionStorage variable
+ */
+function getBulkImports() {
+ $.getJSON('/rest/bulkImports', function(data) {
+ sessionStorage.bulkImports = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the garbage collector,
+ * stores it on a sessionStorage variable
+ */
+function getGarbageCollector() {
+ $.getJSON('/rest/gc', function(data) {
+ sessionStorage.gc = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the server stats, stores it on a sessionStorage variable
+ */
+function getServerStats() {
+ $.getJSON('/rest/tservers/serverStats', function(data) {
+ sessionStorage.serverStats = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the recovery list, stores it on a sessionStorage variable
+ */
+function getRecoveryList() {
+ $.getJSON('/rest/tservers/recovery', function(data) {
+ sessionStorage.recoveryList = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the participating tablet servers,
+ * stores it on a sessionStorage variable
+ *
+ * @param {string} table Table ID
+ */
+function getTableServers(table) {
+ var call = '/rest/tables/' + table;
+ $.getJSON(call, function(data) {
+ sessionStorage.tableServers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace summary, stores it on a sessionStorage variable
+ *
+ * @param {string} minutes Number of minutes to display trace summary
+ */
+function getTraceSummary(minutes) {
+ var call = '/rest/trace/summary/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceSummary = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace type, stores it on a sessionStorage variable
+ *
+ * @param {string} type Type of the trace
+ * @param {string} minutes Number of minutes to display trace
+ */
+function getTraceOfType(type, minutes) {
+ var call = '/rest/trace/listType/' + type + '/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceType = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace id, stores it on a sessionStorage variable
+ *
+ * @param {string} id Trace ID
+ */
+function getTraceShow(id) {
+ var call = '/rest/trace/show/' + id;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceShow = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the logs, stores it on a sessionStorage variable
+ */
+function getLogs() {
+ $.getJSON('/rest/logs', function(data) {
+ sessionStorage.logs = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear logs
+ */
+function clearLogs() {
+ $.post('/rest/logs');
+}
+
+/**
+ * REST GET call for the problems
+ */
+function getProblems() {
+ getProblemSummary();
+ getProblemDetails();
+}
+
+/**
+ * REST POST call to clear all table problems
+ *
+ * @param {string} tableID Table ID
+ */
+function clearTableProblems(tableID) {
+ var call = '/rest/problems/summary?s=' + tableID;
+ // Change plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
+ $.post(call);
+}
+
+/**
+ * REST POST call to clear detail problems
+ *
+ * @param {string} table Table ID
+ * @param {string} resource Resource for problem
+ * @param {string} type Type of problem
+ */
+function clearDetailsProblems(table, resource, type) {
+ var call = '/rest/problems/details?table=' + table + '&resource=' +
+ resource + '&ptype=' + type;
+ // Changes plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
--- End diff --
Good idea! I didn't think of it at the time.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
@milleruntime The only standby behavior we want to retain is that of the log receiver. The active monitor registers itself as the log receiver in ZK, but the standbys shouldn't. Everything else should be the same.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110253032
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
+ }
+
+ // Obtains the hours, if time is 0, return hours, minutes, and units
+ hr = time % 24;
+ time = Math.floor(time / 24);
+ if (time == 0) {
+ return hr + 'h' + ' ' + min + 'm';
+ }
+
+ // Obtains the days, if time is 0, return days, hours, and units
+ day = time % 365;
+ time = Math.floor(time / 365);
+ if (time == 0) {
+ return day + 'd' + ' ' + hr + 'h';
+ }
+
+ // Obtains the years, if time is 0, return years, days, and units
+ yr = Math.floor(time);
+ return yr + 'y' + ' ' + day + 'd';
+}
+
+/**
+ * Sorts the selected table by column in the direction chosen
+ *
+ * @param {string} tableID Table to sort
+ * @param {string} direction Direction to sort table, asc or desc
+ * @param {number} n Column to sort
+ */
+function sortTables(tableID, direction, n) {
+ var table, rows, switching, i, x, y, h, shouldSwitch, dir, xFinal, yFinal;
+ table = document.getElementById(tableID);
+ switching = true;
+
+ dir = direction;
+ sessionStorage.direction = dir;
+
+ // Select the rows of the table
+ rows = table.getElementsByTagName('tr');
+
+ // Clears the sortable class from the table columns
+ var count = 0;
+ while (rows[0].getElementsByTagName('th').length > count) {
+ var tmpH = rows[0].getElementsByTagName('th')[count];
+ tmpH.classList.remove('sortable');
+ if (rows.length > 2) {
+ tmpH.classList.add('sortable');
+ }
+ $(tmpH.getElementsByTagName('span')).remove();
+ count += 1;
+ }
+
+ // If there are more than 2 rows, add arrow to the selected column
+ if (rows.length <= 2) {
+ switching = false;
+ } else {
+ h = rows[0].getElementsByTagName('th')[n];
+ if (dir == 'asc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-up"' +
+ ' width="10px" height="10px" />');
+ } else if (dir == 'desc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-down"' +
+ ' width="10px" height="10px" />');
+ }
+ }
+
+ /*
+ * Make a loop that will continue until
+ * no switching has been done:
+ */
+ while (switching) {
+ switching = false;
+ rows = table.getElementsByTagName('tr');
+
+ /*
+ * Loop through all table rows (except the
+ * first, which contains table headers):
+ */
+ for (i = 1; i < (rows.length - 1); i++) {
+ shouldSwitch = false;
+ /*
+ * Get two elements to compare,
+ * one from current row and one from the next:
+ * If the element is a dash, convert to null, otherwise,
+ * if it is a string, convert to number
+ */
+ x = rows[i].getElementsByTagName('td')[n].getAttribute('data-value');
+ xFinal = (x === '-' || x === '—' ?
+ null : (Number(x) == x ? Number(x) : x));
+
+ y = rows[i + 1].getElementsByTagName('td')[n].getAttribute('data-value');
+ yFinal = (y === '-' || y === '—' ?
+ null : (Number(y) == y ? Number(y) : y));
+
+ /*
+ * Check if the two rows should switch place,
+ * based on the direction, asc or desc:
+ */
+ if (dir == 'asc') {
+ if (xFinal > yFinal || (xFinal !== null && yFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ } else if (dir == 'desc') {
+ if (xFinal < yFinal || (yFinal !== null && xFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ }
+ }
+ if (shouldSwitch) {
+ /*
+ * If a switch has been marked, make the switch
+ * and mark that a switch has been done:
+ */
+ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
+ switching = true;
+ }
+ }
+}
+
+/**
+ * Clears the selected table while leaving the headers
+ *
+ * @param {string} tableID Table to clear
+ */
+function clearTable(tableID) {
+ // JQuery selector to select all rows except for the first row (header)
+ $('#' + tableID).find('tr:not(:first)').remove();
+}
+
+///// REST Calls /////////////
+
+/**
+ * REST GET call for the master information,
+ * stores it on a sessionStorage variable
+ */
+function getMaster() {
+ $.getJSON('/rest/master', function(data) {
+ sessionStorage.master = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the zookeeper information,
+ * stores it on a sessionStorage variable
+ */
+function getZK() {
+ $.getJSON('/rest/zk', function(data) {
+ sessionStorage.zk = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the namespaces, stores it on a global variable
+ */
+function getNamespaces() {
+ $.getJSON('/rest/tables/namespaces', function(data) {
+ NAMESPACES = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tables on each namespace,
+ * stores it on a sessionStorage variable
+ *
+ * @param {array} namespaces Array holding the selected namespaces
+ */
+function getNamespaceTables(namespaces) {
+
+ // Creates a JSON object to store the tables
+ var jsonObj = {};
+ jsonObj.tables = [];
+
+ /* If the namespace array include *, get all tables, otherwise,
+ * get tables from specific namespaces
+ */
+ if (namespaces.indexOf('*') != -1) {
+ getTables();
+ } else {
+ $.each(namespaces, function(key, val) {
+ /* Makes the rest call for each of the namespaces in the array,
+ * stores them on the JSON object
+ */
+ if (val !== '*') {
+ var call = '/rest/tables/namespace/' + val;
+ $.getJSON(call, function(data) {
+ $.each(data.tables, function(key2, val2) {
+ jsonObj.tables.push(val2);
+ });
+ });
+ }
+ });
+ sessionStorage.tables = JSON.stringify(jsonObj);
+ }
+}
+
+/**
+ * REST GET call for the tables, stores it on a sessionStorage variable
+ */
+function getTables() {
+ $.getJSON('/rest/tables', function(data) {
+ sessionStorage.tables = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear a specific dead server
+ *
+ * @param {string} server Dead Server ID
+ */
+function clearDeadServers(server) {
+ var call = '/rest/tservers?server=' + server;
+ $.post(call);
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ */
+function getTServers() {
+ $.getJSON('/rest/tservers', function(data) {
+ sessionStorage.tservers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the tservers, stores it on a sessionStorage variable
+ *
+ * @param {string} server Server ID
+ */
+function getTServer(server) {
+ var call = '/rest/tservers/' + server;
+ $.getJSON(call, function(data) {
+ sessionStorage.server = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the scans, stores it on a sessionStorage variable
+ */
+function getScans() {
+ $.getJSON('/rest/scans', function(data) {
+ sessionStorage.scans = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the bulk imports, stores it on a sessionStorage variable
+ */
+function getBulkImports() {
+ $.getJSON('/rest/bulkImports', function(data) {
+ sessionStorage.bulkImports = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the garbage collector,
+ * stores it on a sessionStorage variable
+ */
+function getGarbageCollector() {
+ $.getJSON('/rest/gc', function(data) {
+ sessionStorage.gc = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the server stats, stores it on a sessionStorage variable
+ */
+function getServerStats() {
+ $.getJSON('/rest/tservers/serverStats', function(data) {
+ sessionStorage.serverStats = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the recovery list, stores it on a sessionStorage variable
+ */
+function getRecoveryList() {
+ $.getJSON('/rest/tservers/recovery', function(data) {
+ sessionStorage.recoveryList = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the participating tablet servers,
+ * stores it on a sessionStorage variable
+ *
+ * @param {string} table Table ID
+ */
+function getTableServers(table) {
+ var call = '/rest/tables/' + table;
+ $.getJSON(call, function(data) {
+ sessionStorage.tableServers = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace summary, stores it on a sessionStorage variable
+ *
+ * @param {string} minutes Number of minutes to display trace summary
+ */
+function getTraceSummary(minutes) {
+ var call = '/rest/trace/summary/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceSummary = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace type, stores it on a sessionStorage variable
+ *
+ * @param {string} type Type of the trace
+ * @param {string} minutes Number of minutes to display trace
+ */
+function getTraceOfType(type, minutes) {
+ var call = '/rest/trace/listType/' + type + '/' + minutes;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceType = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the trace id, stores it on a sessionStorage variable
+ *
+ * @param {string} id Trace ID
+ */
+function getTraceShow(id) {
+ var call = '/rest/trace/show/' + id;
+ $.getJSON(call, function(data) {
+ sessionStorage.traceShow = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST GET call for the logs, stores it on a sessionStorage variable
+ */
+function getLogs() {
+ $.getJSON('/rest/logs', function(data) {
+ sessionStorage.logs = JSON.stringify(data);
+ });
+}
+
+/**
+ * REST POST call to clear logs
+ */
+function clearLogs() {
+ $.post('/rest/logs');
+}
+
+/**
+ * REST GET call for the problems
+ */
+function getProblems() {
+ getProblemSummary();
+ getProblemDetails();
+}
+
+/**
+ * REST POST call to clear all table problems
+ *
+ * @param {string} tableID Table ID
+ */
+function clearTableProblems(tableID) {
+ var call = '/rest/problems/summary?s=' + tableID;
+ // Change plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
+ $.post(call);
+}
+
+/**
+ * REST POST call to clear detail problems
+ *
+ * @param {string} table Table ID
+ * @param {string} resource Resource for problem
+ * @param {string} type Type of problem
+ */
+function clearDetailsProblems(table, resource, type) {
+ var call = '/rest/problems/details?table=' + table + '&resource=' +
+ resource + '&ptype=' + type;
+ // Changes plus sign to use ASCII value to send it as a URL query parameter
+ call = call.split('+').join('%2B');
--- End diff --
Optionally, could use PathParams instead of QueryParams (in some places, anyway).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on the issue:
https://github.com/apache/accumulo/pull/242
@ctubbsii @joshelser let me know what you think of 41a8b5a06085c444dfdafa5f4192f1ada05f724b
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110473929
--- Diff: server/monitor/src/main/resources/resources/problems.js ---
@@ -0,0 +1,232 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshProblems() {
+ $.ajaxSetup({
+ async: false
+ });
+ getProblems();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshProblemSummaryTable();
+ refreshProblemDetailsTable();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshProblems();
+}
+
+/**
+ * Makes REST POST call to clear the problem summary table
+ *
+ * @param {string} tableID Table ID to clear
+ */
+function clearTableProblemsTable(tableID) {
+ clearTableProblems(tableID);
+ refreshProblems();
+ refreshNavBar();
+}
+
+/**
+ * Makes REST POST call to clear the problem details table
+ *
+ * @param {string} table Table ID to clear
+ * @param {string} resource Resource of problem
+ * @param {string} type Type of problem
+ */
+function clearDetailsProblemsTable(table, resource, type) {
+ clearDetailsProblems(table, resource, type);
+ refreshProblems();
+ refreshNavBar();
+}
+
+/**
+ * Generates the problem summary table
+ */
+function refreshProblemSummaryTable() {
+ clearTable('problemSummary');
+ var data = sessionStorage.problemSummary === undefined ?
+ [] : JSON.parse(sessionStorage.problemSummary);
+
+ if (data.length === 0 || Object.keys(data.problemSummary).length === 0) {
+ var items = [];
+ items.push('<td class="center" colspan="5"><i>Empty</i></td>');
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#problemSummary');
+ } else {
+ $.each(data.problemSummary, function(key, val) {
+ var items = [];
+ items.push('<td class="firstcell left"><a href="/problems?table=' +
+ val.tableID.split('+').join('%2B') + '">' + val.tableName +
+ '</a></td>');
+
+ items.push('<td class="right">' + bigNumberForQuantity(val.fileRead) +
+ '</td>');
+
+ items.push('<td class="right">' + bigNumberForQuantity(val.fileWrite) +
+ '</td>');
+
+ items.push('<td class="right">' + bigNumberForQuantity(val.tableLoad) +
+ '</td>');
+ items.push('<td><a href="javascript:clearTableProblemsTable(\'' +
+ val.tableID + '\');">clear ALL ' + val.tableName +
+ ' problems</a></td>');
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#problemSummary');
+ });
+ }
+}
+
+/**
+ * Generates the problem details table
+ */
+function refreshProblemDetailsTable() {
+ clearTable('problemDetails');
+ var data = sessionStorage.problemDetails === undefined ?
+ [] : JSON.parse(sessionStorage.problemDetails);
+
+ if (data.length === 0 || Object.keys(data.problemDetails).length === 0) {
+ var items = [];
+ items.push('<td class="center" colspan="7"><i>Empty</i></td>');
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#problemDetails');
+ } else {
+ $.each(data.problemDetails, function(key, val) {
+ var items = [];
+ // Filters the details problems for the selected tableID
+ if (tableID === val.tableID || tableID === '') {
+ items.push('<td class="firstcell left" data-value="' + val.tableName +
+ '"><a href="/tables/' + val.tableID + '">' + val.tableName +
+ '</a></td>');
+
+ items.push('<td class="right" data-value="' + val.type + '">' +
+ val.type + '</td>');
+
+ items.push('<td class="right" data-value="' + val.server + '">' +
+ val.server + '</td>');
+
+ var date = new Date(val.time);
+ items.push('<td class="right" data-value="' + val.time + '">' +
+ date.toLocaleString() + '</td>');
+
+ items.push('<td class="right" data-value="' + val.resource + '">' +
+ val.resource + '</td>');
+
+ items.push('<td class="right" data-value="' + val.exception + '">' +
+ val.exception + '</td>');
+
+ items.push('<td><a href="javascript:clearDetailsProblemsTable(\'' +
+ val.tableID + '\', \'' + val.resource + '\', \'' + val.type +
+ '\')">clear this problem</a></td>');
--- End diff --
Another spot where a simple function that returns a table cell will help a lot
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076706
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
--- End diff --
E+15 instead of e15? etc
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110469952
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+ /**
+ * Generates a trace summary
+ *
+ * @param minutes
+ * Range of minutes to filter traces
+ * @return Trace summary in specified range
+ */
+ @Path("summary/{minutes}")
+ @GET
+ public RecentTracesList getTraces(@DefaultValue("10") @PathParam("minutes") int minutes) throws Exception {
+
+ RecentTracesList recentTraces = new RecentTracesList();
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return recentTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+ scanner.setRange(range);
+
+ final Map<String,RecentTracesInformation> summary = new TreeMap<>();
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
+ @Override
+ public Void run() {
+ parseSpans(scanner, summary);
+ return null;
+ }
+ });
+ } else {
+ parseSpans(scanner, summary);
+ }
+
+ // Adds the traces to the list
+ for (Entry<String,RecentTracesInformation> entry : summary.entrySet()) {
+ RecentTracesInformation stat = entry.getValue();
+ recentTraces.addTrace(stat);
+ }
+ return recentTraces;
+ }
+
+ /**
+ * Generates a list of traces filtered by type and range of minutes
+ *
+ * @param type
+ * Type of the trace
+ * @param minutes
+ * Range of minutes
+ * @return List of traces filtered by type and range
+ */
+ @Path("listType/{type}/{minutes}")
+ @GET
+ public TraceType getTracesType(@PathParam("type") String type, @PathParam("minutes") int minutes) throws Exception {
+
+ TraceType typeTraces = new TraceType(type);
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return typeTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+
+ scanner.setRange(range);
+
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
+ @Override
+ public Void run() {
+ for (Entry<Key,Value> entry : scanner) {
+ RemoteSpan span = TraceFormatter.getRemoteSpan(entry);
+
+ if (span.description.equals(type)) {
+ typeTraces.addTrace(new TracesForTypeInformation(span));
+ }
+ }
+ return null;
+ }
+ });
+ } else {
+ for (Entry<Key,Value> entry : scanner) {
+ RemoteSpan span = TraceFormatter.getRemoteSpan(entry);
+ if (span.description.equals(type)) {
+ typeTraces.addTrace(new TracesForTypeInformation(span));
+ }
+ }
+ }
+ return typeTraces;
+ }
+
+ /**
+ * Generates a list of traces filtered by ID
+ *
+ * @param id
+ * ID of the trace to display
+ * @return traces by ID
+ */
+ @Path("show/{id}")
+ @GET
+ public TraceList getTracesType(@PathParam("id") String id) throws Exception {
+ TraceList traces = new TraceList(id);
+
+ if (id == null) {
+ return null;
+ }
+
+ Pair<Scanner,UserGroupInformation> entry = getScanner();
+ final Scanner scanner = entry.getFirst();
+ if (scanner == null) {
+ return traces;
+ }
+
+ Range range = new Range(new Text(id));
+ scanner.setRange(range);
+ final SpanTree tree = new SpanTree();
+ long start;
+
+ if (null != entry.getSecond()) {
+ start = entry.getSecond().doAs(new PrivilegedAction<Long>() {
+ @Override
+ public Long run() {
+ return addSpans(scanner, tree, Long.MAX_VALUE);
+ }
+ });
+ } else {
+ start = addSpans(scanner, tree, Long.MAX_VALUE);
+ }
+
+ traces.addStartTime(start);
+
+ final long finalStart = start;
+ Set<Long> visited = tree.visit(new SpanTreeVisitor() {
+ @Override
+ public void visit(int level, RemoteSpan parent, RemoteSpan node, Collection<RemoteSpan> children) {
+ traces.addTrace(addTraceInformation(level, node, finalStart));
+ }
+ });
+ tree.nodes.keySet().removeAll(visited);
+ if (!tree.nodes.isEmpty()) {
+ for (RemoteSpan span : TraceDump.sortByStart(tree.nodes.values())) {
+ traces.addTrace(addTraceInformation(0, span, finalStart));
+ }
+ }
+ return traces;
+ }
+
+ private static TraceInformation addTraceInformation(int level, RemoteSpan node, long finalStart) {
+
+ boolean hasData = node.data != null && !node.data.isEmpty();
+ boolean hasAnnotations = node.annotations != null && !node.annotations.isEmpty();
+
+ AddlInformation addlData = new AddlInformation();
+
+ if (hasData || hasAnnotations) {
+
+ if (hasData) {
+ for (Entry<String,String> entry : node.data.entrySet()) {
+ DataInformation data = new DataInformation(entry.getKey(), entry.getValue());
+ addlData.addData(data);
+ }
+ }
+ if (hasAnnotations) {
+ for (Annotation entry : node.annotations) {
+ AnnotationInformation annotations = new AnnotationInformation(entry.getMsg(), entry.getTime() - finalStart);
+ addlData.addAnnotations(annotations);
+ }
+ }
+ }
+
+ return new TraceInformation(level, node.stop - node.start, node.start - finalStart, node.spanId, node.svc + "@" + node.sender, node.description, addlData);
+ }
+
+ protected Range getRangeForTrace(long minutesSince) {
+ long endTime = System.currentTimeMillis();
+ long millisSince = minutesSince * 60 * 1000;
+ // Catch the overflow
+ if (millisSince < minutesSince) {
+ millisSince = endTime;
+ }
+ long startTime = endTime - millisSince;
+
+ String startHexTime = Long.toHexString(startTime), endHexTime = Long.toHexString(endTime);
+ while (startHexTime.length() < endHexTime.length()) {
+ startHexTime = "0" + startHexTime;
+ }
+
+ return new Range(new Text("start:" + startHexTime), new Text("start:" + endHexTime));
+ }
+
+ private void parseSpans(Scanner scanner, Map<String,RecentTracesInformation> summary) {
+ for (Entry<Key,Value> entry : scanner) {
+ RemoteSpan span = TraceFormatter.getRemoteSpan(entry);
+ RecentTracesInformation stats = summary.get(span.description);
+ if (stats == null) {
+ summary.put(span.description, stats = new RecentTracesInformation(span.description));
+ }
+ stats.addSpan(span);
+ }
+ }
+
+ protected Pair<Scanner,UserGroupInformation> getScanner() throws AccumuloException, AccumuloSecurityException {
+ AccumuloConfiguration conf = Monitor.getContext().getConfiguration();
+ final boolean saslEnabled = conf.getBoolean(Property.INSTANCE_RPC_SASL_ENABLED);
+ UserGroupInformation traceUgi = null;
+ final String principal;
+ final AuthenticationToken at;
+ Map<String,String> loginMap = conf.getAllPropertiesWithPrefix(Property.TRACE_TOKEN_PROPERTY_PREFIX);
+ // May be null
+ String keytab = loginMap.get(Property.TRACE_TOKEN_PROPERTY_PREFIX.getKey() + "keytab");
+ if (keytab == null || keytab.length() == 0) {
+ keytab = conf.getPath(Property.GENERAL_KERBEROS_KEYTAB);
+ }
+
+ if (saslEnabled && null != keytab) {
+ principal = SecurityUtil.getServerPrincipal(conf.get(Property.TRACE_USER));
+ try {
+ traceUgi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal, keytab);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to login as trace user", e);
+ }
+ } else {
+ principal = conf.get(Property.TRACE_USER);
+ }
+
+ if (!saslEnabled) {
+ if (loginMap.isEmpty()) {
+ Property p = Property.TRACE_PASSWORD;
+ at = new PasswordToken(conf.get(p).getBytes(UTF_8));
+ } else {
+ Properties props = new Properties();
+ int prefixLength = Property.TRACE_TOKEN_PROPERTY_PREFIX.getKey().length();
+ for (Entry<String,String> entry : loginMap.entrySet()) {
+ props.put(entry.getKey().substring(prefixLength), entry.getValue());
+ }
+
+ AuthenticationToken token = Property.createInstanceFromPropertyName(conf, Property.TRACE_TOKEN_TYPE, AuthenticationToken.class, new PasswordToken());
+ token.init(props);
+ at = token;
+ }
+ } else {
+ at = null;
+ }
+
+ final String table = conf.get(Property.TRACE_TABLE);
+ Scanner scanner;
+ if (null != traceUgi) {
+ try {
+ scanner = traceUgi.doAs(new PrivilegedExceptionAction<Scanner>() {
+
+ @Override
+ public Scanner run() throws Exception {
+ // Make the KerberosToken inside the doAs
+ AuthenticationToken token = at;
+ if (null == token) {
+ token = new KerberosToken();
+ }
+ return getScanner(table, principal, token);
+ }
+
+ });
+ } catch (IOException | InterruptedException e) {
+ throw new RuntimeException("Failed to obtain scanner", e);
+ }
+ } else {
+ if (null == at) {
+ throw new AssertionError("AuthenticationToken should not be null");
+ }
+ scanner = getScanner(table, principal, at);
+ }
+
+ return new Pair<>(scanner, traceUgi);
+ }
+
+ private Scanner getScanner(String table, String principal, AuthenticationToken at) throws AccumuloException, AccumuloSecurityException {
+ try {
+ Connector conn = HdfsZooInstance.getInstance().getConnector(principal, at);
+ if (!conn.tableOperations().exists(table)) {
+ return null;
+ }
+ return conn.createScanner(table, conn.securityOperations().getUserAuthorizations(principal));
+ } catch (AccumuloSecurityException | TableNotFoundException ex) {
+ return null;
--- End diff --
You should let these exceptions go to calling methods, Users will want to know when these happen.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110015222
--- Diff: server/monitor/src/main/resources/resources/vis.js ---
@@ -0,0 +1,506 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
--- End diff --
There is a `vis.js.old` file that should be removed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111496189
--- Diff: assemble/pom.xml ---
@@ -195,6 +235,86 @@
<optional>true</optional>
</dependency>
<dependency>
+ <groupId>org.freemarker</groupId>
+ <artifactId>freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-utils</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>osgi-resource-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>aopalliance-repackaged</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>javax.inject</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.bundles.repackaged</groupId>
+ <artifactId>jersey-guava</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-jetty-http</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-client</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-common</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-server</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-entity-filtering</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-mvc</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.ext</groupId>
+ <artifactId>jersey-mvc-freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.media</groupId>
+ <artifactId>jersey-media-jaxb</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.media</groupId>
+ <artifactId>jersey-media-json-jackson</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.javassist</groupId>
+ <artifactId>javassist</artifactId>
--- End diff --
kk will trust you to get to the appropriate, minimal set. You probably know the jersey/glassfish stuff better than I do now, anyways.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110464940
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java ---
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import org.apache.accumulo.core.master.thrift.TableInfo;
+
+/**
+ *
+ * Generates table information as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+public class TableInformation {
+
+ // Variable names become JSON keys
+ public String tablename, tableId, tableState;
+
+ public int tablets, onlineTablets;
+ public long recs, recsInMemory;
+
+ public double ingest, ingestByteRate, query, queryByteRate;
+
+ public CompactionsList majorCompactions, minorCompactions, scans;
+
+ private int queuedMajorCompactions, runningMajorCompactions, queuedMinorCompactions, runningMinorCompactions, queuedScans, runningScans;
--- End diff --
More code style to fix
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110216056
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
+
+ <link rel="stylesheet" type="text/css" href="/resources/screen.css" media="screen" />
+ <script>
+ /**
+ * Sets up autorefresh on initial load
+ */
+ $(document).ready(function() {
+ setupAutoRefresh();
+ });
+ </script>
+ <#if js??>
--- End diff --
You are welcome!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110476772
--- Diff: server/monitor/src/main/resources/resources/overview.js ---
@@ -0,0 +1,293 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates overview initial tables
+ */
+$(document).ready(function() {
+ createMasterTable();
+ createZKTable();
+ refreshOverview();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshOverview() {
+ $.ajaxSetup({
+ async: false
+ });
+ getMaster();
+ getZK();
+ getIngestRate();
+ getScanEntries();
+ getIngestByteRate();
+ getQueryByteRate();
+ getLoadAverage();
+ getLookups();
+ getMinorCompactions();
+ getMajorCompactions();
+ getIndexCacheHitRate();
+ getDataCacheHitRate();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshMasterTable();
+ refreshZKTable();
+ makePlots();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshOverview();
+}
+
+/**
+ * Refreshes the master table
+ */
+function refreshMasterTable() {
+ var data = sessionStorage.master === undefined ?
+ [] : JSON.parse(sessionStorage.master);
+
+ $('#master tr td:first').hide();
+ $('#master tr td').hide();
+
+ // If the master is down, show the first row, otherwise refresh old values
+ if (data.length === 0 || data.master === 'No Masters running') {
+ $('#master tr td:first').show();
+ } else {
+ $('#master tr td:not(:first)').show();
+ var table = $('#master td.right');
+
+ table.eq(0).html(bigNumberForQuantity(data.tables));
+ table.eq(1).html(bigNumberForQuantity(data.totalTabletServers));
+ table.eq(2).html(bigNumberForQuantity(data.deadTabletServersCount));
+ table.eq(3).html(bigNumberForQuantity(data.tablets));
+ table.eq(4).html(bigNumberForQuantity(data.numentries));
+ table.eq(5).html(bigNumberForQuantity(data.lookups));
+ table.eq(6).html(timeDuration(data.uptime));
+ }
+}
+
+/**
+ * Generates the master table
+ */
+function createMasterTable() {
+ var items = [];
+ items.push('<tr><th colspan="2"><a href="/master">Accumulo' +
+ ' Master</a></th></tr>');
+
+ items.push('<tr><td colspan="2" class="center">' +
+ '<span class="label label-danger">Master is Down</span></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tables">Tables</a></td>' +
+ '<td class="right"></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tservers">Tablet' +
+ ' Servers</a></td><td class="right"></td></tr>');
+
+ items.push('<tr><td class="left"><a href="/tservers">Dead ' +
+ 'Tablet Servers</a></td><td class="right"></td></tr>');
+
+ items.push('<tr><td class="left">Tablets</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Entries</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Lookups</td><td class="right"></td></tr>');
+ items.push('<tr><td class="left">Uptime</td><td class="right"></td></tr>');
+
+ $('<table/>', {
+ html: items.join(''),
+ class: 'table table-bordered table-striped table-condensed'
+ }).appendTo('#master');
+}
+
+/**
+ * Refresh the zookeeper table
+ */
+function refreshZKTable() {
+ var data = sessionStorage.zk === undefined ?
+ [] : JSON.parse(sessionStorage.zk);
+
+ $('#zookeeper tr td:first').hide();
+ $('#zookeeper tr:gt(2)').remove();
+
+ if (data.length === 0 || data.zkServers.length === 0) {
+ $('#zookeeper tr td:first').show();
+ } else {
+ var items = [];
+ $.each(data.zkServers, function(key, val) {
+ if (val.clients >= 0) {
+ items.push('<td class="left">' + val.server + '</td>');
+ items.push('<td class="left">' + val.mode + '</td>');
+ items.push('<td class="right">' + val.clients + '</td></tr>');
+ } else {
+ items.push('<tr><td class="left">' + val.server + '</td>');
+ items.push('<td class="left"><span class="error">Down</span></td>');
+ items.push('<td class="right"></td>');
+ }
+ });
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#zookeeper table');
+ }
+}
+
+/**
+ * Generates the zookeeper table
+ */
+function createZKTable() {
+ var items = [];
+ items.push('<tr><th colspan="3">Zookeeper</th></tr>');
+ items.push('<tr><th>Server</th><th>Mode</th><th>Clients</th></tr>');
+ items.push('<td class="center" colspan="3"><i>No Zookeepers</i></td>');
+ $('<table/>', {
+ html: items.join(''),
+ class: 'table table-bordered table-striped table-condensed'
+ }).appendTo('#zookeeper');
+}
+
+//// Overview plot creation
+
+/**
+ * Create the plots for the overview page
+ */
+function makePlots() {
+ var d = new Date();
+ var n = d.getTimezoneOffset() * 60000; // Converts offset to milliseconds
+ var tz = new Date().toLocaleTimeString('en-us',
+ {timeZoneName: 'short'}).split(' ')[2]; // Short version of timezone
+ var tzFormat = '%H:%M<br />' + tz;
+
+ // Create Ingest Rate plot
+ var ingestRate = [];
+ var data = sessionStorage.ingestRate === undefined ?
+ [] : JSON.parse(sessionStorage.ingestRate);
+ $.each(data, function(key, val) {
+
+ ingestRate.push([val.first - n, val.second]);
+ });
+ $.plot($('#ingest_entries'), [{ data: ingestRate,
+ lines: { show: true }, color: '#d9534f' }],
+ {yaxis: {}, xaxis: {mode: 'time', minTickSize: [1, 'minute'],
+ timeformat: tzFormat, ticks: 3}});
+
+ // Create Scan Entries plot
+ var scanEntries = {'Read': [], 'Returned': []};
+ var data = sessionStorage.scanEntries === undefined ?
+ [] : JSON.parse(sessionStorage.scanEntries);
+ $.each(data, function(key, val) {
+ $.each(val.second, function(key2, val2) {
+ scanEntries[val.first].push([val2.first - n, val2.second]);
+ });
+ });
+ $.plot($('#scan_entries'), [{ label: 'Read',
+ data: scanEntries.Read, lines: { show: true }, color: '#d9534f' },
+ { label: 'Returned', data: scanEntries.Returned, lines: { show: true },
+ color: '#337ab7' }], {yaxis: {}, xaxis: {mode: 'time',
+ minTickSize: [1, 'minute'], timeformat: tzFormat, ticks: 3}});
--- End diff --
I'll check it out
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111157832
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+ // Variable names become JSON keys
+ private TabletStats total, historical;
+
+ /**
+ * Generates tserver summary
+ *
+ * @return tserver summary
+ */
+ @GET
+ public TabletServers getTserverSummary() {
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ tserverInfo.addTablet(new TabletServer(status));
+ }
+
+ tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+ return tserverInfo;
+ }
+
+ /**
+ * REST call to clear dead servers from list
+ *
+ * @param server
+ * Dead server to clear
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ public void clearDeadServer(@QueryParam("server") String server) throws Exception {
+ DeadServerList obit = new DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + Constants.ZDEADTSERVERS);
+ obit.delete(server);
+ }
+
+ /**
+ * Generates a recovery tserver list
+ *
+ * @return Recovery tserver list
+ */
+ @Path("recovery")
+ @GET
+ public Map<String,List<Map<String,String>>> getTserverRecovery() {
--- End diff --
I decided to just create another class for this, since I think it is much easier to follow the classes than the Map.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
> My point was that I selected "All", and I got an extra two filters (default and accumulo). All should encompass everything -- it was confusing that I had those extra filters which were meaningless (because I already said "give me all the tables").
So the reason this happens is that the user can also remove specific tables from the selected ones (if you selected "All", but then wanted to get rid of the "Accumulo" namespace, you could just select "Accumulo" and it would leave you with the "Default", if that makes sense.
> Docs for the java objects is fine, but I meant REST API docs. As I understand it, Swagger is pretty common (https://jakubstas.com/spring-jersey-swagger-create-documentation/). Describes what REST endpoints exist, what options they accept and the responses they return.
Ah, I see, didn't know these existed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> I'm going to try to merge this in tomorrow, unless there are any further issues which cannot be resolved in a follow-up issue.
For the sake of stabilization, that sounds good. This PR is way too massive to grok it at one point now. I'll try to test this out locally and make sure it's not super-busted for me. Like you said, other things can be addressed as a follow-on. (2.0 blocker or improvement as necessary)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
Entirely up to you. That's in your repo :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076106
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
+ tableNamespace.addTable(new TableNamespace(key));
+ }
+ }
+
+ // Add tables to the list
+ for (Entry<String,String> entry : Tables.getNameToIdMap(HdfsZooInstance.getInstance()).entrySet()) {
+ String tableName = entry.getKey(), tableId = entry.getValue();
+ TableInfo tableInfo = tableStats.get(tableName);
+ if (null != tableInfo) {
+ Double holdTime = compactingByTable.get(tableId);
+ if (holdTime == null)
+ holdTime = Double.valueOf(0.);
+
+ for (TableNamespace name : tableNamespace.tables) {
+ // Check if table has the default namespace
+ if (!tableName.contains(".") && name.namespace.equals("")) {
--- End diff --
`name.namespace.isEmpty()` instead of allocating a new string.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110013733
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/BasicServlet.java ---
@@ -1,285 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.accumulo.monitor.servlets;
-
-import static java.nio.charset.StandardCharsets.UTF_8;
-
-import java.io.IOException;
-import java.io.PrintWriter;
-import java.io.StringWriter;
-import java.io.UnsupportedEncodingException;
-import java.net.HttpURLConnection;
-import java.net.URLDecoder;
-import java.net.URLEncoder;
-import java.util.Date;
-import java.util.List;
-
-import javax.servlet.ServletException;
-import javax.servlet.http.Cookie;
-import javax.servlet.http.HttpServlet;
-import javax.servlet.http.HttpServletRequest;
-import javax.servlet.http.HttpServletResponse;
-
-import org.apache.accumulo.core.Constants;
-import org.apache.accumulo.core.conf.Property;
-import org.apache.accumulo.monitor.Monitor;
-import org.apache.accumulo.server.monitor.DedupedLogEvent;
-import org.apache.accumulo.server.monitor.LogService;
-import org.apache.log4j.Level;
-import org.apache.log4j.Logger;
-
-abstract public class BasicServlet extends HttpServlet {
- public static final String STANDBY_MONITOR_MESSAGE = "This is not the active Monitor";
-
- private static final long serialVersionUID = 1L;
- protected static final Logger log = Logger.getLogger(BasicServlet.class);
- private String bannerText;
- private String bannerColor;
- private String bannerBackground;
-
- abstract protected String getTitle(HttpServletRequest req);
-
- public boolean isActiveMonitor() {
- // If the HighlyAvailableService is not initialized or it's not the active service, throw an exception
- // to prevent processing of the servlet.
- if (null == Monitor.HA_SERVICE_INSTANCE || !Monitor.HA_SERVICE_INSTANCE.isActiveService()) {
- return false;
- }
- return true;
- }
-
- @Override
- public void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
- // Verify that this is the active Monitor instance
- if (!isActiveMonitor()) {
- resp.sendError(HttpURLConnection.HTTP_UNAVAILABLE, STANDBY_MONITOR_MESSAGE);
- return;
- }
- StringBuilder sb = new StringBuilder();
- try {
- Monitor.fetchData();
- bannerText = sanitize(Monitor.getContext().getConfiguration().get(Property.MONITOR_BANNER_TEXT));
--- End diff --
The three monitor banner configuration properties in this file should be deprecated in the `Property` class.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076356
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+ // Variable names become JSON keys
+ private TabletStats total, historical;
+
+ /**
+ * Generates tserver summary
+ *
+ * @return tserver summary
+ */
+ @GET
+ public TabletServers getTserverSummary() {
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ tserverInfo.addTablet(new TabletServer(status));
+ }
+
+ tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+ return tserverInfo;
+ }
+
+ /**
+ * REST call to clear dead servers from list
+ *
+ * @param server
+ * Dead server to clear
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ public void clearDeadServer(@QueryParam("server") String server) throws Exception {
+ DeadServerList obit = new DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + Constants.ZDEADTSERVERS);
+ obit.delete(server);
+ }
+
+ /**
+ * Generates a recovery tserver list
+ *
+ * @return Recovery tserver list
+ */
+ @Path("recovery")
+ @GET
+ public Map<String,List<Map<String,String>>> getTserverRecovery() {
+
+ Map<String,List<Map<String,String>>> jsonObj = new HashMap<String,List<Map<String,String>>>();
+ List<Map<String,String>> recoveryList = new ArrayList<>();
+ Map<String,String> recoveryObj = new HashMap<String,String>();
+
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ for (TabletServerStatus server : mmi.tServerInfo) {
+ if (server.logSorts != null) {
+ for (RecoveryStatus recovery : server.logSorts) {
+ recoveryObj.put("server", AddressUtil.parseAddress(server.name, false).getHostText());
+ recoveryObj.put("log", recovery.name);
+ recoveryObj.put("time", Long.toString(recovery.runtime));
+ recoveryObj.put("copySort", Double.toString(recovery.progress));
+
+ recoveryList.add(recoveryObj);
+ }
+ }
+ }
+
+ jsonObj.put("recoveryList", recoveryList);
+
+ return jsonObj;
+ }
+
+ /**
+ * Generates details for the selected tserver
+ *
+ * @param tserverAddr
+ * TServer name
+ * @return TServer details
+ */
+ @Path("{address}")
+ @GET
+ public TabletServerSummary getTserverDetails(@PathParam("address") String tserverAddr) throws Exception {
+
+ String tserverAddress = tserverAddr;
+
+ boolean tserverExists = false;
+ if (tserverAddress != null && tserverAddress.isEmpty() == false) {
+ for (TabletServerStatus ts : Monitor.getMmi().getTServerInfo()) {
+ if (tserverAddress.equals(ts.getName())) {
+ tserverExists = true;
+ break;
+ }
+ }
+ }
+
+ if (tserverAddress == null || tserverAddress.isEmpty() || tserverExists == false) {
+
+ return null;
+ }
+
+ double totalElapsedForAll = 0;
+ double splitStdDev = 0;
+ double minorStdDev = 0;
+ double minorQueueStdDev = 0;
+ double majorStdDev = 0;
+ double majorQueueStdDev = 0;
+ double currentMinorAvg = 0;
+ double currentMajorAvg = 0;
+ double currentMinorStdDev = 0;
+ double currentMajorStdDev = 0;
+ total = new TabletStats(null, new ActionStats(), new ActionStats(), new ActionStats(), 0, 0, 0, 0);
+ HostAndPort address = HostAndPort.fromString(tserverAddress);
+ historical = new TabletStats(null, new ActionStats(), new ActionStats(), new ActionStats(), 0, 0, 0, 0);
+ List<TabletStats> tsStats = new ArrayList<>();
+
+ try {
+ ClientContext context = Monitor.getContext();
+ TabletClientService.Client client = ThriftUtil.getClient(new TabletClientService.Client.Factory(), address, context);
+ try {
+ for (String tableId : Monitor.getMmi().tableMap.keySet()) {
+ tsStats.addAll(client.getTabletStats(Tracer.traceInfo(), context.rpcCreds(), tableId));
+ }
+ historical = client.getHistoricalStats(Tracer.traceInfo(), context.rpcCreds());
+ } finally {
+ ThriftUtil.returnClient(client);
+ }
+ } catch (Exception e) {
+ return null;
+ }
+
+ List<CurrentOperations> currentOps = doCurrentOperations(tsStats);
+
+ if (total.minors.num != 0)
+ currentMinorAvg = (long) (total.minors.elapsed / total.minors.num);
+ if (total.minors.elapsed != 0 && total.minors.num != 0)
+ currentMinorStdDev = stddev(total.minors.elapsed, total.minors.num, total.minors.sumDev);
+ if (total.majors.num != 0)
+ currentMajorAvg = total.majors.elapsed / total.majors.num;
+ if (total.majors.elapsed != 0 && total.majors.num != 0 && total.majors.elapsed > total.majors.num)
+ currentMajorStdDev = stddev(total.majors.elapsed, total.majors.num, total.majors.sumDev);
+
+ ActionStatsUpdator.update(total.minors, historical.minors);
+ ActionStatsUpdator.update(total.majors, historical.majors);
+ totalElapsedForAll += total.majors.elapsed + historical.splits.elapsed + total.minors.elapsed;
+
+ minorStdDev = stddev(total.minors.elapsed, total.minors.num, total.minors.sumDev);
+ minorQueueStdDev = stddev(total.minors.queueTime, total.minors.num, total.minors.queueSumDev);
+ majorStdDev = stddev(total.majors.elapsed, total.majors.num, total.majors.sumDev);
+ majorQueueStdDev = stddev(total.majors.queueTime, total.majors.num, total.majors.queueSumDev);
+ splitStdDev = stddev(historical.splits.num, historical.splits.elapsed, historical.splits.sumDev);
+
+ TabletServerDetailInformation details = doDetails(address, tsStats.size());
+
+ List<AllTimeTabletResults> allTime = doAllTimeResults(majorQueueStdDev, minorQueueStdDev, totalElapsedForAll, splitStdDev, majorStdDev, minorStdDev);
+
+ CurrentTabletResults currentRes = doCurrentTabletResults(currentMinorAvg, currentMinorStdDev, currentMajorAvg, currentMajorStdDev);
+
+ TabletServerSummary tserverDetails = new TabletServerSummary(details, allTime, currentRes, currentOps);
+
+ return tserverDetails;
+ }
+
+ private static final int concurrentScans = Monitor.getContext().getConfiguration().getCount(Property.TSERV_READ_AHEAD_MAXCONCURRENT);
--- End diff --
This should be re-computed to ensure that the Monitor picks up new values for this configuration property.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076870
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
+ }
+
+ // Obtains the hours, if time is 0, return hours, minutes, and units
+ hr = time % 24;
+ time = Math.floor(time / 24);
+ if (time == 0) {
+ return hr + 'h' + ' ' + min + 'm';
+ }
+
+ // Obtains the days, if time is 0, return days, hours, and units
+ day = time % 365;
+ time = Math.floor(time / 365);
+ if (time == 0) {
+ return day + 'd' + ' ' + hr + 'h';
+ }
+
+ // Obtains the years, if time is 0, return years, days, and units
+ yr = Math.floor(time);
+ return yr + 'y' + ' ' + day + 'd';
+}
+
+/**
+ * Sorts the selected table by column in the direction chosen
+ *
+ * @param {string} tableID Table to sort
+ * @param {string} direction Direction to sort table, asc or desc
+ * @param {number} n Column to sort
+ */
+function sortTables(tableID, direction, n) {
+ var table, rows, switching, i, x, y, h, shouldSwitch, dir, xFinal, yFinal;
+ table = document.getElementById(tableID);
+ switching = true;
+
+ dir = direction;
+ sessionStorage.direction = dir;
+
+ // Select the rows of the table
+ rows = table.getElementsByTagName('tr');
+
+ // Clears the sortable class from the table columns
+ var count = 0;
+ while (rows[0].getElementsByTagName('th').length > count) {
+ var tmpH = rows[0].getElementsByTagName('th')[count];
+ tmpH.classList.remove('sortable');
+ if (rows.length > 2) {
+ tmpH.classList.add('sortable');
+ }
+ $(tmpH.getElementsByTagName('span')).remove();
+ count += 1;
+ }
+
+ // If there are more than 2 rows, add arrow to the selected column
+ if (rows.length <= 2) {
+ switching = false;
+ } else {
+ h = rows[0].getElementsByTagName('th')[n];
+ if (dir == 'asc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-up"' +
+ ' width="10px" height="10px" />');
+ } else if (dir == 'desc') {
+ $(h).append('<span class="glyphicon glyphicon-chevron-down"' +
+ ' width="10px" height="10px" />');
+ }
+ }
+
+ /*
+ * Make a loop that will continue until
+ * no switching has been done:
+ */
+ while (switching) {
+ switching = false;
+ rows = table.getElementsByTagName('tr');
+
+ /*
+ * Loop through all table rows (except the
+ * first, which contains table headers):
+ */
+ for (i = 1; i < (rows.length - 1); i++) {
+ shouldSwitch = false;
+ /*
+ * Get two elements to compare,
+ * one from current row and one from the next:
+ * If the element is a dash, convert to null, otherwise,
+ * if it is a string, convert to number
+ */
+ x = rows[i].getElementsByTagName('td')[n].getAttribute('data-value');
+ xFinal = (x === '-' || x === '—' ?
+ null : (Number(x) == x ? Number(x) : x));
+
+ y = rows[i + 1].getElementsByTagName('td')[n].getAttribute('data-value');
+ yFinal = (y === '-' || y === '—' ?
+ null : (Number(y) == y ? Number(y) : y));
+
+ /*
+ * Check if the two rows should switch place,
+ * based on the direction, asc or desc:
+ */
+ if (dir == 'asc') {
+ if (xFinal > yFinal || (xFinal !== null && yFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ } else if (dir == 'desc') {
+ if (xFinal < yFinal || (yFinal !== null && xFinal === null)) {
+ // if so, mark as a switch and break the loop:
+ shouldSwitch = true;
+ break;
+ }
+ }
+ }
+ if (shouldSwitch) {
+ /*
+ * If a switch has been marked, make the switch
+ * and mark that a switch has been done:
+ */
+ rows[i].parentNode.insertBefore(rows[i + 1], rows[i]);
+ switching = true;
+ }
+ }
+}
+
+/**
+ * Clears the selected table while leaving the headers
+ *
+ * @param {string} tableID Table to clear
+ */
+function clearTable(tableID) {
+ // JQuery selector to select all rows except for the first row (header)
+ $('#' + tableID).find('tr:not(:first)').remove();
+}
+
+///// REST Calls /////////////
+
+/**
+ * REST GET call for the master information,
+ * stores it on a sessionStorage variable
+ */
+function getMaster() {
+ $.getJSON('/rest/master', function(data) {
+ sessionStorage.master = JSON.stringify(data);
--- End diff --
How does jquery react if it receives some non HTTP/200 response? (e.g. if there was an Exception due to a bug in the Monitor rest call). Do we get a nice message?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
> An old, standby monitor would return HTTP/503 (unavailable) to signify that it isn't polling data from the master and (thus) is useless to observe. Does the new REST API still do this?
No, it does not. "Standby" monitors will function as normal (except for log receiving), because it may be desirable to have multiple monitors. The 503 response code was added in ACCUMULO-4424 for 2.0.0, and has never been released, so this won't break anyone. The monitor has never fit into the same active/standby HA paradigm that applies to the other services, because its features do not require exclusivity. The active/standby single-instance situation for the monitor was only introduced for log receiving (so Accumulo could identify and automatically forward logs to the monitor), which introduced some kind of exclusivity (but for logs only). I believe we've made sufficient improvements to the log forwarding features, so that it's no longer necessary to impose exclusivity restrictions on all monitor features simply for log forwarding.
We can probably still do something to improve the way we inform users about log forwarding exclusivity. For example: we could provide `monitor.log.receiver={WITH_LOCK,ON,OFF}` (default: WITH_LOCK) config, and have the 503 response on the REST endpoint for logs, and display a message on the log view based on whether it receives 503 or 200 from the AJAX call. That should be a separate task, if we do that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110886294
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+ /**
+ * Generates a trace summary
+ *
+ * @param minutes
+ * Range of minutes to filter traces
+ * @return Trace summary in specified range
+ */
+ @Path("summary/{minutes}")
+ @GET
+ public RecentTracesList getTraces(@DefaultValue("10") @PathParam("minutes") int minutes) throws Exception {
+
+ RecentTracesList recentTraces = new RecentTracesList();
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return recentTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+ scanner.setRange(range);
+
+ final Map<String,RecentTracesInformation> summary = new TreeMap<>();
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
--- End diff --
This is how the old Traces did it, as with other parts of the code I wrote (and now that I think of it all of it with few exceptions, such as the status report), I didn't change the original logic of getting the information from Accumulo, I just implemented an endpoint to expose the information through REST API.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> it may be desirable to have multiple monitors
Hurm. Apparently we remember things differently. I would have said that we were previously in agreement about only having one active monitor because of users accidentally looking at "the wrong one" and potentially missing the forwarded logs. I thought that's why I built that 503-response in the first place.
> I believe we've made sufficient improvements to the log forwarding features, so that it's no longer necessary to impose exclusivity restrictions on all monitor features simply for log forwarding.
But they still only go to one place, don't they?
> We can probably still do something to improve the way we inform users about log forwarding exclusivity
(operating under the assumption that logs still only go one place) this seems overkill to me. I think the simple solution would just be to return something that isn't `"[]" with HTTP/200` (differentiating the cases "there are no logs" and "we are not expecting to receive any logs") when invoking the REST calls around recent logs. I don't (yet?) understand why we need configuration around this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110074399
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java ---
@@ -461,25 +452,9 @@ public void run(String hostname) {
try {
log.debug("Creating monitor on port " + port);
server = new EmbeddedWebServer(hostname, port);
- server.addServlet(DefaultServlet.class, "/");
- server.addServlet(OperationServlet.class, "/op");
- server.addServlet(MasterServlet.class, "/master");
- server.addServlet(TablesServlet.class, "/tables");
- server.addServlet(TServersServlet.class, "/tservers");
- server.addServlet(ProblemServlet.class, "/problems");
- server.addServlet(GcStatusServlet.class, "/gc");
- server.addServlet(LogServlet.class, "/log");
- server.addServlet(XMLServlet.class, "/xml");
- server.addServlet(JSONServlet.class, "/json");
- server.addServlet(VisServlet.class, "/vis");
- server.addServlet(ScanServlet.class, "/scans");
- server.addServlet(BulkImportServlet.class, "/bulkImports");
- server.addServlet(Summary.class, "/trace/summary");
- server.addServlet(ListType.class, "/trace/listType");
- server.addServlet(ShowTrace.class, "/trace/show");
- server.addServlet(ReplicationServlet.class, "/replication");
- if (server.isUsingSsl())
- server.addServlet(ShellServlet.class, "/shell");
+ server.addServlet(getDefaultServlet(), "/resources/*");
+ server.addServlet(getRestServlet(), "/rest/*");
--- End diff --
+1
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110160390
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
--- End diff --
The original Java code used this format. Should I just change it to this new format?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110192245
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
Ok cool. I will wait to hear from him. I'm sure there's a reason for it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110453818
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --
How come you didn't extend DedupedLogEvent? It looks like LogResource.getRecentLogs() is doing conversion between the objects. You might be able to save some code there.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110192467
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
--- End diff --
Ok -- this might be an interesting discussion. Perhaps he has an opinion on how this should function. Will wait.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
@lstav Sorry, not yet. I have the tests [running here][1], but have been working on other tasks.
[1]: https://jenkins.revelc.net/job/Accumulo-monitor/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
Gotcha, thanks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
https://github.com/stoicflame/enunciate and http://kongchen.github.io/swagger-maven-plugin/ seem to be the two common options.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110905981
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+ /**
+ * Generates a trace summary
+ *
+ * @param minutes
+ * Range of minutes to filter traces
+ * @return Trace summary in specified range
+ */
+ @Path("summary/{minutes}")
+ @GET
+ public RecentTracesList getTraces(@DefaultValue("10") @PathParam("minutes") int minutes) throws Exception {
+
+ RecentTracesList recentTraces = new RecentTracesList();
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return recentTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+ scanner.setRange(range);
+
+ final Map<String,RecentTracesInformation> summary = new TreeMap<>();
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
--- End diff --
Sure thing, thanks for asking
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110014843
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
+
+ <link rel="stylesheet" type="text/css" href="/resources/screen.css" media="screen" />
+ <script>
+ /**
+ * Sets up autorefresh on initial load
+ */
+ $(document).ready(function() {
+ setupAutoRefresh();
+ });
+ </script>
+ <#if js??>
+ <script src="/resources/${js}"></script>
+ </#if>
+ <script src="/resources/sidebar.js"></script>
+ </head>
+
+ <body>
+ <#include "/templates/modals.ftl">
+ <div id="content-wrapper">
+ <div id="content">
+ <div id="sidebar" class="navbar navbar-inverse navbar-fixed-top">
+ <!--<#include "/templates/header.ftl">-->
+ <#include "/templates/sidebar.ftl">
--- End diff --
`sidebar.ftl` could be renamed to something like `navbar.ftl` as its not one side. also `header.ftl` is commented out. Can the comment and the file be removed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
Rebased, squashing what made sense, and merged into master branch. Did my best to preserve existing logs and commits, giving credit to contributing authors.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
Dumping a couple of things here that I'm finding (they're critical comments -- but they're not to be taken as insults)
* I think there should be an "Apache Accumulo" in the top-left of the header (instead of the instance name). Perhaps center the instance name instead? Right now we have 0 branding of our product. We should do better here (Name, Accumulo logo, ASF logo, etc). Could easily put something into the footer.
* When the Monitor is refreshing the graphs, the UI seems to be blocked with auto-refresh -- you can't click any of the drop-downs. This is very painful as, if you're not watching the ajax calls happening, sometimes your button clicks are just lost.
* Auto-refresh also appears to make 13 ajax calls per refresh which is pretty excessive at a refresh of 5s. We have the logic broken up which is nice, but when we're refreshing all of these, we should minimize the number of ajax calls we're making (get all the data in one call)
* On the tables view, why do "default" and "accumulo" get auto-added when I select "all"? This seems repetitive.
* We pull a lot of js files and load them. IIRC, we should try to get down to one JS file for Accumulo (maybe there's a maven plugin we could use to keep the separation in source but concatenate them at build time so we only include one in the HTML).
* OS load should be truncated to a few digits (tenth or hundreth place).
* Is it possible to create some docs about the REST API and publish those with the monitor? We have this great API but no information about the calls and the data. The XML and JSON stuff is there for old users, but we should try to push the new people to the new REST calls.
* Tool-tips are a little intrusive. If I want to sort on a new column, they pop up really fast and make me move my cursor to view the top elements. Maybe make them slower to appear or semi-transparent?
* Not all table column headers have tool-tips. We should make a pass for clarity/simplicity.
* ~~Accumulo instance ID is shown nowhere that I've found. Nor Accumulo version (info lost over the old monitor).~~ **Found it** in the "info" menu. My gut reaction is that we could put instanceid and version somewhere more prominently (it's a common thing I check when debugging user systems), but I think we could drop the date (we're not a clock, we're a database!)
Looks OK on my phone too (chrome), as well as FF and Chrome on my mac -- woo. I don't have IE at work. I'll try to remember to check when I get home..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111441259
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+ /**
+ * Generates a trace summary
+ *
+ * @param minutes
+ * Range of minutes to filter traces
+ * @return Trace summary in specified range
+ */
+ @Path("summary/{minutes}")
+ @GET
+ public RecentTracesList getTraces(@DefaultValue("10") @PathParam("minutes") int minutes) throws Exception {
+
+ RecentTracesList recentTraces = new RecentTracesList();
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return recentTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+ scanner.setRange(range);
+
+ final Map<String,RecentTracesInformation> summary = new TreeMap<>();
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
+ @Override
+ public Void run() {
+ parseSpans(scanner, summary);
+ return null;
+ }
+ });
+ } else {
+ parseSpans(scanner, summary);
+ }
+
+ // Adds the traces to the list
+ for (Entry<String,RecentTracesInformation> entry : summary.entrySet()) {
+ RecentTracesInformation stat = entry.getValue();
+ recentTraces.addTrace(stat);
+ }
+ return recentTraces;
+ }
+
+ /**
+ * Generates a list of traces filtered by type and range of minutes
+ *
+ * @param type
+ * Type of the trace
+ * @param minutes
+ * Range of minutes
+ * @return List of traces filtered by type and range
+ */
+ @Path("listType/{type}/{minutes}")
+ @GET
+ public TraceType getTracesType(@PathParam("type") String type, @PathParam("minutes") int minutes) throws Exception {
+
+ TraceType typeTraces = new TraceType(type);
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return typeTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+
+ scanner.setRange(range);
+
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
+ @Override
+ public Void run() {
+ for (Entry<Key,Value> entry : scanner) {
+ RemoteSpan span = TraceFormatter.getRemoteSpan(entry);
+
+ if (span.description.equals(type)) {
+ typeTraces.addTrace(new TracesForTypeInformation(span));
+ }
+ }
+ return null;
+ }
+ });
+ } else {
+ for (Entry<Key,Value> entry : scanner) {
+ RemoteSpan span = TraceFormatter.getRemoteSpan(entry);
+ if (span.description.equals(type)) {
+ typeTraces.addTrace(new TracesForTypeInformation(span));
+ }
+ }
+ }
+ return typeTraces;
+ }
+
+ /**
+ * Generates a list of traces filtered by ID
+ *
+ * @param id
+ * ID of the trace to display
+ * @return traces by ID
+ */
+ @Path("show/{id}")
+ @GET
+ public TraceList getTracesType(@PathParam("id") String id) throws Exception {
+ TraceList traces = new TraceList(id);
+
+ if (id == null) {
+ return null;
+ }
+
+ Pair<Scanner,UserGroupInformation> entry = getScanner();
+ final Scanner scanner = entry.getFirst();
+ if (scanner == null) {
+ return traces;
+ }
+
+ Range range = new Range(new Text(id));
+ scanner.setRange(range);
+ final SpanTree tree = new SpanTree();
+ long start;
+
+ if (null != entry.getSecond()) {
+ start = entry.getSecond().doAs(new PrivilegedAction<Long>() {
+ @Override
+ public Long run() {
+ return addSpans(scanner, tree, Long.MAX_VALUE);
+ }
+ });
+ } else {
+ start = addSpans(scanner, tree, Long.MAX_VALUE);
+ }
+
+ traces.addStartTime(start);
+
+ final long finalStart = start;
+ Set<Long> visited = tree.visit(new SpanTreeVisitor() {
+ @Override
+ public void visit(int level, RemoteSpan parent, RemoteSpan node, Collection<RemoteSpan> children) {
+ traces.addTrace(addTraceInformation(level, node, finalStart));
+ }
+ });
+ tree.nodes.keySet().removeAll(visited);
+ if (!tree.nodes.isEmpty()) {
+ for (RemoteSpan span : TraceDump.sortByStart(tree.nodes.values())) {
+ traces.addTrace(addTraceInformation(0, span, finalStart));
+ }
+ }
+ return traces;
+ }
+
+ private static TraceInformation addTraceInformation(int level, RemoteSpan node, long finalStart) {
+
+ boolean hasData = node.data != null && !node.data.isEmpty();
+ boolean hasAnnotations = node.annotations != null && !node.annotations.isEmpty();
+
+ AddlInformation addlData = new AddlInformation();
+
+ if (hasData || hasAnnotations) {
+
+ if (hasData) {
+ for (Entry<String,String> entry : node.data.entrySet()) {
+ DataInformation data = new DataInformation(entry.getKey(), entry.getValue());
+ addlData.addData(data);
+ }
+ }
+ if (hasAnnotations) {
+ for (Annotation entry : node.annotations) {
+ AnnotationInformation annotations = new AnnotationInformation(entry.getMsg(), entry.getTime() - finalStart);
+ addlData.addAnnotations(annotations);
+ }
+ }
+ }
+
+ return new TraceInformation(level, node.stop - node.start, node.start - finalStart, node.spanId, node.svc + "@" + node.sender, node.description, addlData);
+ }
+
+ protected Range getRangeForTrace(long minutesSince) {
+ long endTime = System.currentTimeMillis();
+ long millisSince = minutesSince * 60 * 1000;
+ // Catch the overflow
+ if (millisSince < minutesSince) {
+ millisSince = endTime;
+ }
+ long startTime = endTime - millisSince;
+
+ String startHexTime = Long.toHexString(startTime), endHexTime = Long.toHexString(endTime);
+ while (startHexTime.length() < endHexTime.length()) {
+ startHexTime = "0" + startHexTime;
+ }
+
+ return new Range(new Text("start:" + startHexTime), new Text("start:" + endHexTime));
+ }
+
+ private void parseSpans(Scanner scanner, Map<String,RecentTracesInformation> summary) {
+ for (Entry<Key,Value> entry : scanner) {
+ RemoteSpan span = TraceFormatter.getRemoteSpan(entry);
+ RecentTracesInformation stats = summary.get(span.description);
+ if (stats == null) {
+ summary.put(span.description, stats = new RecentTracesInformation(span.description));
+ }
+ stats.addSpan(span);
+ }
+ }
+
+ protected Pair<Scanner,UserGroupInformation> getScanner() throws AccumuloException, AccumuloSecurityException {
+ AccumuloConfiguration conf = Monitor.getContext().getConfiguration();
+ final boolean saslEnabled = conf.getBoolean(Property.INSTANCE_RPC_SASL_ENABLED);
+ UserGroupInformation traceUgi = null;
+ final String principal;
+ final AuthenticationToken at;
+ Map<String,String> loginMap = conf.getAllPropertiesWithPrefix(Property.TRACE_TOKEN_PROPERTY_PREFIX);
+ // May be null
+ String keytab = loginMap.get(Property.TRACE_TOKEN_PROPERTY_PREFIX.getKey() + "keytab");
+ if (keytab == null || keytab.length() == 0) {
+ keytab = conf.getPath(Property.GENERAL_KERBEROS_KEYTAB);
+ }
+
+ if (saslEnabled && null != keytab) {
+ principal = SecurityUtil.getServerPrincipal(conf.get(Property.TRACE_USER));
+ try {
+ traceUgi = UserGroupInformation.loginUserFromKeytabAndReturnUGI(principal, keytab);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to login as trace user", e);
+ }
+ } else {
+ principal = conf.get(Property.TRACE_USER);
+ }
+
+ if (!saslEnabled) {
+ if (loginMap.isEmpty()) {
+ Property p = Property.TRACE_PASSWORD;
+ at = new PasswordToken(conf.get(p).getBytes(UTF_8));
+ } else {
+ Properties props = new Properties();
+ int prefixLength = Property.TRACE_TOKEN_PROPERTY_PREFIX.getKey().length();
+ for (Entry<String,String> entry : loginMap.entrySet()) {
+ props.put(entry.getKey().substring(prefixLength), entry.getValue());
+ }
+
+ AuthenticationToken token = Property.createInstanceFromPropertyName(conf, Property.TRACE_TOKEN_TYPE, AuthenticationToken.class, new PasswordToken());
+ token.init(props);
+ at = token;
+ }
+ } else {
+ at = null;
+ }
+
+ final String table = conf.get(Property.TRACE_TABLE);
+ Scanner scanner;
+ if (null != traceUgi) {
+ try {
+ scanner = traceUgi.doAs(new PrivilegedExceptionAction<Scanner>() {
+
+ @Override
+ public Scanner run() throws Exception {
+ // Make the KerberosToken inside the doAs
+ AuthenticationToken token = at;
+ if (null == token) {
+ token = new KerberosToken();
+ }
+ return getScanner(table, principal, token);
+ }
+
+ });
+ } catch (IOException | InterruptedException e) {
+ throw new RuntimeException("Failed to obtain scanner", e);
+ }
+ } else {
+ if (null == at) {
+ throw new AssertionError("AuthenticationToken should not be null");
+ }
+ scanner = getScanner(table, principal, at);
+ }
+
+ return new Pair<>(scanner, traceUgi);
+ }
+
+ private Scanner getScanner(String table, String principal, AuthenticationToken at) throws AccumuloException, AccumuloSecurityException {
+ try {
+ Connector conn = HdfsZooInstance.getInstance().getConnector(principal, at);
+ if (!conn.tableOperations().exists(table)) {
+ return null;
+ }
+ return conn.createScanner(table, conn.securityOperations().getUserAuthorizations(principal));
+ } catch (AccumuloSecurityException | TableNotFoundException ex) {
+ return null;
--- End diff --
This is how the previous monitor worked. This class exists for the sole purpose of supporting the AJAX calls. For the trace table, these particular exceptions are normal, because the trace table is not always used, or the trace user is not always configured with the right credentials on every server, or the user may use a different table name for storing traces.
So, it's not clear that monitor visitors should be bombarded with backend error messages in this case. It's sufficient to see that the trace data on the monitor page was not populated.
Also, we don't really have a good/consistent way to propagate internal error messages back up to the monitor's REST interface. Ideally, Jersey would take care of that for us, if we just propagate the exception back up the call stack. But, in the case of the trace table, there's no good reason to.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076627
--- Diff: server/monitor/src/main/resources/resources/bulkImport.js ---
@@ -0,0 +1,213 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates bulk import initial table
+ */
+$(document).ready(function() {
+ createBulkImportHeader();
+ createServerBulkHeader();
+ refreshBulkImport();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshBulkImport() {
+ $.ajaxSetup({
+ async: false
+ });
+ getBulkImports();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshBulkImportTable();
+ refreshServerBulkTable();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshBulkImport();
+}
+
+/**
+ * Generates the master bulk import status table
+ */
+function refreshBulkImportTable() {
+
+ clearTable('masterBulkImportStatus');
+
+ /*
+ * Get the bulk import value obtained earlier, if it doesn't exists,
+ * create an empty array
+ */
+ var data = sessionStorage.bulkImports === undefined ?
+ [] : JSON.parse(sessionStorage.bulkImports);
+ var items = [];
+
+ /* If the data is empty, create an empty row, otherwise,
+ * create the rows for the table
+ */
+ if (data.length === 0 || data.bulkImport.length === 0) {
+ items.push('<td class="center" colspan="3"><i>Empty</i></td>');
+ } else {
+ $.each(data.bulkImport, function(key, val) {
+ items.push('<td class="firstcell left" data-value="' + val.filename +
+ '">' + val.filename + '</td>');
+
+ items.push('<td class="right" data-value="' + val.age + '">' + val.age +
+ '</td>');
+
+ items.push('<td class="right" data-value="' + val.state + '">' +
+ val.state + '</td>');
+ });
+ }
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#masterBulkImportStatus');
+}
+
+/**
+ * Generates the bulk import status table
+ */
+function refreshServerBulkTable() {
+
+ clearTable('bulkImportStatus');
+
+ /* Get the bulk import value obtained earlier, if it doesn't exists,
+ * create an empty array
+ */
+ var data = sessionStorage.bulkImports === undefined ?
+ [] : JSON.parse(sessionStorage.bulkImports);
+ var items = [];
+
+ /* If the data is empty, create an empty row, otherwise
+ * create the rows for the table
+ */
+ if (data.length === 0 || data.tabletServerBulkImport.length === 0) {
+ items.push('<td class="center" colspan="3"><i>Empty</i></td>');
+ } else {
+ $.each(data.tabletServerBulkImport, function(key, val) {
+ items.push('<td class="firstcell left" data-value="' + val.server +
+ '"><a href="/tservers?s=' + val.server + '">' + val.server +
+ '</a></td>');
+
+ items.push('<td class="right" data-value="' + val.importSize + '">' +
+ val.importSize + '</td>');
+
+ items.push('<td class="right" data-value="' + val.oldestAge + '">' +
+ (val.oldestAge > 0 ? val.oldestAge : '—') + '</td>');
+ });
+ }
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#bulkImportStatus');
+}
+
+/**
+ * Sorts the bulkImportStatus table on the selected column
+ *
+ * @param {string} table Table ID to sort
+ * @param {number} n Column number to sort by
+ */
+function sortTable(table, n) {
+ var tableIDs = ['bulkImportStatus', 'masterBulkImportStatus'];
+
+ if (sessionStorage.tableColumnSort !== undefined &&
+ sessionStorage.tableColumnSort == n &&
+ sessionStorage.direction !== undefined) {
+ direction = sessionStorage.direction === 'asc' ? 'desc' : 'asc';
+ } else {
+ direction = sessionStorage.direction === undefined ?
+ 'asc' : sessionStorage.direction;
+ }
+ sessionStorage.tableColumn = tableIDs[table];
+ sessionStorage.tableColumnSort = n;
+ sortTables(tableIDs[table], direction, n);
+}
+
+/**
+ * Create tooltip for table column information
+ */
+$(function() {
+ $(document).tooltip();
--- End diff --
is there a reason this isn't in the `$(document).onReady`?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110077410
--- Diff: test/src/main/java/org/apache/accumulo/test/ThriftServerBindsBeforeZooKeeperLockIT.java ---
@@ -94,7 +94,7 @@ public void testMonitorService() throws Exception {
final int responseCode = cnxn.getResponseCode();
final String errorText = FunctionalTestUtils.readAll(cnxn.getErrorStream());
// This is our "assertion", but we want to re-check it if it's not what we expect
- if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains(BasicServlet.STANDBY_MONITOR_MESSAGE)) {
+ if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains("This is not the active Monitor")) {
--- End diff --
Is this text stored in a template now? Please leave a comment for future devs in this test case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
@milleruntime Thanks for the feedback! I'll check the JS stuff, I liked the ideas you mentioned.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110195086
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
I will
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110019365
--- Diff: server/monitor/src/main/resources/templates/overview.ftl ---
@@ -0,0 +1,90 @@
+<#--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <div><h3>${title}</h3></div>
+ <br>
+ <div class="center-block">
+ <table class="overview-table">
--- End diff --
It would be really nice if this page and others used [bootstrap's grid system](http://getbootstrap.com/css/#grid) (rather than `<table>` tags to structure the page). If they did, the charts and tables would collapse into a single column as screen size of the browser shrinks. You can test this with a desktop browser. Check out the Accumulo website for an example. The bootstrap code is very simple. Something like below:
```html
<div class="container">
<div class="row">
<div class="col-md-6">Accumulo master table html</div>
<div class="col-md-6">Zookeeper table html</div>
</div>
<div class="row">
<div class="col-md-6">Chart 1 html</div>
<div class="col-md-6">Chart 2 html</div>
</div>
</div>
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076794
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
--- End diff --
tsk, mixing backend work with presentation logic! ;)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110159175
--- Diff: server/monitor/src/main/resources/resources/bulkImport.js ---
@@ -0,0 +1,213 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates bulk import initial table
+ */
+$(document).ready(function() {
+ createBulkImportHeader();
+ createServerBulkHeader();
+ refreshBulkImport();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshBulkImport() {
+ $.ajaxSetup({
+ async: false
+ });
+ getBulkImports();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshBulkImportTable();
+ refreshServerBulkTable();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshBulkImport();
+}
+
+/**
+ * Generates the master bulk import status table
+ */
+function refreshBulkImportTable() {
+
+ clearTable('masterBulkImportStatus');
+
+ /*
+ * Get the bulk import value obtained earlier, if it doesn't exists,
+ * create an empty array
+ */
+ var data = sessionStorage.bulkImports === undefined ?
+ [] : JSON.parse(sessionStorage.bulkImports);
+ var items = [];
+
+ /* If the data is empty, create an empty row, otherwise,
+ * create the rows for the table
+ */
+ if (data.length === 0 || data.bulkImport.length === 0) {
+ items.push('<td class="center" colspan="3"><i>Empty</i></td>');
+ } else {
+ $.each(data.bulkImport, function(key, val) {
+ items.push('<td class="firstcell left" data-value="' + val.filename +
+ '">' + val.filename + '</td>');
+
+ items.push('<td class="right" data-value="' + val.age + '">' + val.age +
+ '</td>');
+
+ items.push('<td class="right" data-value="' + val.state + '">' +
+ val.state + '</td>');
+ });
+ }
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#masterBulkImportStatus');
+}
+
+/**
+ * Generates the bulk import status table
+ */
+function refreshServerBulkTable() {
+
+ clearTable('bulkImportStatus');
+
+ /* Get the bulk import value obtained earlier, if it doesn't exists,
+ * create an empty array
+ */
+ var data = sessionStorage.bulkImports === undefined ?
+ [] : JSON.parse(sessionStorage.bulkImports);
+ var items = [];
+
+ /* If the data is empty, create an empty row, otherwise
+ * create the rows for the table
+ */
+ if (data.length === 0 || data.tabletServerBulkImport.length === 0) {
+ items.push('<td class="center" colspan="3"><i>Empty</i></td>');
+ } else {
+ $.each(data.tabletServerBulkImport, function(key, val) {
+ items.push('<td class="firstcell left" data-value="' + val.server +
+ '"><a href="/tservers?s=' + val.server + '">' + val.server +
+ '</a></td>');
+
+ items.push('<td class="right" data-value="' + val.importSize + '">' +
+ val.importSize + '</td>');
+
+ items.push('<td class="right" data-value="' + val.oldestAge + '">' +
+ (val.oldestAge > 0 ? val.oldestAge : '—') + '</td>');
+ });
+ }
+
+ $('<tr/>', {
+ html: items.join('')
+ }).appendTo('#bulkImportStatus');
+}
+
+/**
+ * Sorts the bulkImportStatus table on the selected column
+ *
+ * @param {string} table Table ID to sort
+ * @param {number} n Column number to sort by
+ */
+function sortTable(table, n) {
+ var tableIDs = ['bulkImportStatus', 'masterBulkImportStatus'];
+
+ if (sessionStorage.tableColumnSort !== undefined &&
+ sessionStorage.tableColumnSort == n &&
+ sessionStorage.direction !== undefined) {
+ direction = sessionStorage.direction === 'asc' ? 'desc' : 'asc';
+ } else {
+ direction = sessionStorage.direction === undefined ?
+ 'asc' : sessionStorage.direction;
+ }
+ sessionStorage.tableColumn = tableIDs[table];
+ sessionStorage.tableColumnSort = n;
+ sortTables(tableIDs[table], direction, n);
+}
+
+/**
+ * Create tooltip for table column information
+ */
+$(function() {
+ $(document).tooltip();
--- End diff --
There wasn't (I just tested it and it works the same), I'll make the changes necessary on the files.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> What has changed is that the standby serves the REST API and views instead of displaying a message that it does not have the lock, while it is waiting for the lock.
An old, standby monitor would return HTTP/503 (unavailable) to signify that it isn't polling data from the master and (thus) is useless to observe. Does the new REST API still do this?
I am thinking that the test could hit the REST API (instead of a view) to get the same HTTP/503, and the view could display its message about "i am the standby-monitor, blah blah" in response to the 503?
Aside: I'm finding the use of 503 to denote "expected unavailability" to be odd (i'm used to 5xx series being unexpected server-side errors) despite using that code when writing this :). I wonder if there is a 4xx response code which would be more appropriate.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110076465
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/zk/ZKInformation.java ---
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.zk;
+
+import java.util.ArrayList;
+import java.util.List;
+
+/**
+ *
+ * Generates a list of Zookeeper server information
+ *
+ * @since 2.0.0
+ *
+ */
+public class ZKInformation {
--- End diff --
Do we need this class to encapsulate this? Couldn't it just be replaced with List<ZooKeeper>?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110160756
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
--- End diff --
Are you referring to the ` `? This function is the one that creates the time format for the frontend (similar to converting the numbers to include a suffix.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110192066
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
--- End diff --
Ya, there's a bit more later on (building table rows/cells, IIRC). Something we can work on later to reduce.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075495
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/master/MasterResource.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.master;
+
+import java.lang.management.ManagementFactory;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.master.thrift.DeadServer;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerInformation;
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerList;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServerInformation;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServers;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerInformation;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerList;
+import org.apache.accumulo.monitor.rest.tservers.ServerShuttingDownInformation;
+import org.apache.accumulo.monitor.rest.tservers.ServersShuttingDown;
+import org.apache.accumulo.server.master.state.TabletServerState;
+
+/**
+ *
+ * Responsible for generating a new Master information JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/master")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class MasterResource {
+ public static final String NO_MASTERS = "No Masters running";
+
+ /**
+ * Gets the MasterMonitorInfo, allowing for mocking frameworks for testability
+ */
+ protected MasterMonitorInfo getMmi() {
+ return Monitor.getMmi();
+ }
+
+ /**
+ * Generates a master information JSON object
+ *
+ * @return master JSON object
+ */
+ @GET
+ public MasterInformation getTables() {
+
+ MasterInformation masterInformation;
+
+ if (Monitor.getMmi() != null) {
+ String gcStatus = "Waiting";
+ String label = "";
+ if (Monitor.getGcStatus() != null) {
--- End diff --
Get this once and set a local variable. It's redundant to keep invoking `Monitor.getGcStatus()` over and over.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110456625
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --
Think we could replace DedupedLogEvent with your new LogEvent?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110215947
--- Diff: server/monitor/src/main/resources/resources/functions.js ---
@@ -0,0 +1,686 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+// Suffixes for quantity
+var QUANTITY_SUFFIX = ['', 'K', 'M', 'B', 'T', 'e15', 'e18', 'e21'];
+// Suffixes for size
+var SIZE_SUFFIX = ['', 'K', 'M', 'G', 'T', 'P', 'E', 'Z'];
+
+/**
+ * Initializes Auto Refresh to false if it is not set,
+ * and creates listeners for auto refresh
+ */
+function setupAutoRefresh() {
+ // Sets auto refresh to true or false
+ if (!sessionStorage.autoRefresh) {
+ sessionStorage.autoRefresh = 'false';
+ }
+ // Need this to set the initial value for the autorefresh on page load
+ if (sessionStorage.autoRefresh == 'false') {
+ $('.auto-refresh').parent().removeClass('active');
+ } else {
+ $('.auto-refresh').parent().addClass('active');
+ }
+ // Initializes the auto refresh on click listener
+ $('.auto-refresh').click(function(e) {
+ if ($(this).parent().attr('class') == 'active') {
+ $(this).parent().removeClass('active');
+ sessionStorage.autoRefresh = 'false';
+ } else {
+ $(this).parent().addClass('active');
+ sessionStorage.autoRefresh = 'true';
+ }
+ });
+}
+
+/**
+ * Global timer that checks for auto refresh status every 5 seconds
+ */
+TIMER = setInterval(function() {
+ if (sessionStorage.autoRefresh == 'true') {
+ $('.auto-refresh').parent().addClass('active');
+ refresh();
+ refreshNavBar();
+ } else {
+ $('.auto-refresh').parent().removeClass('active');
+ }
+}, 5000);
+
+/**
+ * Empty function in case there is no refresh implementation
+ */
+function refresh() {
+}
+
+/**
+ * Converts a number to a size with suffix
+ *
+ * @param {number} size Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForSize(size) {
+ if (size === null)
+ size = 0;
+ return bigNumber(size, SIZE_SUFFIX, 1024);
+}
+
+/**
+ * Converts a number to a quantity with suffix
+ *
+ * @param {number} quantity Number to convert
+ * @return {string} Number with suffix added
+ */
+function bigNumberForQuantity(quantity) {
+ if (quantity === null)
+ quantity = 0;
+ return bigNumber(quantity, QUANTITY_SUFFIX, 1000);
+}
+
+/**
+ * Adds the suffix to the number, converts the number to one close to the base
+ *
+ * @param {number} big Number to convert
+ * @param {array} suffixes Suffixes to use for convertion
+ * @param {number} base Base to use for convertion
+ * @return {string} The new value with the suffix
+ */
+function bigNumber(big, suffixes, base) {
+ // If the number is smaller than the base, return thee number with no suffix
+ if (big < base) {
+ return big + suffixes[0];
+ }
+ // Finds which suffix to use
+ var exp = Math.floor(Math.log(big) / Math.log(base));
+ // Divides the bumber by the equivalent suffix number
+ var val = big / Math.pow(base, exp);
+ // Keeps the number to 2 decimal places and adds the suffix
+ return val.toFixed(2) + suffixes[exp];
+}
+
+/**
+ * Converts the time to short number and adds unit
+ *
+ * @param {number} time Time in microseconds
+ * @return {string} The time with units
+ */
+function timeDuration(time) {
+ var ms, sec, min, hr, day, yr;
+ ms = sec = min = hr = day = yr = -1;
+
+ time = Math.floor(time);
+
+ // If time is 0 return a dash
+ if (time == 0) {
+ return '—';
+ }
+
+ // Obtains the milliseconds, if time is 0, return milliseconds, and units
+ ms = time % 1000;
+ time = Math.floor(time / 1000);
+ if (time == 0) {
+ return ms + 'ms';
+ }
+
+ // Obtains the seconds, if time is 0, return seconds, milliseconds, and units
+ sec = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return sec + 's' + ' ' + ms + 'ms';
+ }
+
+ // Obtains the minutes, if time is 0, return minutes, seconds, and units
+ min = time % 60;
+ time = Math.floor(time / 60);
+ if (time == 0) {
+ return min + 'm' + ' ' + sec + 's';
--- End diff --
Ah gotcha, I'll try to check it out and see if I can reduce it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110189876
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/view/Indexes.java ---
@@ -0,0 +1,383 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.view;
+
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.glassfish.jersey.server.mvc.Template;
+
+/**
+ *
+ * Index is responsible of specifying Monitor paths and setting the templates for the HTML code
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/")
+@Produces(MediaType.TEXT_HTML)
+public class Indexes {
--- End diff --
The name of this class could be better. Could be `WebApp` `WebViews` or `WebResources`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> One of the design changes I made was to remove the footer (makes it for a cleaner look on large tables) and move it to the "About" section. However, we could add the Accumulo logo later on to the header (maybe even set a link to the Accumulo website?).
I don't have a good suggestion here, just that my gut-reaction was that it was very light on "apache accumulo". I think we should do more here, but I don't have concrete suggestions at the moment. We should definitely make sure it's clearly branded and something we're proud of.
> I believe the 13 calls happen only on the overview (since we are getting all the information for the plots), but in other pages we also do multiple calls (like in the master page).
Yes, I was talking about index (with the graphs).
> "default" != "accumulo". The "default" namespace is the one where created tables get "assigned" to if the user didn't specify a namespace, the "accumulo" namespace I believe is just for the Replication, Root, and Metadata tables (could be mistaken?).
My point was that I selected "All", and I got an extra two filters (default and accumulo). All should encompass everything -- it was confusing that I had those extra filters which were meaningless (because I already said "give me all the tables").
> I tried to write some Java documentation that would be useful for generating the JavaDocs, but it might be good for someone with more knowledge of Accumulo (and Java) to check that the wording is good for it.
Docs for the java objects is fine, but I meant REST API docs. As I understand it, Swagger is pretty common (https://jakubstas.com/spring-jersey-swagger-create-documentation/). Describes what REST endpoints exist, what options they accept and the responses they return.
For context: a big knock against the "old" monitor is that the XML/json endpoints did not return all of the information that the Monitor had available/used. Ensuring that endpoints for all of the data we're using to render HTML was also available to administrators was a big design goal of this approach. Thus, organizations how have custom monitoring/dashboards can scrape these endpoints and integrate the data with whatever solution(s) they use.
> we should revisit the summaries in the future to clean them up (I didn't change the wording from the original Monitor).
Agreed. This is a pretty clear-cut one that needs revisiting before release.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110471980
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerResource.java ---
@@ -0,0 +1,336 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.lang.management.ManagementFactory;
+import java.security.MessageDigest;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.WebApplicationException;
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response.Status;
+
+import org.apache.accumulo.core.Constants;
+import org.apache.accumulo.core.client.impl.ClientContext;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.rpc.ThriftUtil;
+import org.apache.accumulo.core.tabletserver.thrift.ActionStats;
+import org.apache.accumulo.core.tabletserver.thrift.TabletClientService;
+import org.apache.accumulo.core.tabletserver.thrift.TabletStats;
+import org.apache.accumulo.core.trace.Tracer;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.core.zookeeper.ZooUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.master.MasterResource;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.DeadServerList;
+import org.apache.accumulo.server.util.ActionStatsUpdator;
+
+import com.google.common.net.HostAndPort;
+
+/**
+ *
+ * Generates tserver lists as JSON objects
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tservers")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TabletServerResource {
+
+ // Variable names become JSON keys
+ private TabletStats total, historical;
+
+ /**
+ * Generates tserver summary
+ *
+ * @return tserver summary
+ */
+ @GET
+ public TabletServers getTserverSummary() {
+ MasterMonitorInfo mmi = Monitor.getMmi();
+ if (null == mmi) {
+ throw new WebApplicationException(Status.INTERNAL_SERVER_ERROR);
+ }
+
+ TabletServers tserverInfo = new TabletServers(mmi.tServerInfo.size());
+ for (TabletServerStatus status : mmi.tServerInfo) {
+ tserverInfo.addTablet(new TabletServer(status));
+ }
+
+ tserverInfo.addBadTabletServer(new MasterResource().getTables());
+
+ return tserverInfo;
+ }
+
+ /**
+ * REST call to clear dead servers from list
+ *
+ * @param server
+ * Dead server to clear
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ public void clearDeadServer(@QueryParam("server") String server) throws Exception {
+ DeadServerList obit = new DeadServerList(ZooUtil.getRoot(Monitor.getContext().getInstance()) + Constants.ZDEADTSERVERS);
+ obit.delete(server);
+ }
+
+ /**
+ * Generates a recovery tserver list
+ *
+ * @return Recovery tserver list
+ */
+ @Path("recovery")
+ @GET
+ public Map<String,List<Map<String,String>>> getTserverRecovery() {
--- End diff --
I assume this has to return a Map of Maps for the front end? Anyway to use just one List or Map?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075168
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/gc/GarbageCollection.java ---
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.gc;
+
+import org.apache.accumulo.core.gc.thrift.GcCycleStats;
+
+/**
+ *
+ * GarbageCollection is responsible for creating the gc JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+public class GarbageCollection {
+
+ public static final GarbageCollection EMPTY = new GarbageCollection();
--- End diff --
nit: make this private and expose it through a public static method.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110075565
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/problems/ProblemsResource.java ---
@@ -0,0 +1,175 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.problems;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Iterator;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+
+import javax.ws.rs.Consumes;
+import javax.ws.rs.GET;
+import javax.ws.rs.POST;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.QueryParam;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.problems.ProblemReport;
+import org.apache.accumulo.server.problems.ProblemReports;
+import org.apache.accumulo.server.problems.ProblemType;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates a problem summary and details as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/problems")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class ProblemsResource {
+
+ /**
+ * Generates a list with the problem summary
+ *
+ * @return problem summary list
+ */
+ @GET
+ @Path("summary")
+ public Map<String,List<ProblemSummaryInformation>> getSummary() {
+
+ Map<String,List<ProblemSummaryInformation>> jsonObj = new HashMap<String,List<ProblemSummaryInformation>>();
+
+ List<ProblemSummaryInformation> problems = new ArrayList<>();
+
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(HdfsZooInstance.getInstance());
+
+ if (Monitor.getProblemException() == null) {
+ for (Entry<String,Map<ProblemType,Integer>> entry : Monitor.getProblemSummary().entrySet()) {
+ Integer readCount = null, writeCount = null, loadCount = null;
+
+ for (ProblemType pt : ProblemType.values()) {
+ Integer pcount = entry.getValue().get(pt);
+ if (pt.equals(ProblemType.FILE_READ)) {
+ readCount = pcount;
+ } else if (pt.equals(ProblemType.FILE_WRITE)) {
+ writeCount = pcount;
+ } else if (pt.equals(ProblemType.TABLET_LOAD)) {
+ loadCount = pcount;
+ }
+ }
+
+ String tableName = Tables.getPrintableTableNameFromId(tidToNameMap, entry.getKey());
+
+ problems.add(new ProblemSummaryInformation(tableName, entry.getKey(), readCount, writeCount, loadCount));
+ }
+ }
+ jsonObj.put("problemSummary", problems);
+
+ return jsonObj;
+ }
+
+ /**
+ * REST call to clear problem reports from a table
+ *
+ * @param tableID
+ * Table ID to clear problems
+ */
+ @POST
+ @Consumes(MediaType.TEXT_PLAIN)
+ @Path("summary")
+ public void clearTableProblems(@QueryParam("s") String tableID) {
+ Logger log = LoggerFactory.getLogger(Monitor.class);
+ try {
+ ProblemReports.getInstance(Monitor.getContext()).deleteProblemReports(tableID);
+ } catch (Exception e) {
+ log.error("Failed to delete problem reports for table " + tableID, e);
+ }
+ }
+
+ /**
+ * Generates a list of the problem details as a JSON object
+ *
+ * @return problem details list
+ */
+ @GET
+ @Path("details")
+ public Map<String,List<ProblemDetailInformation>> getDetails() {
+
+ Map<String,List<ProblemDetailInformation>> jsonObj = new HashMap<String,List<ProblemDetailInformation>>();
--- End diff --
`new HashMap<>()`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110246658
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/master/MasterResource.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.master;
+
+import java.lang.management.ManagementFactory;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.master.thrift.DeadServer;
+import org.apache.accumulo.core.master.thrift.MasterMonitorInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.util.AddressUtil;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerInformation;
+import org.apache.accumulo.monitor.rest.logs.DeadLoggerList;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServerInformation;
+import org.apache.accumulo.monitor.rest.tservers.BadTabletServers;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerInformation;
+import org.apache.accumulo.monitor.rest.tservers.DeadServerList;
+import org.apache.accumulo.monitor.rest.tservers.ServerShuttingDownInformation;
+import org.apache.accumulo.monitor.rest.tservers.ServersShuttingDown;
+import org.apache.accumulo.server.master.state.TabletServerState;
+
+/**
+ *
+ * Responsible for generating a new Master information JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/master")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class MasterResource {
+ public static final String NO_MASTERS = "No Masters running";
+
+ /**
+ * Gets the MasterMonitorInfo, allowing for mocking frameworks for testability
+ */
+ protected MasterMonitorInfo getMmi() {
+ return Monitor.getMmi();
+ }
+
+ /**
+ * Generates a master information JSON object
+ *
+ * @return master JSON object
+ */
+ @GET
+ public MasterInformation getTables() {
+
+ MasterInformation masterInformation;
+
+ if (Monitor.getMmi() != null) {
+ String gcStatus = "Waiting";
+ String label = "";
+ if (Monitor.getGcStatus() != null) {
--- End diff --
I agree with the sentiment here, as well as the same sentiment for `Monitor.getMmi()`.
I'm actually curious if (in a future task, perhaps), we can inject an instance of the monitor into the HttpContext to be available to Jersey resources, rather than rely on all this static state at all. But, cleaning up the repeated method calls would at least look cleaner for now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110471346
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tservers/TabletServerInformation.java ---
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tservers;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+import org.apache.accumulo.core.master.thrift.RecoveryStatus;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tables.CompactionsList;
+import org.apache.accumulo.monitor.rest.tables.CompactionsTypes;
+import org.apache.accumulo.monitor.rest.trace.RecoveryStatusInformation;
+import org.apache.accumulo.server.util.TableInfoUtil;
+
+/**
+ *
+ * Generates tserver information
+ *
+ * @since 2.0.0
+ *
+ */
+public class TabletServerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute(name = "id")
+ public String server;
+
+ public String hostname;
+ public long lastContact;
+ public double osload;
+
+ public CompactionsTypes compactions;
+
+ public int tablets;
+ public double ingest, query, ingestMB, queryMB;
+ public Integer scans; // For backwards compatibility, has same information as scansRunning
+ public Double scansessions;
+ public Double scanssessions; // For backwards compatibility
+ public long holdtime;
+
+ // New variables
+
+ public String ip;
+ private Integer scansRunning, scansQueued, minorRunning, minorQueued, majorRunning, majorQueued;
+ private CompactionsList scansCompacting, major, minor; // if scans is removed, change scansCompacting to scans
+ public long entries, lookups, indexCacheHits, indexCacheRequests, dataCacheHits, dataCacheRequests;
--- End diff --
More style
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> @joshelser any objection to squashing things down (retaining @lstav as author)?
Just give me a shout-out in the commit message, please.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110241650
--- Diff: test/src/main/java/org/apache/accumulo/test/ThriftServerBindsBeforeZooKeeperLockIT.java ---
@@ -94,7 +94,7 @@ public void testMonitorService() throws Exception {
final int responseCode = cnxn.getResponseCode();
final String errorText = FunctionalTestUtils.readAll(cnxn.getErrorStream());
// This is our "assertion", but we want to re-check it if it's not what we expect
- if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains(BasicServlet.STANDBY_MONITOR_MESSAGE)) {
+ if (HttpURLConnection.HTTP_UNAVAILABLE == responseCode && null != errorText && errorText.contains("This is not the active Monitor")) {
--- End diff --
Honestly, I don't expect this test to pass anymore. I inline'd the variable so I could delete its source class, before I offered that change as a PR to @lstav 's branch. I need to revisit this and figure out what the test was checking for, and see whether the checks still make sense for the current behavior of the monitor.
After this PR, there shouldn't be any reason the monitor would be unavailable. However, as a secondary monitor, it won't be receiving logs, and we should probably have a message on the log page for that, at the very least (maybe also a special symbol in the menu to indicate that it's not receiving logs at the moment).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110156099
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
This is one of the changes @ctubbsii worked on, I had made a Resource class with that information, but I believe he removed it since it was conflicting with setting the path of each rest call. He might be better equipped to answer this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110241064
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/Monitor.java ---
@@ -461,25 +452,9 @@ public void run(String hostname) {
try {
log.debug("Creating monitor on port " + port);
server = new EmbeddedWebServer(hostname, port);
- server.addServlet(DefaultServlet.class, "/");
- server.addServlet(OperationServlet.class, "/op");
- server.addServlet(MasterServlet.class, "/master");
- server.addServlet(TablesServlet.class, "/tables");
- server.addServlet(TServersServlet.class, "/tservers");
- server.addServlet(ProblemServlet.class, "/problems");
- server.addServlet(GcStatusServlet.class, "/gc");
- server.addServlet(LogServlet.class, "/log");
- server.addServlet(XMLServlet.class, "/xml");
- server.addServlet(JSONServlet.class, "/json");
- server.addServlet(VisServlet.class, "/vis");
- server.addServlet(ScanServlet.class, "/scans");
- server.addServlet(BulkImportServlet.class, "/bulkImports");
- server.addServlet(Summary.class, "/trace/summary");
- server.addServlet(ListType.class, "/trace/listType");
- server.addServlet(ShowTrace.class, "/trace/show");
- server.addServlet(ReplicationServlet.class, "/replication");
- if (server.isUsingSsl())
- server.addServlet(ShellServlet.class, "/shell");
+ server.addServlet(getDefaultServlet(), "/resources/*");
+ server.addServlet(getRestServlet(), "/rest/*");
--- End diff --
@ctubbsii Helped me to set this up
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
@ctubbsii any updates on the ITs and on the merge for this?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111506082
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
In these cases, the annotation is required for those specific XML tags, here's an example
`<servers>
<server id="aws.revelc.net:9997">
<hostname>aws.revelc.net:9997</hostname>
</server>
</servers>`
Here the 'id' on the 'server' tag is an attribute of that tag, if I didn't include the `@XmlAttribute` annotation then it would show up like this
`<servers>
<server>
<id>aws.revelc.net:9997</id>
<hostname>aws.revelc.net:9997</hostname>
</server>
</servers>`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110464998
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/LogEvent.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+/**
+ *
+ * A single message logged
+ *
+ * @since 2.0.0
+ *
+ */
+public class LogEvent {
--- End diff --
I think my LogEvent would only work for the REST API, I would need to check it out and see if it would though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110254394
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
> it created a StackOverflowError while processing sub-resources on the parent class
In the annotation processing itself? That's... unexpected o.O
As long as there is a known reason to putting it everywhere, that's fine.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
> I'm going to try to merge this in tomorrow, unless there are any further issues which cannot be resolved in a follow-up issue. @lstav @joshelser any objection to squashing things down (retaining @lstav as author)?
It's fine by me.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110465485
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TableInformation.java ---
@@ -0,0 +1,131 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import org.apache.accumulo.core.master.thrift.TableInfo;
+
+/**
+ *
+ * Generates table information as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+public class TableInformation {
+
+ // Variable names become JSON keys
+ public String tablename, tableId, tableState;
+
+ public int tablets, onlineTablets;
+ public long recs, recsInMemory;
+
+ public double ingest, ingestByteRate, query, queryByteRate;
+
+ public CompactionsList majorCompactions, minorCompactions, scans;
+
+ private int queuedMajorCompactions, runningMajorCompactions, queuedMinorCompactions, runningMinorCompactions, queuedScans, runningScans;
--- End diff --
Yup, currently working on a fix for these, will be pushing soon (in it there's other fixes from the rest of the suggestions from the PR (not all of them though))
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110077036
--- Diff: server/monitor/src/main/resources/resources/gc.js ---
@@ -0,0 +1,278 @@
+/*
+* Licensed to the Apache Software Foundation (ASF) under one or more
+* contributor license agreements. See the NOTICE file distributed with
+* this work for additional information regarding copyright ownership.
+* The ASF licenses this file to You under the Apache License, Version 2.0
+* (the "License"); you may not use this file except in compliance with
+* the License. You may obtain a copy of the License at
+*
+* http://www.apache.org/licenses/LICENSE-2.0
+*
+* Unless required by applicable law or agreed to in writing, software
+* distributed under the License is distributed on an "AS IS" BASIS,
+* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+* See the License for the specific language governing permissions and
+* limitations under the License.
+*/
+
+/**
+ * Creates garbage collector initial table
+ */
+$(document).ready(function() {
+ createHeader();
+ doBanner('gcBanner', 'danger', 'Collector is Unavailable');
+ refreshGC();
+});
+
+/**
+ * Makes the REST calls, generates the tables with the new information
+ */
+function refreshGC() {
+ $.ajaxSetup({
+ async: false
+ });
+ getGarbageCollector();
+ $.ajaxSetup({
+ async: true
+ });
+ refreshGCTable();
+}
+
+/**
+ * Used to redraw the page
+ */
+function refresh() {
+ refreshGC();
+}
+
+/**
+ * Generates the garbage collector table
+ */
+function refreshGCTable() {
+ // Checks the status of the garbage collector
+ var status = JSON.parse(sessionStorage.status).gcStatus;
+
+ // Hides the banner, removes any rows from the table and hides the table
+ $('#gcBanner').hide();
+ $('#gcActivity tr:gt(0)').remove();
+ $('#gcActivity').hide();
+
+ /* Check if the status of the gc is an error, if so, show banner, otherwise,
+ * create the table
+ */
+ if (status === 'ERROR') {
+ $('#gcBanner').show();
+ } else {
+ $('#gcActivity').show();
+ var data = JSON.parse(sessionStorage.gc);
+
+ // Checks if there is a collection activity
+ if (data.files.lastCycle.finished <= 0 &&
+ data.files.currentCycle.started <= 0 &&
+ data.wals.lastCycle.finished <= 0 &&
+ data.wals.currentCycle.started <= 0) {
+ var item = '<td class="center" colspan="7"><i>No Collection Activity' +
--- End diff --
We seem to have quite a bit of co-mingled backend work and presentation logic in here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/accumulo/pull/242
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by milleruntime <gi...@git.apache.org>.
Github user milleruntime commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110905603
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/trace/TracesResource.java ---
@@ -0,0 +1,372 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.trace;
+
+import static java.lang.Math.min;
+import static java.nio.charset.StandardCharsets.UTF_8;
+
+import java.io.IOException;
+import java.security.PrivilegedAction;
+import java.security.PrivilegedExceptionAction;
+import java.util.Collection;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+import java.util.TreeMap;
+
+import javax.ws.rs.DefaultValue;
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.Scanner;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken;
+import org.apache.accumulo.core.client.security.tokens.AuthenticationToken.Properties;
+import org.apache.accumulo.core.client.security.tokens.KerberosToken;
+import org.apache.accumulo.core.client.security.tokens.PasswordToken;
+import org.apache.accumulo.core.conf.AccumuloConfiguration;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.util.Pair;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.security.SecurityUtil;
+import org.apache.accumulo.tracer.SpanTree;
+import org.apache.accumulo.tracer.SpanTreeVisitor;
+import org.apache.accumulo.tracer.TraceDump;
+import org.apache.accumulo.tracer.TraceFormatter;
+import org.apache.accumulo.tracer.thrift.Annotation;
+import org.apache.accumulo.tracer.thrift.RemoteSpan;
+import org.apache.hadoop.io.Text;
+import org.apache.hadoop.security.UserGroupInformation;
+
+/**
+ *
+ * Generates a list of traces with the summary, by type, and trace details
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/trace")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TracesResource {
+
+ /**
+ * Generates a trace summary
+ *
+ * @param minutes
+ * Range of minutes to filter traces
+ * @return Trace summary in specified range
+ */
+ @Path("summary/{minutes}")
+ @GET
+ public RecentTracesList getTraces(@DefaultValue("10") @PathParam("minutes") int minutes) throws Exception {
+
+ RecentTracesList recentTraces = new RecentTracesList();
+
+ Pair<Scanner,UserGroupInformation> pair = getScanner();
+ final Scanner scanner = pair.getFirst();
+ if (scanner == null) {
+ return recentTraces;
+ }
+
+ Range range = getRangeForTrace(minutes);
+ scanner.setRange(range);
+
+ final Map<String,RecentTracesInformation> summary = new TreeMap<>();
+ if (null != pair.getSecond()) {
+ pair.getSecond().doAs(new PrivilegedAction<Void>() {
--- End diff --
Ok no problem, I was just curious.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> I changed to use the public variables at the suggestion of @ctubbsii, but we could discuss the best approach and change it to that.
Ok. Mostly I was thinking that if we are moving these _towards_ being publicly consumable, it would be better if we encapsulate the state and just expose it via methods (that gives us more flexibility).
> I tested on both FF and Chrome, I couldn't get the proxy to work for IE so I haven't been able to test it.
Ok, this would be good to keep in mind before merging. IE still has a bit of prevalence in "the enterprise". We should perform some diligence before merge to make sure that we aren't shipping something entirely busted on the major IE versions.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on the issue:
https://github.com/apache/accumulo/pull/242
> let me know what you think of 41a8b5a
Looks fine to me to fix the test.
Is there work to be done for the "recent logs" API as mentioned in https://github.com/apache/accumulo/pull/242#issuecomment-303823067 ? Would be best to make sure there's test coverage for that, at worst file a follow-on issue..
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by mikewalch <gi...@git.apache.org>.
Github user mikewalch commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110252701
--- Diff: assemble/pom.xml ---
@@ -195,6 +235,86 @@
<optional>true</optional>
</dependency>
<dependency>
+ <groupId>org.freemarker</groupId>
+ <artifactId>freemarker</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-api</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>hk2-utils</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2</groupId>
+ <artifactId>osgi-resource-locator</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>aopalliance-repackaged</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.hk2.external</groupId>
+ <artifactId>javax.inject</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.bundles.repackaged</groupId>
+ <artifactId>jersey-guava</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-jetty-http</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.containers</groupId>
+ <artifactId>jersey-container-servlet-core</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
+ <artifactId>jersey-client</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>org.glassfish.jersey.core</groupId>
--- End diff --
Adding to @joshelser comment below, I think some of these dependencies are transitive (like `jersey-common`) and don't need to be explicitly included.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
@mikewalch thanks, I'll merge the PR when it is available.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by ctubbsii <gi...@git.apache.org>.
Github user ctubbsii commented on the issue:
https://github.com/apache/accumulo/pull/242
I think the data model in the classes could be improved/refactored a bit, but it's the REST API we want to make consumable, not the internal implementation. I don't think it matters whether these POJO fields are private with getters, or public, or whether there's a hierarchy of POJOs vs. custom classes with named fields. It would be nice to clean up all of them so that there is some consistency, and a lot of simplicity, but I don't think we should expect these internal classes representing the REST data model to be publicly consumable.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110077186
--- Diff: server/monitor/src/main/resources/templates/index.ftl ---
@@ -0,0 +1,72 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+<html>
+ <head>
+ <title>${title} - Accumulo ${version}</title>
+ <meta http-equiv="Content-Type" content="test/html" />
+ <meta http-equiv="Content-Script-Type" content="text/javascript" />
+ <meta http-equiv="Content-Style-Type" content="text/css" />
+ <link rel="shortcut icon" type="image/jng" href="/resources/favicon.png" />
+ <script src="/resources/global.js" type="text/javascript"></script>
+ <script src="/resources/functions.js" type="text/javascript"></script>
+
+ <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
+ <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
+
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
+ <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" integrity="sha384-Tc5IQib027qvyjSMfHjOMaLkfuWVxZxUPnCJA7l2mCWNIpG9mGCD8wGNIcPD7Txa" crossorigin="anonymous"></script>
+
+ <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.js"></script>
+ <script language="javascript" type="text/javascript" src="/resources/flot/jquery.flot.time.js"></script>
+ <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/css/select2.min.css" rel="stylesheet" />
+ <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.3/js/select2.min.js"></script>
--- End diff --
I assume we want to bundle this? Is it permissively licensed?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo issue #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on the issue:
https://github.com/apache/accumulo/pull/242
@joshelser Just a couple of things responding to your comments:
> I think there should be an "Apache Accumulo" in the top-left of the header (instead of the instance name). Perhaps center the instance name instead? Right now we have 0 branding of our product. We should do better here (Name, Accumulo logo, ASF logo, etc). Could easily put something into the footer.
One of the design changes I made was to remove the footer (makes it for a cleaner look on large tables) and move it to the "About" section. However, we could add the Accumulo logo later on to the header (maybe even set a link to the Accumulo website?).
> Auto-refresh also appears to make 13 ajax calls per refresh which is pretty excessive at a refresh of 5s. We have the logic broken up which is nice, but when we're refreshing all of these, we should minimize the number of ajax calls we're making (get all the data in one call)
I didn't think of this until I was wrapping up, but it would be good to make an issue about this to simplify those calls. I believe the 13 calls happen only on the overview (since we are getting all the information for the plots), but in other pages we also do multiple calls (like in the master page). Thanks for pointing it out.
> On the tables view, why do "default" and "accumulo" get auto-added when I select "all"? This seems repetitive.
"default" != "accumulo". The "default" namespace is the one where created tables get "assigned" to if the user didn't specify a namespace, the "accumulo" namespace I believe is just for the Replication, Root, and Metadata tables (could be mistaken?).
> Is it possible to create some docs about the REST API and publish those with the monitor? We have this great API but no information about the calls and the data. The XML and JSON stuff is there for old users, but we should try to push the new people to the new REST calls.
I tried to write some Java documentation that would be useful for generating the JavaDocs, but it might be good for someone with more knowledge of Accumulo (and Java) to check that the wording is good for it.
> Not all table column headers have tool-tips. We should make a pass for clarity/simplicity.
The tooltips replace the "Show legend" section of the old Monitor, and not all columns had explanations. @ctubbsii had mentioned that we should revisit the summaries in the future to clean them up (I didn't change the wording from the original Monitor).
> Edit: also, I'm +1 on a merge. Great job seeing this through, @lstav
Thanks!
Most of the rest of the things you mention are fixes that could be done in a future issue (mostly changes in the HTML or JavaScript code.. no need to touch the Java code for these anymore! Woo!).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by joshelser <gi...@git.apache.org>.
Github user joshelser commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r111495881
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/replication/ReplicationResource.java ---
@@ -0,0 +1,204 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.replication;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.Set;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.AccumuloException;
+import org.apache.accumulo.core.client.AccumuloSecurityException;
+import org.apache.accumulo.core.client.BatchScanner;
+import org.apache.accumulo.core.client.Connector;
+import org.apache.accumulo.core.client.TableNotFoundException;
+import org.apache.accumulo.core.client.TableOfflineException;
+import org.apache.accumulo.core.client.admin.TableOperations;
+import org.apache.accumulo.core.conf.Property;
+import org.apache.accumulo.core.data.Key;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.Value;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.core.replication.ReplicationSchema.WorkSection;
+import org.apache.accumulo.core.replication.ReplicationTable;
+import org.apache.accumulo.core.replication.ReplicationTarget;
+import org.apache.accumulo.core.security.Authorizations;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.server.replication.ReplicaSystem;
+import org.apache.accumulo.server.replication.ReplicaSystemFactory;
+import org.apache.hadoop.io.Text;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ *
+ * Generates the replication table with information from the Monitor
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/replication")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
--- End diff --
sgtm
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110155338
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/Totals.java ---
@@ -0,0 +1,53 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest;
+
+/**
+ *
+ * Generates the totals for XML summary
+ *
+ * @since 2.0.0
+ *
+ */
+public class Totals {
+
+ // Variable names become JSON keys
+ public double ingestrate, queryrate, diskrate = 0.0;
--- End diff --
Thanks, I'll recheck the code to see where else this happens.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110156683
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/tables/TablesResource.java ---
@@ -0,0 +1,232 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.tables;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+import java.util.Map;
+import java.util.Map.Entry;
+import java.util.SortedMap;
+import java.util.TreeMap;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+
+import javax.ws.rs.GET;
+import javax.ws.rs.Path;
+import javax.ws.rs.PathParam;
+import javax.ws.rs.Produces;
+import javax.ws.rs.core.MediaType;
+
+import org.apache.accumulo.core.client.Instance;
+import org.apache.accumulo.core.client.impl.Namespaces;
+import org.apache.accumulo.core.client.impl.Tables;
+import org.apache.accumulo.core.data.Range;
+import org.apache.accumulo.core.data.impl.KeyExtent;
+import org.apache.accumulo.core.master.thrift.TableInfo;
+import org.apache.accumulo.core.master.thrift.TabletServerStatus;
+import org.apache.accumulo.core.metadata.MetadataTable;
+import org.apache.accumulo.core.metadata.RootTable;
+import org.apache.accumulo.monitor.Monitor;
+import org.apache.accumulo.monitor.rest.tservers.TabletServer;
+import org.apache.accumulo.monitor.rest.tservers.TabletServers;
+import org.apache.accumulo.server.client.HdfsZooInstance;
+import org.apache.accumulo.server.master.state.MetaDataTableScanner;
+import org.apache.accumulo.server.master.state.TabletLocationState;
+import org.apache.accumulo.server.tables.TableManager;
+import org.apache.accumulo.server.util.TableInfoUtil;
+import org.apache.hadoop.io.Text;
+
+/**
+ *
+ * Generates a tables list from the Monitor as a JSON object
+ *
+ * @since 2.0.0
+ *
+ */
+@Path("/tables")
+@Produces({MediaType.APPLICATION_JSON, MediaType.APPLICATION_XML})
+public class TablesResource {
+
+ private static final TabletServerStatus NO_STATUS = new TabletServerStatus();
+
+ /**
+ * Generates a table list based on the namespace
+ *
+ * @param namespace
+ * Namespace used to filter the tables
+ * @return Table list
+ */
+ private TablesList generateTables(String namespace) {
+ Instance inst = Monitor.getContext().getInstance();
+ Map<String,String> tidToNameMap = Tables.getIdToNameMap(inst);
+ SortedMap<String,TableInfo> tableStats = new TreeMap<>();
+
+ if (Monitor.getMmi() != null && Monitor.getMmi().tableMap != null)
+ for (Entry<String,TableInfo> te : Monitor.getMmi().tableMap.entrySet())
+ tableStats.put(Tables.getPrintableTableNameFromId(tidToNameMap, te.getKey()), te.getValue());
+
+ Map<String,Double> compactingByTable = TableInfoUtil.summarizeTableStats(Monitor.getMmi());
+ TableManager tableManager = TableManager.getInstance();
+
+ SortedMap<String,String> namespaces = Namespaces.getNameToIdMap(Monitor.getContext().getInstance());
+
+ TablesList tableNamespace = new TablesList();
+ List<TableInformation> tables = new ArrayList<>();
+
+ // Add the tables that have the selected namespace
+ for (String key : namespaces.keySet()) {
+ if (namespace.equals("*") || namespace.equals(key) || (key.equals("") && namespace.equals("-"))) {
+ tableNamespace.addTable(new TableNamespace(key));
+ }
+ }
+
+ // Add tables to the list
+ for (Entry<String,String> entry : Tables.getNameToIdMap(HdfsZooInstance.getInstance()).entrySet()) {
+ String tableName = entry.getKey(), tableId = entry.getValue();
+ TableInfo tableInfo = tableStats.get(tableName);
+ if (null != tableInfo) {
+ Double holdTime = compactingByTable.get(tableId);
+ if (holdTime == null)
+ holdTime = Double.valueOf(0.);
+
+ for (TableNamespace name : tableNamespace.tables) {
+ // Check if table has the default namespace
+ if (!tableName.contains(".") && name.namespace.equals("")) {
--- End diff --
Wasn't aware of this, I'll make the change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] accumulo pull request #242: ACCUMULO-2181/3005 REST API and new Monitor UI
Posted by lstav <gi...@git.apache.org>.
Github user lstav commented on a diff in the pull request:
https://github.com/apache/accumulo/pull/242#discussion_r110471868
--- Diff: server/monitor/src/main/java/org/apache/accumulo/monitor/rest/logs/DeadLoggerInformation.java ---
@@ -0,0 +1,57 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.accumulo.monitor.rest.logs;
+
+import javax.xml.bind.annotation.XmlAttribute;
+
+/**
+ *
+ * Stores dead logger information
+ *
+ * @since 2.0.0
+ *
+ */
+public class DeadLoggerInformation {
+
+ // Variable names become JSON keys
+ @XmlAttribute
--- End diff --
@joshelser I checked out the code, in all cases it is used in both a regular REST call and for the XML summary (which the old monitor had them as an attribute instead of a value)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---