You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@storm.apache.org by harshach <gi...@git.apache.org> on 2014/06/24 01:07:27 UTC
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
GitHub user harshach opened a pull request:
https://github.com/apache/incubator-storm/pull/157
STORM-367. Storm UI REST api documentation.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/harshach/incubator-storm STORM-367
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-storm/pull/157.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #157
----
commit 1613a978cecf01c37b5d131eccb4dcc43966d058
Author: Sriharsha Chintalapani <ma...@harsha.io>
Date: 2014-06-23T23:05:41Z
STORM-367. Storm UI REST api documentation.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14192196
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
+|topologyStats.windowPretty| Duration passed in HH:MM:SS format|
+|topologyStats.window| User requested time window for metrics|
+|topologyStats.emitted| Number of messages emitted in given window|
+|topologyStats.trasferred| Number messages transferred in given window|
+|topologyStats.acked| Number of messages acked in given window|
+|topologyStats.failed| Number of messages failed in given window|
+|spouts| Array of all the spout components in the topology|
+|spouts.spoutId| Spout id|
+|spouts.executors| Number of executors for the spout|
+|spouts.completeLatency| Total latency for processing the message|
+|spouts.transferred| Total number of messages transferred|
+|spouts.tasks| Total number of tasks for the spout|
+|spouts.lastError| Shows the last error happened in a spout|
+|spouts.acked| Number of messages acked|
+|spouts.failed| Number of messages failed|
+|bolts| Array of bolt components in the topology|
+|bolts.boltId| Bolt id|
+|bolts.capacity| This value indicates number of mesages executed * average execute latency / time window|
+|bolts.processLatency| Bolt's average time to ack a message after it's received|
+|bolts.executeLatency| Average time for bolt's execute method |
--- End diff --
These are strings. Would it be better if I add the Value column like the parameters section which shows the type of the field.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14187785
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
--- End diff --
Can we add in some description of these two? msgTimeout is the timeout in seconds a tuple has before the spout considers it failed.
windowHint should probably be tied back to the parameters section.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14188029
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
+|topologyStats.windowPretty| Duration passed in HH:MM:SS format|
+|topologyStats.window| User requested time window for metrics|
+|topologyStats.emitted| Number of messages emitted in given window|
+|topologyStats.trasferred| Number messages transferred in given window|
+|topologyStats.acked| Number of messages acked in given window|
+|topologyStats.failed| Number of messages failed in given window|
+|spouts| Array of all the spout components in the topology|
+|spouts.spoutId| Spout id|
+|spouts.executors| Number of executors for the spout|
+|spouts.completeLatency| Total latency for processing the message|
+|spouts.transferred| Total number of messages transferred|
+|spouts.tasks| Total number of tasks for the spout|
+|spouts.lastError| Shows the last error happened in a spout|
+|spouts.acked| Number of messages acked|
+|spouts.failed| Number of messages failed|
+|bolts| Array of bolt components in the topology|
+|bolts.boltId| Bolt id|
+|bolts.capacity| This value indicates number of mesages executed * average execute latency / time window|
+|bolts.processLatency| Bolt's average time to ack a message after it's received|
+|bolts.executeLatency| Average time for bolt's execute method |
--- End diff --
Same as the spout latency, are these strings? if so we should document the format.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14192148
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
+|topologyStats.windowPretty| Duration passed in HH:MM:SS format|
+|topologyStats.window| User requested time window for metrics|
+|topologyStats.emitted| Number of messages emitted in given window|
+|topologyStats.trasferred| Number messages transferred in given window|
+|topologyStats.acked| Number of messages acked in given window|
+|topologyStats.failed| Number of messages failed in given window|
+|spouts| Array of all the spout components in the topology|
+|spouts.spoutId| Spout id|
+|spouts.executors| Number of executors for the spout|
+|spouts.completeLatency| Total latency for processing the message|
+|spouts.transferred| Total number of messages transferred|
+|spouts.tasks| Total number of tasks for the spout|
+|spouts.lastError| Shows the last error happened in a spout|
+|spouts.acked| Number of messages acked|
+|spouts.failed| Number of messages failed|
+|bolts| Array of bolt components in the topology|
+|bolts.boltId| Bolt id|
+|bolts.capacity| This value indicates number of mesages executed * average execute latency / time window|
+|bolts.processLatency| Bolt's average time to ack a message after it's received|
+|bolts.executeLatency| Average time for bolt's execute method |
+|bolts.executors| Number of executor tasks in the bolt component|
+|bolts.tasks| Number of instances of bolt|
+|bolts.acked| Number of tuples acked by the bolt|
+|bolts.failed| Number of tuples failed by the bolt|
+|bolts.lastError| Shows the last error occured in the bolt|
+|bolts.emitted| Number of tuples emitted|
+
+
+
+Examples:
+```no-highlight
+ 1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825
+ 2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?sys=1
+ 3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?window=600
+```
+
+Sample Response:
+```json
+ {
+ "name": "WordCount3",
+ "id": "WordCount3-1-1402960825",
+ "workersTotal": 3,
+ "window": "600",
+ "status": "ACTIVE",
+ "tasksTotal": 28,
+ "executorsTotal": 28,
+ "uptime": "29m 19s",
+ "msgTimeout": 30,
+ "windowHint": "10m 0s",
+ "topologyStats": [
+ {
+ "windowPretty": "10m 0s",
+ "window": "600",
+ "emitted": 397960,
+ "transferred": 213380,
+ "completeLatency": "0.000",
+ "acked": 213460,
+ "failed": 0
+ },
+ {
+ "windowPretty": "3h 0m 0s",
+ "window": "10800",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ },
+ {
+ "windowPretty": "1d 0h 0m 0s",
+ "window": "86400",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ },
+ {
+ "windowPretty": "All time",
+ "window": ":all-time",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ }
+ ],
+ "spouts": [
+ {
+ "executors": 5,
+ "emitted": 28880,
+ "completeLatency": "0.000",
+ "transferred": 28880,
+ "acked": 0,
+ "spoutId": "spout",
+ "tasks": 5,
+ "lastError": "",
+ "failed": 0
+ }
+ ],
+ "bolts": [
+ {
+ "executors": 12,
+ "emitted": 184580,
+ "transferred": 0,
+ "acked": 184640,
+ "executeLatency": "0.048",
+ "tasks": 12,
+ "executed": 184620,
+ "processLatency": "0.043",
+ "boltId": "count",
+ "lastError": "",
+ "capacity": "0.003",
+ "failed": 0
+ },
+ {
+ "executors": 8,
+ "emitted": 184500,
+ "transferred": 184500,
+ "acked": 28820,
+ "executeLatency": "0.024",
+ "tasks": 8,
+ "executed": 28780,
+ "processLatency": "2.112",
+ "boltId": "split",
+ "lastError": "",
+ "capacity": "0.000",
+ "failed": 0
+ }
+ ],
+ "configuration": {
+ "storm.id": "WordCount3-1-1402960825",
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ "storm.local.dir": "storm-local",
+ "storm.messaging.netty.buffer_size": 5242880,
+ "supervisor.worker.start.timeout.secs": 120,
+ "topology.enable.message.timeouts": true,
+ "nimbus.cleanup.inbox.freq.secs": 600,
+ "nimbus.inbox.jar.expiration.secs": 3600,
+ "drpc.worker.threads": 64,
+ "topology.worker.shared.thread.pool.size": 4,
+ "nimbus.host": "hw10843.local",
+ "storm.messaging.netty.min_wait_ms": 100,
+ "storm.zookeeper.port": 2181,
+ "transactional.zookeeper.port": null,
+ "topology.executor.receive.buffer.size": 1024,
+ "transactional.zookeeper.servers": null,
+ "storm.zookeeper.root": "/storm",
+ "storm.zookeeper.retry.intervalceiling.millis": 30000,
+ "supervisor.enable": true,
+ "storm.messaging.netty.server_worker_threads": 1
+ },
+}
+```
+
+
+### /api/v1/topology/:id/component/:component (GET)
+
+Returns detailed metrics and executor information
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|component |String (required)| Component Id |
+|window |String. Default value :all-time| window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| controls including sys stats part of the response|
+
+
+Examples:
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout
+2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?sys=1
+3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?window=600
+```
+
--- End diff --
I missed it I'll document the response fields
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the pull request:
https://github.com/apache/incubator-storm/pull/157#issuecomment-47585041
+1 look good.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on the pull request:
https://github.com/apache/incubator-storm/pull/157#issuecomment-47116198
Thanks for the feedback.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-storm/pull/157
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on the pull request:
https://github.com/apache/incubator-storm/pull/157#issuecomment-47107577
Did one pass through the documentation for the most part it looks really good. Thanks for doing this.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14188180
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
+|topologyStats.windowPretty| Duration passed in HH:MM:SS format|
+|topologyStats.window| User requested time window for metrics|
+|topologyStats.emitted| Number of messages emitted in given window|
+|topologyStats.trasferred| Number messages transferred in given window|
+|topologyStats.acked| Number of messages acked in given window|
+|topologyStats.failed| Number of messages failed in given window|
+|spouts| Array of all the spout components in the topology|
+|spouts.spoutId| Spout id|
+|spouts.executors| Number of executors for the spout|
+|spouts.completeLatency| Total latency for processing the message|
+|spouts.transferred| Total number of messages transferred|
+|spouts.tasks| Total number of tasks for the spout|
+|spouts.lastError| Shows the last error happened in a spout|
+|spouts.acked| Number of messages acked|
+|spouts.failed| Number of messages failed|
+|bolts| Array of bolt components in the topology|
+|bolts.boltId| Bolt id|
+|bolts.capacity| This value indicates number of mesages executed * average execute latency / time window|
+|bolts.processLatency| Bolt's average time to ack a message after it's received|
+|bolts.executeLatency| Average time for bolt's execute method |
+|bolts.executors| Number of executor tasks in the bolt component|
+|bolts.tasks| Number of instances of bolt|
+|bolts.acked| Number of tuples acked by the bolt|
+|bolts.failed| Number of tuples failed by the bolt|
+|bolts.lastError| Shows the last error occured in the bolt|
+|bolts.emitted| Number of tuples emitted|
+
+
+
+Examples:
+```no-highlight
+ 1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825
+ 2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?sys=1
+ 3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?window=600
+```
+
+Sample Response:
+```json
+ {
+ "name": "WordCount3",
+ "id": "WordCount3-1-1402960825",
+ "workersTotal": 3,
+ "window": "600",
+ "status": "ACTIVE",
+ "tasksTotal": 28,
+ "executorsTotal": 28,
+ "uptime": "29m 19s",
+ "msgTimeout": 30,
+ "windowHint": "10m 0s",
+ "topologyStats": [
+ {
+ "windowPretty": "10m 0s",
+ "window": "600",
+ "emitted": 397960,
+ "transferred": 213380,
+ "completeLatency": "0.000",
+ "acked": 213460,
+ "failed": 0
+ },
+ {
+ "windowPretty": "3h 0m 0s",
+ "window": "10800",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ },
+ {
+ "windowPretty": "1d 0h 0m 0s",
+ "window": "86400",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ },
+ {
+ "windowPretty": "All time",
+ "window": ":all-time",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ }
+ ],
+ "spouts": [
+ {
+ "executors": 5,
+ "emitted": 28880,
+ "completeLatency": "0.000",
+ "transferred": 28880,
+ "acked": 0,
+ "spoutId": "spout",
+ "tasks": 5,
+ "lastError": "",
+ "failed": 0
+ }
+ ],
+ "bolts": [
+ {
+ "executors": 12,
+ "emitted": 184580,
+ "transferred": 0,
+ "acked": 184640,
+ "executeLatency": "0.048",
+ "tasks": 12,
+ "executed": 184620,
+ "processLatency": "0.043",
+ "boltId": "count",
+ "lastError": "",
+ "capacity": "0.003",
+ "failed": 0
+ },
+ {
+ "executors": 8,
+ "emitted": 184500,
+ "transferred": 184500,
+ "acked": 28820,
+ "executeLatency": "0.024",
+ "tasks": 8,
+ "executed": 28780,
+ "processLatency": "2.112",
+ "boltId": "split",
+ "lastError": "",
+ "capacity": "0.000",
+ "failed": 0
+ }
+ ],
+ "configuration": {
+ "storm.id": "WordCount3-1-1402960825",
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ "storm.local.dir": "storm-local",
+ "storm.messaging.netty.buffer_size": 5242880,
+ "supervisor.worker.start.timeout.secs": 120,
+ "topology.enable.message.timeouts": true,
+ "nimbus.cleanup.inbox.freq.secs": 600,
+ "nimbus.inbox.jar.expiration.secs": 3600,
+ "drpc.worker.threads": 64,
+ "topology.worker.shared.thread.pool.size": 4,
+ "nimbus.host": "hw10843.local",
+ "storm.messaging.netty.min_wait_ms": 100,
+ "storm.zookeeper.port": 2181,
+ "transactional.zookeeper.port": null,
+ "topology.executor.receive.buffer.size": 1024,
+ "transactional.zookeeper.servers": null,
+ "storm.zookeeper.root": "/storm",
+ "storm.zookeeper.retry.intervalceiling.millis": 30000,
+ "supervisor.enable": true,
+ "storm.messaging.netty.server_worker_threads": 1
+ },
+}
+```
+
+
+### /api/v1/topology/:id/component/:component (GET)
+
+Returns detailed metrics and executor information
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|component |String (required)| Component Id |
+|window |String. Default value :all-time| window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| controls including sys stats part of the response|
+
+
+Examples:
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout
+2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?sys=1
+3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?window=600
+```
+
--- End diff --
Is there any plan to document the response fields?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14187961
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
+|topologyStats.windowPretty| Duration passed in HH:MM:SS format|
+|topologyStats.window| User requested time window for metrics|
+|topologyStats.emitted| Number of messages emitted in given window|
+|topologyStats.trasferred| Number messages transferred in given window|
+|topologyStats.acked| Number of messages acked in given window|
+|topologyStats.failed| Number of messages failed in given window|
+|spouts| Array of all the spout components in the topology|
+|spouts.spoutId| Spout id|
+|spouts.executors| Number of executors for the spout|
+|spouts.completeLatency| Total latency for processing the message|
--- End diff --
If this is a string, like the example suggests then we should document it here, otherwise we should modify the example.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14187503
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
--- End diff --
It would be good to document this format, so that others can reliably parse it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by harshach <gi...@git.apache.org>.
Github user harshach commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14192389
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
--- End diff --
By format you mean type of the fields?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14200066
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
+|slotsTotal| Total number of available worker slots for this supervisor|
+|slotsUsed| Number of worker slots used on this supervisor|
+
+Sample Response:
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "slotsTotal": 4,
+ "slotsUsed": 3
+ }
+ ]
+}
+```
+
+### /api/v1/topology/summary (GET)
+Returns all topologies summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+
+Sample Response:
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28
+ }
+ ]
+}
+```
+
+### /api/v1/topology/:id (GET)
+ Returns details topology information. Subsititute id with topology id.
+
+Request Parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in ms|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Topology Id|
+|name| Topology Name|
+|uptime| Shows how long the topology is running|
+|tasksTotal| Total number of tasks for this topology|
+|workersTotal| Number of workers used for this topology|
+|executorsTotal| Number of executors used for this topology|
+|msgTimeout| |
+|windowHint| |
+|topologyStats.windowPretty| Duration passed in HH:MM:SS format|
+|topologyStats.window| User requested time window for metrics|
+|topologyStats.emitted| Number of messages emitted in given window|
+|topologyStats.trasferred| Number messages transferred in given window|
+|topologyStats.acked| Number of messages acked in given window|
+|topologyStats.failed| Number of messages failed in given window|
+|spouts| Array of all the spout components in the topology|
+|spouts.spoutId| Spout id|
+|spouts.executors| Number of executors for the spout|
+|spouts.completeLatency| Total latency for processing the message|
+|spouts.transferred| Total number of messages transferred|
+|spouts.tasks| Total number of tasks for the spout|
+|spouts.lastError| Shows the last error happened in a spout|
+|spouts.acked| Number of messages acked|
+|spouts.failed| Number of messages failed|
+|bolts| Array of bolt components in the topology|
+|bolts.boltId| Bolt id|
+|bolts.capacity| This value indicates number of mesages executed * average execute latency / time window|
+|bolts.processLatency| Bolt's average time to ack a message after it's received|
+|bolts.executeLatency| Average time for bolt's execute method |
--- End diff --
I think having a type would be good, especially in the cases where it is a string holding a number.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14187457
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
--- End diff --
s/runninge/running/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14187640
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
--- End diff --
We should document too that any unknown fields in the response should be ignored. This allows us to add new fields and still maintain backwards compatibility.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by ptgoetz <gi...@git.apache.org>.
Github user ptgoetz commented on the pull request:
https://github.com/apache/incubator-storm/pull/157#issuecomment-46918426
+1
If this is approved, I plan on pulling this in via the Storm website documentation (in svn), and asking @harshach to close the pull request.
I don't want to clutter up the root directory too much.
I will also add docs for updating the website.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-storm pull request: STORM-367. Storm UI REST api documen...
Posted by revans2 <gi...@git.apache.org>.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/incubator-storm/pull/157#discussion_r14187663
--- Diff: STORM-UI-REST-API.md ---
@@ -0,0 +1,513 @@
+# Storm UI REST API
+Storm UI server provides a REST Api to access cluster, topology, component overview and metrics.
+This api returns json response.
+
+## Using the UI REST Api
+
+### /api/v1/cluster/configuration (GET)
+ returns cluster configuration.
+
+Sample Response:
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": true,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+returns cluster summary such as nimbus uptime,number of supervisors,slots etc..
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|stormVersion| Storm version|
+|nimbusUptime| Shows how long the cluster is running|
+|supervisors| Number of supervisors running|
+|slotsTotal| Total number of available worker slots|
+|slotsUsed| Number of worker slots used|
+|slotsFree| Number of worker slots available|
+|executorsTotal| Total number of executors|
+|tasksTotal| Total tasks|
+
+Sample Response:
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "nimbusUptime": "3m 53s",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+returns all supervisors summary
+
+Response Fields:
+
+|Field |Description|
+|--- |--- |
+|id| Supervisor's id|
+|host| Supervisor's host name|
+|uptime| Shows how long the supervisor is runninge|
--- End diff --
Similarly we should document the format
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---