You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by ra...@apache.org on 2026/05/18 22:02:57 UTC
(storm-site) branch main updated (31ed887c32 -> 8a696cb52d)
This is an automated email from the ASF dual-hosted git repository.
reiabreu pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/storm-site.git
from 31ed887c32 Set up default protection ruleset for default and release branches (#72)
new ade1ec25f7 Releasing Apache Storm 2.8.8
new 8a696cb52d Releasing Apache Storm 2.8.8
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_posts/2026-05-18-storm288-released.md | 68 +++++++++++++++++++++
.../Acking-framework-implementation.md | 0
releases/{2.6.0 => 2.8.8}/Classpath-handling.md | 0
releases/{2.5.0 => 2.8.8}/Clojure-DSL.md | 4 +-
releases/{2.6.0 => 2.8.8}/ClusterMetrics.md | 0
releases/{2.5.0 => 2.8.8}/Command-line-client.md | 2 +-
releases/{2.6.0 => 2.8.8}/Common-patterns.md | 6 +-
releases/{2.8.0 => 2.8.8}/Concepts.md | 42 ++++++-------
releases/{2.3.0 => 2.8.8}/Configuration.md | 10 +--
releases/{1.2.3 => 2.8.8}/Contributing-to-Storm.md | 0
.../Creating-a-new-Storm-project.md | 0
.../DSLs-and-multilang-adapters.md | 0
.../{1.2.3 => 2.8.8}/Daemon-Fault-Tolerance.md | 0
.../Defining-a-non-jvm-language-dsl-for-storm.md | 0
releases/{2.8.0 => 2.8.8}/Distributed-RPC.md | 2 +-
releases/{2.3.0 => 2.8.8}/Docker-support.md | 2 +-
releases/{2.0.0 => 2.8.8}/Eventlogging.md | 4 +-
releases/{2.7.0 => 2.8.8}/FAQ.md | 0
releases/{1.2.3 => 2.8.8}/Fault-tolerance.md | 0
releases/{2.2.0 => 2.8.8}/Generic-resources.md | 0
.../Guaranteeing-message-processing.md | 8 +--
releases/{2.6.0 => 2.8.8}/Hooks.md | 10 +--
releases/{2.0.0 => 2.8.8}/IConfigLoader.md | 0
releases/{2.8.3 => 2.8.8}/Implementation-docs.md | 0
.../Installing-native-dependencies.md | 0
releases/{1.2.3 => 2.8.8}/Joins.md | 0
releases/{2.0.0 => 2.8.8}/Kestrel-and-Storm.md | 0
.../{2.0.0 => 2.8.8}/Lifecycle-of-a-topology.md | 0
releases/{2.0.0 => 2.8.8}/Local-mode.md | 4 +-
releases/{2.3.0 => 2.8.8}/LocalityAwareness.md | 0
releases/{2.3.0 => 2.8.8}/Logs.md | 0
releases/{2.0.0 => 2.8.8}/Maven.md | 0
.../Message-passing-implementation.md | 0
releases/{2.6.3 => 2.8.8}/Metrics.md | 6 +-
releases/{2.5.0 => 2.8.8}/Multilang-protocol.md | 0
releases/{2.6.0 => 2.8.8}/NUMA.md | 0
releases/{2.5.0 => 2.8.8}/OCI-support.md | 0
releases/{2.0.0 => 2.8.8}/Pacemaker.md | 0
releases/{2.6.0 => 2.8.8}/Performance.md | 0
releases/{2.8.2 => 2.8.8}/Project-ideas.md | 0
releases/{1.2.3 => 2.8.8}/Rationale.md | 0
.../Resource_Aware_Scheduler_overview.md | 0
.../Running-topologies-on-a-production-cluster.md | 6 +-
releases/{2.8.7 => 2.8.8}/SECURITY.md | 2 +-
releases/{2.8.7 => 2.8.8}/STORM-UI-REST-API.md | 0
.../Serialization-(prior-to-0.6.0).md | 4 +-
releases/{2.2.1 => 2.8.8}/Serialization.md | 2 +-
releases/{1.2.3 => 2.8.8}/Serializers.md | 0
.../{2.6.0 => 2.8.8}/Setting-up-a-Storm-cluster.md | 0
.../Setting-up-a-Storm-project-in-Eclipse.md | 0
.../Setting-up-development-environment.md | 0
releases/{1.2.3 => 2.8.8}/Spout-implementations.md | 0
releases/{2.8.3 => 2.8.8}/State-checkpointing.md | 0
releases/{2.0.0 => 2.8.8}/Storm-Scheduler.md | 0
...language-protocol-(versions-0.7.0-and-below).md | 0
releases/{2.5.0 => 2.8.8}/Stream-API.md | 0
.../{2.1.0 => 2.8.8}/Structure-of-the-codebase.md | 10 +--
.../Support-for-non-java-languages.md | 0
.../{2.8.0 => 2.8.8}/Transactional-topologies.md | 18 +++---
releases/{2.1.0 => 2.8.8}/Trident-API-Overview.md | 2 +-
releases/{1.2.3 => 2.8.8}/Trident-RAS-API.md | 0
releases/{2.0.0 => 2.8.8}/Trident-spouts.md | 0
releases/{2.2.0 => 2.8.8}/Trident-state.md | 0
releases/{2.0.0 => 2.8.8}/Trident-tutorial.md | 0
releases/{2.6.2 => 2.8.8}/Troubleshooting.md | 0
releases/{2.8.0 => 2.8.8}/Tutorial.md | 4 +-
...standing-the-parallelism-of-a-Storm-topology.md | 16 ++---
.../Using-non-JVM-languages-with-Storm.md | 0
releases/{2.5.0 => 2.8.8}/Windowing.md | 0
releases/{2.3.0 => 2.8.8}/cgroups_in_storm.md | 0
releases/{2.5.0 => 2.8.8}/distcache-blobstore.md | 0
.../{1.2.3 => 2.8.8}/dynamic-log-level-settings.md | 0
.../{1.2.3 => 2.8.8}/dynamic-worker-profiling.md | 0
favicon.ico => releases/2.8.8/favicon.ico | Bin
releases/{2.6.0 => 2.8.8}/flux.md | 0
.../{1.2.3 => 2.8.8}/images/Storm-JMS-Example.png | Bin
releases/{1.2.3 => 2.8.8}/images/ack_tree.png | Bin
{images => releases/2.8.8/images}/architecture.png | Bin
{images => releases/2.8.8/images}/architecture.svg | 0
.../{1.2.3 => 2.8.8}/images/batched-stream.png | Bin
{images => releases/2.8.8/images}/bolt.png | Bin
releases/{1.2.3 => 2.8.8}/images/bolt.svg | 0
{images => releases/2.8.8/images}/bullet.gif | Bin
.../images/disable-event-logging-topology.png | Bin
{images => releases/2.8.8/images}/download.png | Bin
releases/{1.2.3 => 2.8.8}/images/drpc-workflow.png | Bin
.../images/dynamic_log_level_settings_1.png | Bin
.../images/dynamic_log_level_settings_2.png | Bin
.../images/dynamic_profiling_debugging_1.png | Bin
.../images/dynamic_profiling_debugging_2.png | Bin
.../images/dynamic_profiling_debugging_3.png | Bin
.../images/dynamic_profiling_debugging_4.png | Bin
.../images/eclipse-project-properties.png | Bin
.../images/enable-event-logging-spout.png | Bin
.../images/enable-event-logging-topology.png | Bin
.../{1.2.3 => 2.8.8}/images/event-logs-view.png | Bin
.../images/example-of-a-running-topology.png | Bin
releases/{1.2.3 => 2.8.8}/images/footer-bg.png | Bin
releases/{1.2.3 => 2.8.8}/images/grouping.png | Bin
.../{1.2.3 => 2.8.8}/images/hdfs_blobstore.png | Bin
releases/{1.2.3 => 2.8.8}/images/header-bg.png | Bin
.../2.8.8/images}/incubator-logo.png | Bin
.../images/ld-library-path-eclipse-linux.png | Bin
{images => releases/2.8.8/images}/loading.gif | Bin
.../{1.2.3 => 2.8.8}/images/local_blobstore.png | Bin
{images => releases/2.8.8/images}/logo.png | Bin
releases/{1.2.3 => 2.8.8}/images/logos/aeris.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/alibaba.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/bai.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/cerner.jpg | Bin
.../{1.2.3 => 2.8.8}/images/logos/flipboard.jpg | Bin
.../{1.2.3 => 2.8.8}/images/logos/fullcontact.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/groupon.jpg | Bin
.../images/logos/health-market-science.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/images.png | Bin
.../{1.2.3 => 2.8.8}/images/logos/infochimp.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/klout.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/loggly.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/ooyala.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/parc.png | Bin
releases/{1.2.3 => 2.8.8}/images/logos/premise.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/qiy.jpg | Bin
.../{1.2.3 => 2.8.8}/images/logos/quicklizard.jpg | Bin
.../{1.2.3 => 2.8.8}/images/logos/rocketfuel.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/rubicon.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/spider.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/spotify.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/taobao.jpg | Bin
.../images/logos/the-weather-channel.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/twitter.jpg | Bin
.../{1.2.3 => 2.8.8}/images/logos/verisign.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/webmd.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/wego.jpg | Bin
.../{1.2.3 => 2.8.8}/images/logos/yahoo-japan.jpg | Bin
releases/{1.2.3 => 2.8.8}/images/logos/yahoo.png | Bin
releases/{1.2.3 => 2.8.8}/images/logos/yelp.jpg | Bin
{images => releases/2.8.8/images}/mailinglist.png | Bin
.../images/nimbus_ha_blobstore.png | Bin
.../nimbus_ha_leader_election_and_failover.png | Bin
.../images/nimbus_ha_topology_submission.png | Bin
.../images/ras_new_strategy_network_cdf_random.png | Bin
...trategy_network_metric_cdf_yahoo_topologies.png | Bin
..._strategy_network_metric_improvement_random.png | Bin
.../ras_new_strategy_network_metric_random.png | Bin
...ew_strategy_network_metric_yahoo_topologies.png | Bin
.../images/ras_new_strategy_runtime_random.png | Bin
.../images/ras_new_strategy_runtime_yahoo.png | Bin
...ationships-worker-processes-executors-tasks.png | Bin
...e_aware_scheduler_default_eviction_strategy.png | Bin
...e_aware_scheduler_default_eviction_strategy.svg | 0
.../{1.2.3 => 2.8.8}/images/search-a-topology.png | Bin
.../images/search-for-a-single-worker-log.png | Bin
{images => releases/2.8.8/images}/security.png | Bin
{images => releases/2.8.8/images}/security.svg | 0
.../{1.2.3 => 2.8.8}/images/spout-vs-state.png | Bin
{images => releases/2.8.8/images}/spout.png | Bin
releases/{1.2.3 => 2.8.8}/images/spout.svg | 0
.../2.8.8/images}/storm-cluster.png | Bin
{images => releases/2.8.8/images}/storm-flow.png | Bin
...m-sql-internal-example-exported-from-drawio.xml | 0
.../images/storm-sql-internal-example.png | Bin
...-sql-internal-workflow-exported-from-drawio.xml | 0
.../images/storm-sql-internal-workflow.png | Bin
{images => releases/2.8.8/images}/storm.svg | 0
{images => releases/2.8.8/images}/storm_header.png | Bin
.../2.8.8/images}/storm_logo_tagline_color.png | Bin
releases/{2.2.0 => 2.8.8}/images/storm_ui.png | Bin
.../{1.2.3 => 2.8.8}/images/supervisor_page.png | Bin
{images => releases/2.8.8/images}/top_bg.gif | Bin
.../{1.2.3 => 2.8.8}/images/topology-tasks.png | Bin
releases/{1.2.3 => 2.8.8}/images/topology.png | Bin
{images => releases/2.8.8/images}/topology.svg | 0
.../2.8.8/images}/topology_dark.png | Bin
.../2.8.8/images}/topology_dark.svg | 0
.../images/transactional-batches.png | Bin
.../images/transactional-commit-flow.png | Bin
.../images/transactional-design-2.png | Bin
.../images/transactional-spout-structure.png | Bin
.../{1.2.3 => 2.8.8}/images/trident-to-storm1.png | Bin
.../{1.2.3 => 2.8.8}/images/trident-to-storm2.png | Bin
releases/{1.2.3 => 2.8.8}/images/tuple-dag.png | Bin
releases/{1.2.3 => 2.8.8}/images/tuple_tree.png | Bin
.../2.8.8/images}/ui_topology_viz.png | Bin
releases/{2.8.3 => 2.8.8}/index.md | 2 +-
releases/{2.3.0 => 2.8.8}/metrics_v2.md | 0
releases/{1.2.3 => 2.8.8}/nimbus-ha-design.md | 0
releases/{2.0.0 => 2.8.8}/storm-hdfs.md | 0
releases/{2.1.0 => 2.8.8}/storm-jdbc.md | 0
releases/{2.5.0 => 2.8.8}/storm-jms-example.md | 0
releases/{2.5.0 => 2.8.8}/storm-jms-spring.md | 0
releases/{1.2.3 => 2.8.8}/storm-jms.md | 0
releases/{2.1.1 => 2.8.8}/storm-kafka-client.md | 22 +++----
releases/{2.0.0 => 2.8.8}/storm-metricstore.md | 0
releases/{2.0.0 => 2.8.8}/storm-redis.md | 0
releases/{1.2.3 => 2.8.8}/windows-users-guide.md | 0
releases/current | 2 +-
196 files changed, 163 insertions(+), 95 deletions(-)
create mode 100644 _posts/2026-05-18-storm288-released.md
copy releases/{2.0.0 => 2.8.8}/Acking-framework-implementation.md (100%)
copy releases/{2.6.0 => 2.8.8}/Classpath-handling.md (100%)
copy releases/{2.5.0 => 2.8.8}/Clojure-DSL.md (99%)
copy releases/{2.6.0 => 2.8.8}/ClusterMetrics.md (100%)
copy releases/{2.5.0 => 2.8.8}/Command-line-client.md (99%)
copy releases/{2.6.0 => 2.8.8}/Common-patterns.md (95%)
copy releases/{2.8.0 => 2.8.8}/Concepts.md (87%)
copy releases/{2.3.0 => 2.8.8}/Configuration.md (90%)
copy releases/{1.2.3 => 2.8.8}/Contributing-to-Storm.md (100%)
copy releases/{1.2.3 => 2.8.8}/Creating-a-new-Storm-project.md (100%)
copy releases/{2.0.0 => 2.8.8}/DSLs-and-multilang-adapters.md (100%)
copy releases/{1.2.3 => 2.8.8}/Daemon-Fault-Tolerance.md (100%)
copy releases/{2.5.0 => 2.8.8}/Defining-a-non-jvm-language-dsl-for-storm.md (100%)
copy releases/{2.8.0 => 2.8.8}/Distributed-RPC.md (99%)
copy releases/{2.3.0 => 2.8.8}/Docker-support.md (99%)
copy releases/{2.0.0 => 2.8.8}/Eventlogging.md (97%)
copy releases/{2.7.0 => 2.8.8}/FAQ.md (100%)
copy releases/{1.2.3 => 2.8.8}/Fault-tolerance.md (100%)
copy releases/{2.2.0 => 2.8.8}/Generic-resources.md (100%)
copy releases/{1.2.4 => 2.8.8}/Guaranteeing-message-processing.md (98%)
copy releases/{2.6.0 => 2.8.8}/Hooks.md (88%)
copy releases/{2.0.0 => 2.8.8}/IConfigLoader.md (100%)
copy releases/{2.8.3 => 2.8.8}/Implementation-docs.md (100%)
copy releases/{1.2.3 => 2.8.8}/Installing-native-dependencies.md (100%)
copy releases/{1.2.3 => 2.8.8}/Joins.md (100%)
copy releases/{2.0.0 => 2.8.8}/Kestrel-and-Storm.md (100%)
copy releases/{2.0.0 => 2.8.8}/Lifecycle-of-a-topology.md (100%)
copy releases/{2.0.0 => 2.8.8}/Local-mode.md (97%)
copy releases/{2.3.0 => 2.8.8}/LocalityAwareness.md (100%)
copy releases/{2.3.0 => 2.8.8}/Logs.md (100%)
copy releases/{2.0.0 => 2.8.8}/Maven.md (100%)
copy releases/{1.2.3 => 2.8.8}/Message-passing-implementation.md (100%)
copy releases/{2.6.3 => 2.8.8}/Metrics.md (99%)
copy releases/{2.5.0 => 2.8.8}/Multilang-protocol.md (100%)
copy releases/{2.6.0 => 2.8.8}/NUMA.md (100%)
copy releases/{2.5.0 => 2.8.8}/OCI-support.md (100%)
copy releases/{2.0.0 => 2.8.8}/Pacemaker.md (100%)
copy releases/{2.6.0 => 2.8.8}/Performance.md (100%)
copy releases/{2.8.2 => 2.8.8}/Project-ideas.md (100%)
copy releases/{1.2.3 => 2.8.8}/Rationale.md (100%)
copy releases/{2.5.0 => 2.8.8}/Resource_Aware_Scheduler_overview.md (100%)
copy releases/{1.2.3 => 2.8.8}/Running-topologies-on-a-production-cluster.md (96%)
copy releases/{2.8.7 => 2.8.8}/SECURITY.md (99%)
copy releases/{2.8.7 => 2.8.8}/STORM-UI-REST-API.md (100%)
copy releases/{2.0.0 => 2.8.8}/Serialization-(prior-to-0.6.0).md (97%)
copy releases/{2.2.1 => 2.8.8}/Serialization.md (99%)
copy releases/{1.2.3 => 2.8.8}/Serializers.md (100%)
copy releases/{2.6.0 => 2.8.8}/Setting-up-a-Storm-cluster.md (100%)
copy releases/{1.2.3 => 2.8.8}/Setting-up-a-Storm-project-in-Eclipse.md (100%)
copy releases/{1.2.3 => 2.8.8}/Setting-up-development-environment.md (100%)
copy releases/{1.2.3 => 2.8.8}/Spout-implementations.md (100%)
copy releases/{2.8.3 => 2.8.8}/State-checkpointing.md (100%)
copy releases/{2.0.0 => 2.8.8}/Storm-Scheduler.md (100%)
copy releases/{1.2.3 => 2.8.8}/Storm-multi-language-protocol-(versions-0.7.0-and-below).md (100%)
copy releases/{2.5.0 => 2.8.8}/Stream-API.md (100%)
copy releases/{2.1.0 => 2.8.8}/Structure-of-the-codebase.md (97%)
copy releases/{1.2.3 => 2.8.8}/Support-for-non-java-languages.md (100%)
copy releases/{2.8.0 => 2.8.8}/Transactional-topologies.md (98%)
copy releases/{2.1.0 => 2.8.8}/Trident-API-Overview.md (99%)
copy releases/{1.2.3 => 2.8.8}/Trident-RAS-API.md (100%)
copy releases/{2.0.0 => 2.8.8}/Trident-spouts.md (100%)
copy releases/{2.2.0 => 2.8.8}/Trident-state.md (100%)
copy releases/{2.0.0 => 2.8.8}/Trident-tutorial.md (100%)
copy releases/{2.6.2 => 2.8.8}/Troubleshooting.md (100%)
copy releases/{2.8.0 => 2.8.8}/Tutorial.md (99%)
copy releases/{2.1.1 => 2.8.8}/Understanding-the-parallelism-of-a-Storm-topology.md (92%)
copy releases/{2.5.0 => 2.8.8}/Using-non-JVM-languages-with-Storm.md (100%)
copy releases/{2.5.0 => 2.8.8}/Windowing.md (100%)
copy releases/{2.3.0 => 2.8.8}/cgroups_in_storm.md (100%)
copy releases/{2.5.0 => 2.8.8}/distcache-blobstore.md (100%)
copy releases/{1.2.3 => 2.8.8}/dynamic-log-level-settings.md (100%)
copy releases/{1.2.3 => 2.8.8}/dynamic-worker-profiling.md (100%)
copy favicon.ico => releases/2.8.8/favicon.ico (100%)
copy releases/{2.6.0 => 2.8.8}/flux.md (100%)
copy releases/{1.2.3 => 2.8.8}/images/Storm-JMS-Example.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ack_tree.png (100%)
copy {images => releases/2.8.8/images}/architecture.png (100%)
copy {images => releases/2.8.8/images}/architecture.svg (100%)
copy releases/{1.2.3 => 2.8.8}/images/batched-stream.png (100%)
copy {images => releases/2.8.8/images}/bolt.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/bolt.svg (100%)
copy {images => releases/2.8.8/images}/bullet.gif (100%)
copy releases/{1.2.3 => 2.8.8}/images/disable-event-logging-topology.png (100%)
copy {images => releases/2.8.8/images}/download.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/drpc-workflow.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/dynamic_log_level_settings_1.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/dynamic_log_level_settings_2.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/dynamic_profiling_debugging_1.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/dynamic_profiling_debugging_2.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/dynamic_profiling_debugging_3.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/dynamic_profiling_debugging_4.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/eclipse-project-properties.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/enable-event-logging-spout.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/enable-event-logging-topology.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/event-logs-view.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/example-of-a-running-topology.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/footer-bg.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/grouping.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/hdfs_blobstore.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/header-bg.png (100%)
copy {images => releases/2.8.8/images}/incubator-logo.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ld-library-path-eclipse-linux.png (100%)
copy {images => releases/2.8.8/images}/loading.gif (100%)
copy releases/{1.2.3 => 2.8.8}/images/local_blobstore.png (100%)
copy {images => releases/2.8.8/images}/logo.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/aeris.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/alibaba.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/bai.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/cerner.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/flipboard.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/fullcontact.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/groupon.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/health-market-science.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/images.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/infochimp.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/klout.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/loggly.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/ooyala.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/parc.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/premise.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/qiy.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/quicklizard.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/rocketfuel.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/rubicon.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/spider.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/spotify.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/taobao.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/the-weather-channel.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/twitter.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/verisign.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/webmd.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/wego.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/yahoo-japan.jpg (100%)
copy releases/{1.2.3 => 2.8.8}/images/logos/yahoo.png (100%)
mode change 100644 => 100755
copy releases/{1.2.3 => 2.8.8}/images/logos/yelp.jpg (100%)
copy {images => releases/2.8.8/images}/mailinglist.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/nimbus_ha_blobstore.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/nimbus_ha_leader_election_and_failover.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/nimbus_ha_topology_submission.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_network_cdf_random.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_network_metric_cdf_yahoo_topologies.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_network_metric_improvement_random.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_network_metric_random.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_network_metric_yahoo_topologies.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_runtime_random.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/ras_new_strategy_runtime_yahoo.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/relationships-worker-processes-executors-tasks.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/resource_aware_scheduler_default_eviction_strategy.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/resource_aware_scheduler_default_eviction_strategy.svg (100%)
copy releases/{1.2.3 => 2.8.8}/images/search-a-topology.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/search-for-a-single-worker-log.png (100%)
copy {images => releases/2.8.8/images}/security.png (100%)
copy {images => releases/2.8.8/images}/security.svg (100%)
copy releases/{1.2.3 => 2.8.8}/images/spout-vs-state.png (100%)
copy {images => releases/2.8.8/images}/spout.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/spout.svg (100%)
copy {images => releases/2.8.8/images}/storm-cluster.png (100%)
copy {images => releases/2.8.8/images}/storm-flow.png (100%)
copy releases/{2.0.0 => 2.8.8}/images/storm-sql-internal-example-exported-from-drawio.xml (100%)
copy releases/{2.0.0 => 2.8.8}/images/storm-sql-internal-example.png (100%)
copy releases/{2.0.0 => 2.8.8}/images/storm-sql-internal-workflow-exported-from-drawio.xml (100%)
copy releases/{2.0.0 => 2.8.8}/images/storm-sql-internal-workflow.png (100%)
copy {images => releases/2.8.8/images}/storm.svg (100%)
copy {images => releases/2.8.8/images}/storm_header.png (100%)
copy {images => releases/2.8.8/images}/storm_logo_tagline_color.png (100%)
copy releases/{2.2.0 => 2.8.8}/images/storm_ui.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/supervisor_page.png (100%)
copy {images => releases/2.8.8/images}/top_bg.gif (100%)
copy releases/{1.2.3 => 2.8.8}/images/topology-tasks.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/topology.png (100%)
copy {images => releases/2.8.8/images}/topology.svg (100%)
copy {images => releases/2.8.8/images}/topology_dark.png (100%)
copy {images => releases/2.8.8/images}/topology_dark.svg (100%)
copy releases/{1.2.3 => 2.8.8}/images/transactional-batches.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/transactional-commit-flow.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/transactional-design-2.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/transactional-spout-structure.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/trident-to-storm1.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/trident-to-storm2.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/tuple-dag.png (100%)
copy releases/{1.2.3 => 2.8.8}/images/tuple_tree.png (100%)
copy {images => releases/2.8.8/images}/ui_topology_viz.png (100%)
copy releases/{2.8.3 => 2.8.8}/index.md (99%)
copy releases/{2.3.0 => 2.8.8}/metrics_v2.md (100%)
copy releases/{1.2.3 => 2.8.8}/nimbus-ha-design.md (100%)
copy releases/{2.0.0 => 2.8.8}/storm-hdfs.md (100%)
copy releases/{2.1.0 => 2.8.8}/storm-jdbc.md (100%)
copy releases/{2.5.0 => 2.8.8}/storm-jms-example.md (100%)
copy releases/{2.5.0 => 2.8.8}/storm-jms-spring.md (100%)
copy releases/{1.2.3 => 2.8.8}/storm-jms.md (100%)
copy releases/{2.1.1 => 2.8.8}/storm-kafka-client.md (96%)
copy releases/{2.0.0 => 2.8.8}/storm-metricstore.md (100%)
copy releases/{2.0.0 => 2.8.8}/storm-redis.md (100%)
copy releases/{1.2.3 => 2.8.8}/windows-users-guide.md (100%)
(storm-site) 02/02: Releasing Apache Storm 2.8.8
Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
reiabreu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/storm-site.git
commit 8a696cb52d55f89daa91f5d150d30eb06f0bfc0e
Author: Rui Abreu <ru...@gmail.com>
AuthorDate: Mon May 18 22:53:38 2026 +0100
Releasing Apache Storm 2.8.8
---
_posts/2026-05-18-storm288-released.md | 68 ++++++++++++++++++++++++++++++++++
releases/current | 2 +-
2 files changed, 69 insertions(+), 1 deletion(-)
diff --git a/_posts/2026-05-18-storm288-released.md b/_posts/2026-05-18-storm288-released.md
new file mode 100644
index 0000000000..140beed6af
--- /dev/null
+++ b/_posts/2026-05-18-storm288-released.md
@@ -0,0 +1,68 @@
+---
+layout: post
+title: Apache Storm 2.8.8 Released
+author: Rui Abreu
+---
+
+The Apache Storm community is pleased to announce that version 2.8.8 has been
+released and is available from [the downloads page](/downloads.html).
+
+This release includes security fixes, new features, bug fixes, and library updates.
+We encourage users of previous versions to upgrade to this latest release.
+
+## Thanks
+
+Special thanks are due to all those who have contributed to Apache Storm -- whether
+through direct code contributions, documentation, bug reports, or helping other
+users on the mailing lists. Your efforts are much appreciated.
+
+
+## Changes in this Release - Storm 2.8.8
+
+<p>JIRA issues addressed in the 2.8.8 release of Storm. Documentation for this release is available at the <a href="https://storm.apache.org/">Apache Storm project site</a>.</p>
+
+<p>Issues addressed in 2.8.8.</p>
+
+<h2>Bug fixing</h2>
+<ul>
+ <li>[<a href="https://github.com/apache/storm/pull/8591">#8591</a>] - STORM-4041: fix(topology_lag): Kafka Topology Lag breaking when no offsets are committed (Backport #8589)</li>
+</ul>
+
+<h2>Dependency upgrades</h2>
+<ul>
+ <li>[<a href="https://github.com/apache/storm/pull/8643">#8643</a>] - Bump com.fasterxml.jackson.core:jackson-databind from 2.21.2 to 2.21.3</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8642">#8642</a>] - Bump joda-time:joda-time from 2.14.1 to 2.14.2</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8641">#8641</a>] - Bump org.checkerframework:checker-qual from 3.55.1 to 4.1.0</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8640">#8640</a>] - Bump io.netty:netty-bom from 4.2.12.Final to 4.2.13.Final</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8639">#8639</a>] - Bump mockito.version from 4.11.0 to 5.23.0</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8637">#8637</a>] - Bump org.yaml:snakeyaml from 2.2 to 2.6</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8636">#8636</a>] - Bump commons-io:commons-io from 2.21.0 to 2.22.0</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8635">#8635</a>] - Bump activemq.version from 6.2.4 to 6.2.5</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8632">#8632</a>] - Bump org.glassfish.jersey:jersey-bom from 3.1.11 to 4.0.2</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8631">#8631</a>] - Bump org.apache.maven:maven-resolver-provider from 3.9.14 to 3.9.15</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8630">#8630</a>] - Bump com.google.code.gson:gson from 2.13.2 to 2.14.0</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8628">#8628</a>] - Bump netty-tcnative.version from 2.0.75.Final to 2.0.77.Final</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8627">#8627</a>] - Bump com.puppycrawl.tools:checkstyle from 8.2 to 13.4.2</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8626">#8626</a>] - Bump redis.clients:jedis from 7.4.1 to 7.5.0</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8624">#8624</a>] - Bump com.github.ben-manes.caffeine:caffeine from 3.2.3 to 3.2.4</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8622">#8622</a>] - Bump jetty.version from 12.1.8 to 12.1.9</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8621">#8621</a>] - Bump datatables.net-bs5 from 2.3.7 to 2.3.8 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8620">#8620</a>] - Bump start-server-and-test from 3.0.0 to 3.0.4 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8619">#8619</a>] - Bump cytoscape from 3.33.2 to 3.33.3 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8617">#8617</a>] - Bump datatables.net-dt from 2.3.7 to 2.3.8 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8614">#8614</a>] - Bump prometheus.client.version from 1.5.1 to 1.6.1</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8613">#8613</a>] - Bump webpack from 5.105.4 to 5.106.2 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8610">#8610</a>] - Bump terser-webpack-plugin from 5.4.0 to 5.6.0 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8609">#8609</a>] - Bump datatables.net from 2.3.7 to 2.3.8 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8607">#8607</a>] - Bump commons-codec:commons-codec from 1.21.0 to 1.22.0</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8606">#8606</a>] - Bump vis-network from 10.0.2 to 10.0.3 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8604">#8604</a>] - Bump org.apache:apache from 37 to 38</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8602">#8602</a>] - Bump vis-data from 8.0.3 to 8.0.4 in /storm-webapp</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8601">#8601</a>] - Bump spring.version from 7.0.6 to 7.0.7</li>
+ <li>[<a href="https://github.com/apache/storm/pull/8599">#8599</a>] - Bump cypress from 15.13.0 to 15.14.2 in /storm-webapp</li>
+</ul>
+
+<h2>Enhancements</h2>
+<ul>
+ <li>[<a href="https://github.com/apache/storm/pull/8595">#8595</a>] - Update 2.x documentation links to javadoc.io, fix Jekyll rendering, and optimize CI matrix</li>
+</ul>
diff --git a/releases/current b/releases/current
index d45e55c217..c3e2bd4cc1 120000
--- a/releases/current
+++ b/releases/current
@@ -1 +1 @@
-2.8.7
\ No newline at end of file
+2.8.8
\ No newline at end of file
(storm-site) 01/02: Releasing Apache Storm 2.8.8
Posted by ra...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
reiabreu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/storm-site.git
commit ade1ec25f750fed6ed1c1c80229904733736dc28
Author: Rui Abreu <ru...@gmail.com>
AuthorDate: Mon May 18 22:25:08 2026 +0100
Releasing Apache Storm 2.8.8
---
releases/2.8.8/Acking-framework-implementation.md | 39 +
releases/2.8.8/Classpath-handling.md | 29 +
releases/2.8.8/Clojure-DSL.md | 266 +++
releases/2.8.8/ClusterMetrics.md | 278 +++
releases/2.8.8/Command-line-client.md | 358 ++++
releases/2.8.8/Common-patterns.md | 84 +
releases/2.8.8/Concepts.md | 120 ++
releases/2.8.8/Configuration.md | 41 +
releases/2.8.8/Contributing-to-Storm.md | 33 +
releases/2.8.8/Creating-a-new-Storm-project.md | 25 +
releases/2.8.8/DSLs-and-multilang-adapters.md | 11 +
releases/2.8.8/Daemon-Fault-Tolerance.md | 30 +
.../Defining-a-non-jvm-language-dsl-for-storm.md | 45 +
releases/2.8.8/Distributed-RPC.md | 216 +++
releases/2.8.8/Docker-support.md | 135 ++
releases/2.8.8/Eventlogging.md | 125 ++
releases/2.8.8/FAQ.md | 127 ++
releases/2.8.8/Fault-tolerance.md | 28 +
releases/2.8.8/Generic-resources.md | 39 +
releases/2.8.8/Guaranteeing-message-processing.md | 181 ++
releases/2.8.8/Hooks.md | 26 +
releases/2.8.8/IConfigLoader.md | 51 +
releases/2.8.8/Implementation-docs.md | 13 +
releases/2.8.8/Installing-native-dependencies.md | 38 +
releases/2.8.8/Joins.md | 139 ++
releases/2.8.8/Kestrel-and-Storm.md | 202 +++
releases/2.8.8/Lifecycle-of-a-topology.md | 82 +
releases/2.8.8/Local-mode.md | 86 +
releases/2.8.8/LocalityAwareness.md | 84 +
releases/2.8.8/Logs.md | 30 +
releases/2.8.8/Maven.md | 22 +
releases/2.8.8/Message-passing-implementation.md | 30 +
releases/2.8.8/Metrics.md | 313 ++++
releases/2.8.8/Multilang-protocol.md | 337 ++++
releases/2.8.8/NUMA.md | 97 ++
releases/2.8.8/OCI-support.md | 1710 +++++++++++++++++++
releases/2.8.8/Pacemaker.md | 112 ++
releases/2.8.8/Performance.md | 179 ++
releases/2.8.8/Project-ideas.md | 6 +
releases/2.8.8/Rationale.md | 33 +
.../2.8.8/Resource_Aware_Scheduler_overview.md | 513 ++++++
.../Running-topologies-on-a-production-cluster.md | 77 +
releases/2.8.8/SECURITY.md | 752 +++++++++
releases/2.8.8/STORM-UI-REST-API.md | 1616 ++++++++++++++++++
releases/2.8.8/Serialization-(prior-to-0.6.0).md | 50 +
releases/2.8.8/Serialization.md | 68 +
releases/2.8.8/Serializers.md | 4 +
releases/2.8.8/Setting-up-a-Storm-cluster.md | 135 ++
.../2.8.8/Setting-up-a-Storm-project-in-Eclipse.md | 1 +
.../2.8.8/Setting-up-development-environment.md | 33 +
releases/2.8.8/Spout-implementations.md | 10 +
releases/2.8.8/State-checkpointing.md | 289 ++++
releases/2.8.8/Storm-Scheduler.md | 27 +
...language-protocol-(versions-0.7.0-and-below).md | 122 ++
releases/2.8.8/Stream-API.md | 491 ++++++
releases/2.8.8/Structure-of-the-codebase.md | 129 ++
releases/2.8.8/Support-for-non-java-languages.md | 9 +
releases/2.8.8/Transactional-topologies.md | 361 ++++
releases/2.8.8/Trident-API-Overview.md | 642 +++++++
releases/2.8.8/Trident-RAS-API.md | 56 +
releases/2.8.8/Trident-spouts.md | 44 +
releases/2.8.8/Trident-state.md | 331 ++++
releases/2.8.8/Trident-tutorial.md | 256 +++
releases/2.8.8/Troubleshooting.md | 148 ++
releases/2.8.8/Tutorial.md | 293 ++++
...standing-the-parallelism-of-a-Storm-topology.md | 123 ++
.../2.8.8/Using-non-JVM-languages-with-Storm.md | 53 +
releases/2.8.8/Windowing.md | 376 +++++
releases/2.8.8/cgroups_in_storm.md | 181 ++
releases/2.8.8/distcache-blobstore.md | 757 +++++++++
releases/2.8.8/dynamic-log-level-settings.md | 45 +
releases/2.8.8/dynamic-worker-profiling.md | 37 +
releases/2.8.8/favicon.ico | Bin 0 -> 1150 bytes
releases/2.8.8/flux.md | 869 ++++++++++
releases/2.8.8/images/Storm-JMS-Example.png | Bin 0 -> 40432 bytes
releases/2.8.8/images/ack_tree.png | Bin 0 -> 31463 bytes
releases/2.8.8/images/architecture.png | Bin 0 -> 69825 bytes
releases/2.8.8/images/architecture.svg | 1458 ++++++++++++++++
releases/2.8.8/images/batched-stream.png | Bin 0 -> 66336 bytes
releases/2.8.8/images/bolt.png | Bin 0 -> 24796 bytes
releases/2.8.8/images/bolt.svg | 743 ++++++++
releases/2.8.8/images/bullet.gif | Bin 0 -> 82 bytes
.../images/disable-event-logging-topology.png | Bin 0 -> 128737 bytes
releases/2.8.8/images/download.png | Bin 0 -> 16272 bytes
releases/2.8.8/images/drpc-workflow.png | Bin 0 -> 66199 bytes
.../2.8.8/images/dynamic_log_level_settings_1.png | Bin 0 -> 93689 bytes
.../2.8.8/images/dynamic_log_level_settings_2.png | Bin 0 -> 78785 bytes
.../2.8.8/images/dynamic_profiling_debugging_1.png | Bin 0 -> 56876 bytes
.../2.8.8/images/dynamic_profiling_debugging_2.png | Bin 0 -> 99164 bytes
.../2.8.8/images/dynamic_profiling_debugging_3.png | Bin 0 -> 96974 bytes
.../2.8.8/images/dynamic_profiling_debugging_4.png | Bin 0 -> 121994 bytes
.../2.8.8/images/eclipse-project-properties.png | Bin 0 -> 80810 bytes
.../2.8.8/images/enable-event-logging-spout.png | Bin 0 -> 136954 bytes
.../2.8.8/images/enable-event-logging-topology.png | Bin 0 -> 161055 bytes
releases/2.8.8/images/event-logs-view.png | Bin 0 -> 192749 bytes
.../2.8.8/images/example-of-a-running-topology.png | Bin 0 -> 81430 bytes
releases/2.8.8/images/footer-bg.png | Bin 0 -> 138 bytes
releases/2.8.8/images/grouping.png | Bin 0 -> 39701 bytes
releases/2.8.8/images/hdfs_blobstore.png | Bin 0 -> 82180 bytes
releases/2.8.8/images/header-bg.png | Bin 0 -> 470 bytes
releases/2.8.8/images/incubator-logo.png | Bin 0 -> 11651 bytes
.../2.8.8/images/ld-library-path-eclipse-linux.png | Bin 0 -> 114597 bytes
releases/2.8.8/images/loading.gif | Bin 0 -> 12150 bytes
releases/2.8.8/images/local_blobstore.png | Bin 0 -> 81212 bytes
releases/2.8.8/images/logo.png | Bin 0 -> 26889 bytes
releases/2.8.8/images/logos/aeris.jpg | Bin 0 -> 7420 bytes
releases/2.8.8/images/logos/alibaba.jpg | Bin 0 -> 10317 bytes
releases/2.8.8/images/logos/bai.jpg | Bin 0 -> 10026 bytes
releases/2.8.8/images/logos/cerner.jpg | Bin 0 -> 7244 bytes
releases/2.8.8/images/logos/flipboard.jpg | Bin 0 -> 8318 bytes
releases/2.8.8/images/logos/fullcontact.jpg | Bin 0 -> 6172 bytes
releases/2.8.8/images/logos/groupon.jpg | Bin 0 -> 9849 bytes
.../2.8.8/images/logos/health-market-science.jpg | Bin 0 -> 6509 bytes
releases/2.8.8/images/logos/images.png | Bin 0 -> 7339 bytes
releases/2.8.8/images/logos/infochimp.jpg | Bin 0 -> 5290 bytes
releases/2.8.8/images/logos/klout.jpg | Bin 0 -> 7251 bytes
releases/2.8.8/images/logos/loggly.jpg | Bin 0 -> 9258 bytes
releases/2.8.8/images/logos/ooyala.jpg | Bin 0 -> 5675 bytes
releases/2.8.8/images/logos/parc.png | Bin 0 -> 13720 bytes
releases/2.8.8/images/logos/premise.jpg | Bin 0 -> 5391 bytes
releases/2.8.8/images/logos/qiy.jpg | Bin 0 -> 7441 bytes
releases/2.8.8/images/logos/quicklizard.jpg | Bin 0 -> 7382 bytes
releases/2.8.8/images/logos/rocketfuel.jpg | Bin 0 -> 10007 bytes
releases/2.8.8/images/logos/rubicon.jpg | Bin 0 -> 7120 bytes
releases/2.8.8/images/logos/spider.jpg | Bin 0 -> 6265 bytes
releases/2.8.8/images/logos/spotify.jpg | Bin 0 -> 6445 bytes
releases/2.8.8/images/logos/taobao.jpg | Bin 0 -> 16814 bytes
.../2.8.8/images/logos/the-weather-channel.jpg | Bin 0 -> 13295 bytes
releases/2.8.8/images/logos/twitter.jpg | Bin 0 -> 7139 bytes
releases/2.8.8/images/logos/verisign.jpg | Bin 0 -> 5982 bytes
releases/2.8.8/images/logos/webmd.jpg | Bin 0 -> 8226 bytes
releases/2.8.8/images/logos/wego.jpg | Bin 0 -> 6836 bytes
releases/2.8.8/images/logos/yahoo-japan.jpg | Bin 0 -> 10350 bytes
releases/2.8.8/images/logos/yahoo.png | Bin 0 -> 13067 bytes
releases/2.8.8/images/logos/yelp.jpg | Bin 0 -> 7220 bytes
releases/2.8.8/images/mailinglist.png | Bin 0 -> 4245 bytes
releases/2.8.8/images/nimbus_ha_blobstore.png | Bin 0 -> 113991 bytes
.../nimbus_ha_leader_election_and_failover.png | Bin 0 -> 154316 bytes
.../2.8.8/images/nimbus_ha_topology_submission.png | Bin 0 -> 134180 bytes
.../images/ras_new_strategy_network_cdf_random.png | Bin 0 -> 157859 bytes
...trategy_network_metric_cdf_yahoo_topologies.png | Bin 0 -> 152034 bytes
..._strategy_network_metric_improvement_random.png | Bin 0 -> 68976 bytes
.../ras_new_strategy_network_metric_random.png | Bin 0 -> 79004 bytes
...ew_strategy_network_metric_yahoo_topologies.png | Bin 0 -> 75195 bytes
.../images/ras_new_strategy_runtime_random.png | Bin 0 -> 76534 bytes
.../images/ras_new_strategy_runtime_yahoo.png | Bin 0 -> 71727 bytes
...ationships-worker-processes-executors-tasks.png | Bin 0 -> 54804 bytes
...e_aware_scheduler_default_eviction_strategy.png | Bin 0 -> 331891 bytes
...e_aware_scheduler_default_eviction_strategy.svg | 3 +
releases/2.8.8/images/search-a-topology.png | Bin 0 -> 671031 bytes
.../images/search-for-a-single-worker-log.png | Bin 0 -> 736579 bytes
releases/2.8.8/images/security.png | Bin 0 -> 72415 bytes
releases/2.8.8/images/security.svg | 1779 ++++++++++++++++++++
releases/2.8.8/images/spout-vs-state.png | Bin 0 -> 24804 bytes
releases/2.8.8/images/spout.png | Bin 0 -> 22911 bytes
releases/2.8.8/images/spout.svg | 833 +++++++++
releases/2.8.8/images/storm-cluster.png | Bin 0 -> 34604 bytes
releases/2.8.8/images/storm-flow.png | Bin 0 -> 59688 bytes
...m-sql-internal-example-exported-from-drawio.xml | 2 +
.../2.8.8/images/storm-sql-internal-example.png | Bin 0 -> 63265 bytes
...-sql-internal-workflow-exported-from-drawio.xml | 2 +
.../2.8.8/images/storm-sql-internal-workflow.png | Bin 0 -> 42408 bytes
releases/2.8.8/images/storm.svg | 1326 +++++++++++++++
releases/2.8.8/images/storm_header.png | Bin 0 -> 17291 bytes
releases/2.8.8/images/storm_logo_tagline_color.png | Bin 0 -> 33568 bytes
releases/2.8.8/images/storm_ui.png | Bin 0 -> 1054600 bytes
releases/2.8.8/images/supervisor_page.png | Bin 0 -> 133290 bytes
releases/2.8.8/images/top_bg.gif | Bin 0 -> 113 bytes
releases/2.8.8/images/topology-tasks.png | Bin 0 -> 45960 bytes
releases/2.8.8/images/topology.png | Bin 0 -> 23147 bytes
releases/2.8.8/images/topology.svg | 1044 ++++++++++++
releases/2.8.8/images/topology_dark.png | Bin 0 -> 49692 bytes
releases/2.8.8/images/topology_dark.svg | 1101 ++++++++++++
releases/2.8.8/images/transactional-batches.png | Bin 0 -> 21797 bytes
.../2.8.8/images/transactional-commit-flow.png | Bin 0 -> 17179 bytes
releases/2.8.8/images/transactional-design-2.png | Bin 0 -> 13352 bytes
.../2.8.8/images/transactional-spout-structure.png | Bin 0 -> 25067 bytes
releases/2.8.8/images/trident-to-storm1.png | Bin 0 -> 67173 bytes
releases/2.8.8/images/trident-to-storm2.png | Bin 0 -> 68943 bytes
releases/2.8.8/images/tuple-dag.png | Bin 0 -> 18849 bytes
releases/2.8.8/images/tuple_tree.png | Bin 0 -> 58186 bytes
releases/2.8.8/images/ui_topology_viz.png | Bin 0 -> 112831 bytes
releases/2.8.8/index.md | 103 ++
releases/2.8.8/metrics_v2.md | 177 ++
releases/2.8.8/nimbus-ha-design.md | 222 +++
releases/2.8.8/storm-hdfs.md | 582 +++++++
releases/2.8.8/storm-jdbc.md | 288 ++++
releases/2.8.8/storm-jms-example.md | 111 ++
releases/2.8.8/storm-jms-spring.md | 25 +
releases/2.8.8/storm-jms.md | 33 +
releases/2.8.8/storm-kafka-client.md | 353 ++++
releases/2.8.8/storm-metricstore.md | 86 +
releases/2.8.8/storm-redis.md | 273 +++
releases/2.8.8/windows-users-guide.md | 32 +
194 files changed, 25274 insertions(+)
diff --git a/releases/2.8.8/Acking-framework-implementation.md b/releases/2.8.8/Acking-framework-implementation.md
new file mode 100644
index 0000000000..f181e9823f
--- /dev/null
+++ b/releases/2.8.8/Acking-framework-implementation.md
@@ -0,0 +1,39 @@
+---
+title: Acking framework implementation
+layout: documentation
+documentation: true
+---
+
+[Storm's acker]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/daemon/Acker.java) tracks completion of each tupletree with a checksum hash: each time a tuple is sent, its value is XORed into the checksum, and each time a tuple is acked its value is XORed in again. If all tuples have been successfully acked, the checksum will be zero (the odds that the checksum will be zero otherwise are vanishingly small).
+
+You can read a bit more about the [reliability mechanism](Guaranteeing-message-processing.html#what-is-storms-reliability-api) elsewhere on the wiki -- this explains the internal details.
+
+### acker `execute()`
+
+The acker is actually a regular bolt. When a new tupletree is born, the spout sends the XORed edge-ids of each tuple recipient, which the acker records in its `pending` ledger. Every time an executor acks a tuple, the acker receives a partial checksum that is the XOR of the tuple's own edge-id (clearing it from the ledger) and the edge-id of each downstream tuple the executor emitted (thus entering them into the ledger).
+
+This is accomplished as follows.
+
+On a tick tuple, just advance pending tupletree checksums towards death and return. Otherwise, update or create the record for this tupletree:
+
+* on init: initialize with the given checksum value, and record the spout's id for later.
+* on ack: xor the partial checksum into the existing checksum value
+* on fail: just mark it as failed
+
+Next, put the record into the RotatingMap (thus resetting is countdown to expiry) and take action:
+
+* if the total checksum is zero, the tupletree is complete: remove it from the pending collection and notify the spout of success
+* if the tupletree has failed, it is also complete: remove it from the pending collection and notify the spout of failure
+
+Finally, pass on an ack of our own.
+
+### Pending tuples and the `RotatingMap`
+
+The acker stores pending tuples in a [`RotatingMap`]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/utils/RotatingMap.java), a simple device used in several places within Storm to efficiently time-expire a process.
+
+The RotatingMap behaves as a HashMap, and offers the same O(1) access guarantees.
+
+Internally, it holds several HashMaps ('buckets') of its own, each holding a cohort of records that will expire at the same time. Let's call the longest-lived bucket death row, and the most recent the nursery. Whenever a value is `.put()` to the RotatingMap, it is relocated to the nursery -- and removed from any other bucket it might have been in (effectively resetting its death clock).
+
+Whenever its owner calls `.rotate()`, the RotatingMap advances each cohort one step further towards expiration. (Typically, Storm objects call rotate on every receipt of a system tick stream tuple.) If there are any key-value pairs in the former death row bucket, the RotatingMap invokes a callback (given in the constructor) for each key-value pair, letting its owner take appropriate action (eg, failing a tuple.
+
diff --git a/releases/2.8.8/Classpath-handling.md b/releases/2.8.8/Classpath-handling.md
new file mode 100644
index 0000000000..c9e319b966
--- /dev/null
+++ b/releases/2.8.8/Classpath-handling.md
@@ -0,0 +1,29 @@
+---
+title: Classpath Handling
+layout: documentation
+documentation: true
+---
+### Storm is an Application Container
+
+Storm provides an application container environment, a la Apache Tomcat, which creates a potential for classpath conflicts between Storm and your application. The most common way of using Storm involves submitting an "uber JAR" containing your application code with all of its dependencies bundled in, and then Storm distributes this JAR to Worker nodes. Then Storm runs your application within a Storm process called a `Worker` -- thus the JVM's classpath contains the dependencies of your [...]
+
+### Adding Extra Dependencies to Classpath
+
+You no longer *need* to bundle your dependencies into your topology and create an uber JAR, there are now facilities for separately handling your topology's dependencies. Furthermore, there are facilities for adding external dependencies into the Storm daemons.
+
+The `storm.py` launcher script allows you to include dependencies into the launched program's classpath via a few different mechanisms:
+
+1. The `--jar` and `--artifacts` options for the `storm jar` command: allow the inclusion of non-bundled dependencies with your topology; i.e., allowing specification of JARs that were not bundled into the topology uber-jar. This is required when using the `storm sql` command, which constructs a topology automatically without needing you to write code and build a topology JAR.
+2. The `${STORM_DIR}/extlib/` and `${STORM_DIR}/extlib-daemon/` directories can have dependencies added to them for the inclusion of plugins & 3rd-party libraries into the Storm daemons (e.g., Nimbus, UI, Supervisor, etc. -- use `extlib-daemon/`) and other commands launched via the `storm.py` script, e.g., `storm sql` and `storm jar` (use `extlib`). Notably, this means that the Storm Worker process does not include the `extlib-daemon/` directory into its classpath.
+3. The `STORM_EXT_CLASSPATH` and `STORM_EXT_CLASSPATH_DAEMON` environment variables provide a similar functionality as those directories, but allows the user to place their external dependencies in alternative locations.
+ * There is a wrinkle here: because the Supervisor daemon launches the Worker process, if you want `STORM_EXT_CLASSPATH` to impact your Workers, you will need to specify the `STORM_EXT_CLASSPATH` for the Supervisor daemon. That will allow the Supervisor to consult this environment variable as it constructs the classpath of the Worker processes.
+
+#### Which Facility to Choose?
+
+You might have noticed the overlap between the first mechanism and the others. If you consider the `--jar` / `--artifacts` option versus the `extlib/` / `STORM_EXT_CLASSPATH` it is not obvious which one you should choose for using dependencies with your Worker processes. i.e., both mechanisms allow including JARs to be used for running your Worker processes. Here is my understanding of the difference: `--jar` / `--artifacts` will result in the dependencies being used for running the `sto [...]
+
+### Abbreviation of Classpaths and Process Commands
+
+When the `storm.py` script launches a `java` command, it first constructs the classpath from the optional settings mentioned above, as well as including some default locations such as the `${STORM_DIR}/`, `${STORM_DIR}/lib/`, `${STORM_DIR}/extlib/` and `${STORM_DIR}/extlib-daemon/` directories. In past releases, Storm would enumerate all JARs in those directories and then explicitly add all of those JARs into the `-cp` / `--classpath` argument to the launched `java` commands. As such, [...]
+
+After Storm dropped support for Java 5, this classpath expansion was no longer necessary, because Java 6 supports classpath wildcards. Classpath wildcards allow you to specify a directory ending with a `*` element, such as `foo/bar/*`, and the JVM will automatically expand the classpath to include all `.jar` files in the wildcard directory. As of [STORM-2191](https://issues.apache.org/jira/browse/STORM-2191) Storm just uses classpath wildcards instead of explicitly listing all JARs, the [...]
diff --git a/releases/2.8.8/Clojure-DSL.md b/releases/2.8.8/Clojure-DSL.md
new file mode 100644
index 0000000000..83e1c43fde
--- /dev/null
+++ b/releases/2.8.8/Clojure-DSL.md
@@ -0,0 +1,266 @@
+---
+title: Clojure DSL
+layout: documentation
+documentation: true
+---
+Storm offers a Clojure DSL through the storm-clojure package for defining spouts, bolts, and topologies. The Clojure DSL has access to everything the Java API exposes, so if you're a Clojure user you can code Storm topologies without touching Java at all. The Clojure DSL is defined in the source in the [org.apache.storm.clojure]({{page.git-blob-base}}/storm-clojure/src/clj/org/apache/storm/clojure.clj) namespace.
+
+This page outlines all the pieces of the Clojure DSL, including:
+
+1. Defining topologies
+2. `defbolt`
+3. `defspout`
+4. Running topologies in local mode or on a cluster
+5. Testing topologies
+
+### Defining topologies
+
+To define a topology, use the `topology` function. `topology` takes in two arguments: a map of "spout specs" and a map of "bolt specs". Each spout and bolt spec wires the code for the component into the topology by specifying things like inputs and parallelism.
+
+Let's take a look at an example topology definition [from the storm-starter project]({{page.git-blob-base}}/examples/storm-starter/src/clj/org/apache/storm/starter/clj/word_count.clj):
+
+```clojure
+(topology
+ {"1" (spout-spec sentence-spout)
+ "2" (spout-spec (sentence-spout-parameterized
+ ["the cat jumped over the door"
+ "greetings from a faraway land"])
+ :p 2)}
+ {"3" (bolt-spec {"1" :shuffle "2" :shuffle}
+ split-sentence
+ :p 5)
+ "4" (bolt-spec {"3" ["word"]}
+ word-count
+ :p 6)})
+```
+
+The maps of spout and bolt specs are maps from the component id to the corresponding spec. The component ids must be unique across the maps. Just like defining topologies in Java, component ids are used when declaring inputs for bolts in the topology.
+
+#### spout-spec
+
+`spout-spec` takes as arguments the spout implementation (an object that implements [IRichSpout](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/IRichSpout.html)) and optional keyword arguments. The only option that exists currently is the `:p` option, which specifies the parallelism for the spout. If you omit `:p`, the spout will execute as a single task.
+
+#### bolt-spec
+
+`bolt-spec` takes as arguments the input declaration for the bolt, the bolt implementation (an object that implements [IRichBolt](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/IRichBolt.html)), and optional keyword arguments.
+
+The input declaration is a map from stream ids to stream groupings. A stream id can have one of two forms:
+
+1. `[==component id== ==stream id==]`: Subscribes to a specific stream on a component
+2. `==component id==`: Subscribes to the default stream on a component
+
+A stream grouping can be one of the following:
+
+1. `:shuffle`: subscribes with a shuffle grouping
+2. Vector of field names, like `["id" "name"]`: subscribes with a fields grouping on the specified fields
+3. `:global`: subscribes with a global grouping
+4. `:all`: subscribes with an all grouping
+5. `:direct`: subscribes with a direct grouping
+
+See [Concepts](Concepts.html) for more info on stream groupings. Here's an example input declaration showcasing the various ways to declare inputs:
+
+```clojure
+{["2" "1"] :shuffle
+ "3" ["field1" "field2"]
+ ["4" "2"] :global}
+```
+
+This input declaration subscribes to three streams total. It subscribes to stream "1" on component "2" with a shuffle grouping, subscribes to the default stream on component "3" with a fields grouping on the fields "field1" and "field2", and subscribes to stream "2" on component "4" with a global grouping.
+
+Like `spout-spec`, the only current supported keyword argument for `bolt-spec` is `:p` which specifies the parallelism for the bolt.
+
+#### shell-bolt-spec
+
+`shell-bolt-spec` is used for defining bolts that are implemented in a non-JVM language. It takes as arguments the input declaration, the command line program to run, the name of the file implementing the bolt, an output specification, and then the same keyword arguments that `bolt-spec` accepts.
+
+Here's an example `shell-bolt-spec`:
+
+```clojure
+(shell-bolt-spec {"1" :shuffle "2" ["id"]}
+ "python3"
+ "mybolt.py"
+ ["outfield1" "outfield2"]
+ :p 25)
+```
+
+The syntax of output declarations is described in more detail in the `defbolt` section below. See [Using non JVM languages with Storm](Using-non-JVM-languages-with-Storm.html) for more details on how multilang works within Storm.
+
+### defbolt
+
+`defbolt` is used for defining bolts in Clojure. Bolts have the constraint that they must be serializable, and this is why you can't just reify `IRichBolt` to implement a bolt (closures aren't serializable). `defbolt` works around this restriction and provides a nicer syntax for defining bolts than just implementing a Java interface.
+
+At its fullest expressiveness, `defbolt` supports parameterized bolts and maintaining state in a closure around the bolt implementation. It also provides shortcuts for defining bolts that don't need this extra functionality. The signature for `defbolt` looks like the following:
+
+(defbolt _name_ _output-declaration_ *_option-map_ & _impl_)
+
+Omitting the option map is equivalent to having an option map of `{:prepare false}`.
+
+#### Simple bolts
+
+Let's start with the simplest form of `defbolt`. Here's an example bolt that splits a tuple containing a sentence into a tuple for each word:
+
+```clojure
+(defbolt split-sentence ["word"] [tuple collector]
+ (let [words (.split (.getString tuple 0) " ")]
+ (doseq [w words]
+ (emit-bolt! collector [w] :anchor tuple))
+ (ack! collector tuple)
+ ))
+```
+
+Since the option map is omitted, this is a non-prepared bolt. The DSL simply expects an implementation for the `execute` method of `IRichBolt`. The implementation takes two parameters, the tuple and the `OutputCollector`, and is followed by the body of the `execute` function. The DSL automatically type-hints the parameters for you so you don't need to worry about reflection if you use Java interop.
+
+This implementation binds `split-sentence` to an actual `IRichBolt` object that you can use in topologies, like so:
+
+```clojure
+(bolt-spec {"1" :shuffle}
+ split-sentence
+ :p 5)
+```
+
+
+#### Parameterized bolts
+
+Many times you want to parameterize your bolts with other arguments. For example, let's say you wanted to have a bolt that appends a suffix to every input string it receives, and you want that suffix to be set at runtime. You do this with `defbolt` by including a `:params` option in the option map, like so:
+
+```clojure
+(defbolt suffix-appender ["word"] {:params [suffix]}
+ [tuple collector]
+ (emit-bolt! collector [(str (.getString tuple 0) suffix)] :anchor tuple)
+ )
+```
+
+Unlike the previous example, `suffix-appender` will be bound to a function that returns an `IRichBolt` rather than be an `IRichBolt` object directly. This is caused by specifying `:params` in its option map. So to use `suffix-appender` in a topology, you would do something like:
+
+```clojure
+(bolt-spec {"1" :shuffle}
+ (suffix-appender "-suffix")
+ :p 10)
+```
+
+#### Prepared bolts
+
+To do more complex bolts, such as ones that do joins and streaming aggregations, the bolt needs to store state. You can do this by creating a prepared bolt which is specified by including `{:prepare true}` in the option map. Consider, for example, this bolt that implements word counting:
+
+```clojure
+(defbolt word-count ["word" "count"] {:prepare true}
+ [conf context collector]
+ (let [counts (atom {})]
+ (bolt
+ (execute [tuple]
+ (let [word (.getString tuple 0)]
+ (swap! counts (partial merge-with +) {word 1})
+ (emit-bolt! collector [word (@counts word)] :anchor tuple)
+ (ack! collector tuple)
+ )))))
+```
+
+The implementation for a prepared bolt is a function that takes as input the topology config, `TopologyContext`, and `OutputCollector`, and returns an implementation of the `IBolt` interface. This design allows you to have a closure around the implementation of `execute` and `cleanup`.
+
+In this example, the word counts are stored in the closure in a map called `counts`. The `bolt` macro is used to create the `IBolt` implementation. The `bolt` macro is a more concise way to implement the interface than reifying, and it automatically type-hints all of the method parameters. This bolt implements the execute method which updates the count in the map and emits the new word count.
+
+Note that the `execute` method in prepared bolts only takes as input the tuple since the `OutputCollector` is already in the closure of the function (for simple bolts the collector is a second parameter to the `execute` function).
+
+Prepared bolts can be parameterized just like simple bolts.
+
+#### Output declarations
+
+The Clojure DSL has a concise syntax for declaring the outputs of a bolt. The most general way to declare the outputs is as a map from stream id a stream spec. For example:
+
+```clojure
+{"1" ["field1" "field2"]
+ "2" (direct-stream ["f1" "f2" "f3"])
+ "3" ["f1"]}
+```
+
+The stream id is a string, while the stream spec is either a vector of fields or a vector of fields wrapped by `direct-stream`. `direct stream` marks the stream as a direct stream (See [Concepts](Concepts.html) and [Direct groupings]() for more details on direct streams).
+
+If the bolt only has one output stream, you can define the default stream of the bolt by using a vector instead of a map for the output declaration. For example:
+
+```clojure
+["word" "count"]
+```
+This declares the output of the bolt as the fields ["word" "count"] on the default stream id.
+
+#### Emitting, acking, and failing
+
+Rather than use the Java methods on `OutputCollector` directly, the DSL provides a nicer set of functions for using `OutputCollector`: `emit-bolt!`, `emit-direct-bolt!`, `ack!`, and `fail!`.
+
+1. `emit-bolt!`: takes as parameters the `OutputCollector`, the values to emit (a Clojure sequence), and keyword arguments for `:anchor` and `:stream`. `:anchor` can be a single tuple or a list of tuples, and `:stream` is the id of the stream to emit to. Omitting the keyword arguments emits an unanchored tuple to the default stream.
+2. `emit-direct-bolt!`: takes as parameters the `OutputCollector`, the task id to send the tuple to, the values to emit, and keyword arguments for `:anchor` and `:stream`. This function can only emit to streams declared as direct streams.
+2. `ack!`: takes as parameters the `OutputCollector` and the tuple to ack.
+3. `fail!`: takes as parameters the `OutputCollector` and the tuple to fail.
+
+See [Guaranteeing message processing](Guaranteeing-message-processing.html) for more info on acking and anchoring.
+
+### defspout
+
+`defspout` is used for defining spouts in Clojure. Like bolts, spouts must be serializable so you can't just reify `IRichSpout` to do spout implementations in Clojure. `defspout` works around this restriction and provides a nicer syntax for defining spouts than just implementing a Java interface.
+
+The signature for `defspout` looks like the following:
+
+(defspout _name_ _output-declaration_ *_option-map_ & _impl_)
+
+If you leave out the option map, it defaults to {:prepare true}. The output declaration for `defspout` has the same syntax as `defbolt`.
+
+Here's an example `defspout` implementation from [storm-starter]({{page.git-blob-base}}/examples/storm-starter/src/clj/org/apache/storm/starter/clj/word_count.clj):
+
+```clojure
+(defspout sentence-spout ["sentence"]
+ [conf context collector]
+ (let [sentences ["a little brown dog"
+ "the man petted the dog"
+ "four score and seven years ago"
+ "an apple a day keeps the doctor away"]]
+ (spout
+ (nextTuple []
+ (Thread/sleep 100)
+ (emit-spout! collector [(rand-nth sentences)])
+ )
+ (ack [id]
+ ;; You only need to define this method for reliable spouts
+ ;; (such as one that reads off of a queue like Kestrel)
+ ;; This is an unreliable spout, so it does nothing here
+ ))))
+```
+
+The implementation takes in as input the topology config, `TopologyContext`, and `SpoutOutputCollector`. The implementation returns an `ISpout` object. Here, the `nextTuple` function emits a random sentence from `sentences`.
+
+This spout isn't reliable, so the `ack` and `fail` methods will never be called. A reliable spout will add a message id when emitting tuples, and then `ack` or `fail` will be called when the tuple is completed or failed respectively. See [Guaranteeing message processing](Guaranteeing-message-processing.html) for more info on how reliability works within Storm.
+
+`emit-spout!` takes in as parameters the `SpoutOutputCollector` and the new tuple to be emitted, and accepts as keyword arguments `:stream` and `:id`. `:stream` specifies the stream to emit to, and `:id` specifies a message id for the tuple (used in the `ack` and `fail` callbacks). Omitting these arguments emits an unanchored tuple to the default output stream.
+
+There is also a `emit-direct-spout!` function that emits a tuple to a direct stream and takes an additional argument as the second parameter of the task id to send the tuple to.
+
+Spouts can be parameterized just like bolts, in which case the symbol is bound to a function returning `IRichSpout` instead of the `IRichSpout` itself. You can also declare an unprepared spout which only defines the `nextTuple` method. Here is an example of an unprepared spout that emits random sentences parameterized at runtime:
+
+```clojure
+(defspout sentence-spout-parameterized ["word"] {:params [sentences] :prepare false}
+ [collector]
+ (Thread/sleep 500)
+ (emit-spout! collector [(rand-nth sentences)]))
+```
+
+The following example illustrates how to use this spout in a `spout-spec`:
+
+```clojure
+(spout-spec (sentence-spout-parameterized
+ ["the cat jumped over the door"
+ "greetings from a faraway land"])
+ :p 2)
+```
+
+### Running topologies in local mode or on a cluster
+
+That's all there is to the Clojure DSL. To submit topologies in remote mode or local mode, just use the `StormSubmitter` class just like you would from Java.
+
+To create topology configs, it's easiest to use the `org.apache.storm.config` namespace which defines constants for all of the possible configs. The constants are the same as the static constants in the `Config` class, except with dashes instead of underscores. For example, here's a topology config that sets the number of workers to 15 and configures the topology in debug mode:
+
+```clojure
+{TOPOLOGY-DEBUG true
+ TOPOLOGY-WORKERS 15}
+```
+
+### Testing topologies
+
+[This blog post](http://www.pixelmachine.org/2011/12/17/Testing-Storm-Topologies.html) and its [follow-up](http://www.pixelmachine.org/2011/12/21/Testing-Storm-Topologies-Part-2.html) give a good overview of Storm's powerful built-in facilities for testing topologies in Clojure.
diff --git a/releases/2.8.8/ClusterMetrics.md b/releases/2.8.8/ClusterMetrics.md
new file mode 100644
index 0000000000..6b18099224
--- /dev/null
+++ b/releases/2.8.8/ClusterMetrics.md
@@ -0,0 +1,278 @@
+---
+title: Cluster Metrics
+layout: documentation
+documentation: true
+---
+
+# Cluster Metrics
+
+There are lots of metrics to help you monitor a running cluster. Many of these metrics are still a work in progress and so is the metrics system itself so any of them may change, even between minor version releases. We will try to keep them as stable as possible, but they should all be considered somewhat unstable. Some of the metrics may also be for experimental features, or features that are not complete yet, so please read the description of the metric before using it for monitoring [...]
+
+Also be aware that depending on the metrics system you use, the names are likely to be translated into a different format that is compatible with the system. Typically this means that the ':' separating character will be replaced with a '.' character.
+
+Most metrics should have the units that they are reported as a part of the description. For Timers often this is configured by the reporter that is uploading them to your system. Pay attention because even if the metric name has a time unit in it, it may be false.
+
+Also, most metrics, except for gauges and counters, are a collection of numbers, and not a single value. Often these result in multiple metrics being uploaded to a reporting system, such as percentiles for a histogram, or rates for a meter. It is dependent on the configured metrics reporter how this happens, or how the name here corresponds to the metric in your reporting system.
+
+## Cluster Metrics (From Nimbus)
+
+These are metrics that come from the active nimbus instance and report the state of the cluster as a whole, as seen by nimbus.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| cluster:num-nimbus-leaders | gauge | Number of nimbuses marked as a leader. This should really only ever be 1 in a healthy cluster, or 0 for a short period of time while a failover happens. |
+| cluster:num-nimbuses | gauge | Number of nimbuses, leader or standby. |
+| cluster:num-supervisors | gauge | Number of supervisors. |
+| cluster:num-topologies | gauge | Number of topologies. |
+| cluster:num-total-used-workers | gauge | Number of used workers/slots. |
+| cluster:num-total-workers | gauge | Number of workers/slots. |
+| cluster:total-fragmented-cpu-non-negative | gauge | Total fragmented CPU (% of core). This is CPU that the system thinks it cannot use because other resources on the node are used up. |
+| cluster:total-fragmented-memory-non-negative | gauge | Total fragmented memory (MB). This is the memory that the system thinks it cannot use because other resources on the node are used up. |
+| topologies:assigned-cpu | histogram | CPU scheduled per topology (% of a core) |
+| topologies:assigned-mem-off-heap | histogram | Off heap memory scheduled per topology (MB) |
+| topologies:assigned-mem-on-heap | histogram | On heap memory scheduled per topology (MB) |
+| topologies:num-executors | histogram | Number of executors per topology. |
+| topologies:num-tasks | histogram | Number of tasks per topology. |
+| topologies:num-workers | histogram | Number of workers per topology. |
+| topologies:replication-count | histogram | Replication count per topology. |
+| topologies:requested-cpu | histogram | CPU requested per topology (% of a core). |
+| topologies:requested-mem-off-heap | histogram | Off heap memory requested per topology (MB). |
+| topologies:requested-mem-on-heap | histogram | On heap memory requested per topology (MB). |
+| topologies:uptime-secs | histogram | Uptime per topology (seconds). |
+| nimbus:available-cpu-non-negative | gauge | Available cpu on the cluster (% of a core). |
+| nimbus:total-cpu | gauge | total CPU on the cluster (% of a core) |
+| nimbus:total-memory | gauge | total memory on the cluster MB |
+| supervisors:fragmented-cpu | histogram | fragmented cpu per supervisor (% of a core) |
+| supervisors:fragmented-mem | histogram | fragmented memory per supervisor (MB) |
+| supervisors:num-used-workers | histogram | workers used per supervisor |
+| supervisors:num-workers | histogram | number of workers per supervisor |
+| supervisors:uptime-secs | histogram | uptime of supervisors |
+| supervisors:used-cpu | histogram | cpu used per supervisor (% of a core) |
+| supervisors:used-mem | histogram | memory used per supervisor MB |
+
+## Nimbus Metrics
+
+These are metrics that are specific to a nimbus instance. In many instances, only the active nimbus will be reporting these metrics, but they could come from standby nimbus instances as well.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| nimbus:files-upload-duration-ms | timer | Time it takes to upload a file from start to finish (Not Blobs, but this may change) |
+| nimbus:longest-scheduling-time-ms | gauge | Longest time ever taken so far to schedule. This includes the current scheduling run, which is intended to detect if scheduling is stuck for some reason. |
+| nimbus:mkAssignments-Errors | meter | tracks exceptions from mkAssignments |
+| nimbus:num-activate-calls | meter | calls to the activate thrift method. |
+| nimbus:num-added-executors-per-scheduling | histogram | number of executors added after a scheduling run. |
+| nimbus:num-added-slots-per-scheduling | histogram | number of slots added after a scheduling run. |
+| nimbus:num-beginFileUpload-calls | meter | calls to the beginFileUpload thrift method. |
+| nimbus:num-blacklisted-supervisor | gauge | Number of supervisors currently marked as blacklisted because they appear to be somewhat unstable. |
+| nimbus:num-deactivate-calls | meter | calls to deactivate thrift method. |
+| nimbus:num-debug-calls | meter | calls to debug thrift method.|
+| nimbus:num-downloadChunk-calls | meter | calls to downloadChunk thrift method. |
+| nimbus:num-finishFileUpload-calls | meter | calls to finishFileUpload thrift method.|
+| nimbus:num-gained-leadership | meter | number of times this nimbus gained leadership. |
+| nimbus:num-getClusterInfo-calls | meter | calls to getClusterInfo thrift method. |
+| nimbus:num-getComponentPageInfo-calls | meter | calls to getComponentPageInfo thrift method. |
+| nimbus:num-getComponentPendingProfileActions-calls | meter | calls to getComponentPendingProfileActions thrift method. |
+| nimbus:num-getLeader-calls | meter | calls to getLeader thrift method. |
+| nimbus:num-getLogConfig-calls | meter | calls to getLogConfig thrift method. |
+| nimbus:num-getNimbusConf-calls | meter | calls to getNimbusConf thrift method. |
+| nimbus:num-getOwnerResourceSummaries-calls | meter | calls to getOwnerResourceSummaries thrift method. |
+| nimbus:num-getSupervisorPageInfo-calls | meter | calls to getSupervisorPageInfo thrift method. |
+| nimbus:num-getTopology-calls | meter | calls to getTopology thrift method. |
+| nimbus:num-getTopologyConf-calls | meter | calls to getTopologyConf thrift method. |
+| nimbus:num-getTopologyInfo-calls | meter | calls to getTopologyInfo thrift method. |
+| nimbus:num-getTopologyInfoWithOpts-calls | meter | calls to getTopologyInfoWithOpts thrift method includes calls to getTopologyInfo. |
+| nimbus:num-getTopologyPageInfo-calls | meter | calls to getTopologyPageInfo thrift method. |
+| nimbus:num-getUserTopology-calls | meter | calls to getUserTopology thrift method. |
+| nimbus:num-isTopologyNameAllowed-calls | meter | calls to isTopologyNameAllowed thrift method. |
+| nimbus:num-killTopology-calls | meter | calls to killTopology thrift method. |
+| nimbus:num-killTopologyWithOpts-calls | meter | calls to killTopologyWithOpts thrift method includes calls to killTopology. |
+| nimbus:num-launched | meter | number of times a nimbus was launched |
+| nimbus:num-lost-leadership | meter | number of times this nimbus lost leadership |
+| nimbus:num-negative-resource-events | meter | Any time a resource goes negative (either CPU or Memory). This metric is not ideal as it is measured in a data structure that is used for internal calculations that may go negative and not actually represent over scheduling of a resource. |
+| nimbus:num-net-executors-increase-per-scheduling | histogram | added executors minus removed executors after a scheduling run |
+| nimbus:num-net-slots-increase-per-scheduling | histogram | added slots minus removed slots after a scheduling run |
+| nimbus:num-rebalance-calls | meter | calls to rebalance thrift method. |
+| nimbus:num-removed-executors-per-scheduling | histogram | number of executors removed after a scheduling run |
+| nimbus:num-scheduling-timeouts | meter | number of timeouts during scheduling |
+| nimbus:num-removed-slots-per-scheduling | histogram | number of slots removed after a scheduling run |
+| nimbus:num-setLogConfig-calls | meter | calls to setLogConfig thrift method. |
+| nimbus:num-setWorkerProfiler-calls | meter | calls to setWorkerProfiler thrift method. |
+| nimbus:num-shutdown-calls | meter | times nimbus is shut down (this may not actually be reported as nimbus is in the middle of shutting down) |
+| nimbus:num-submitTopology-calls | meter | calls to submitTopology thrift method. |
+| nimbus:num-submitTopologyWithOpts-calls | meter | calls to submitTopologyWithOpts thrift method includes calls to submitTopology. |
+| nimbus:num-uploadChunk-calls | meter | calls to uploadChunk thrift method. |
+| nimbus:num-uploadNewCredentials-calls | meter | calls to uploadNewCredentials thrift method. |
+| nimbus:process-worker-metric-calls | meter | calls to processWorkerMetrics thrift method. |
+| nimbus:scheduler-internal-errors | meter | tracks internal scheduling errors |
+| nimbus:topology-scheduling-duration-ms | timer | time it takes to do a scheduling run. |
+| nimbus:total-available-memory-non-negative | gauge | available memory on the cluster MB |
+| nimbuses:uptime-secs | histogram | uptime of nimbuses |
+| MetricsCleaner:purgeTimestamp | gauge | last time metrics were purged (Unfinished Feature) |
+| RocksDB:metric-failures | meter | generally any failure that happens in the rocksdb metrics store. (Unfinished Feature) |
+
+
+## DRPC Metrics
+
+Metrics related to DRPC servers.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| drpc:HTTP-request-response-duration | timer | how long it takes to execute an http drpc request |
+| drpc:num-execute-calls | meter | calls to execute a DRPC request |
+| drpc:num-execute-http-requests | meter | http requests to the DRPC server |
+| drpc:num-failRequest-calls | meter | calls to failRequest |
+| drpc:num-fetchRequest-calls | meter | calls to fetchRequest |
+| drpc:num-result-calls | meter | calls to returnResult |
+| drpc:num-server-timedout-requests | meter | times a DRPC request timed out without a response |
+| drpc:num-shutdown-calls | meter | number of times shutdown is called on the drpc server |
+
+## Logviewer Metrics
+
+Metrics related to the logviewer process. This process currently also handles cleaning up worker logs when they get too large or too old.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| logviewer:cleanup-routine-duration-ms | timer | how long it takes to run the log cleanup routine |
+| logviewer:deep-search-request-duration-ms | timer | how long it takes for /deepSearch/{topoId} |
+| logviewer:disk-space-freed-in-bytes | histogram | number of bytes cleaned up each time through the cleanup routine. |
+| logviewer:download-file-size-rounded-MB | histogram | size in MB of files being downloaded |
+| logviewer:num-daemonlog-page-http-requests | meter | calls to /daemonlog |
+| logviewer:num-deep-search-no-result | meter | number of deep search requests that did not return any results |
+| logviewer:num-deep-search-requests-with-archived | meter | calls to /deepSearch/{topoId} with ?search-archived=true |
+| logviewer:num-deep-search-requests-without-archived | meter | calls to /deepSearch/{topoId} with ?search-archived=false |
+| logviewer:num-download-daemon-log-exceptions | meter | num errors in calls to /daemondownload |
+| logviewer:num-download-dump-exceptions | meter | num errors in calls to /dumps/{topo-id}/{host-port}/{filename} |
+| logviewer:num-download-log-daemon-file-http-requests | meter | calls to /daemondownload |
+| logviewer:num-download-log-exceptions | meter | num errors in calls to /download |
+| logviewer:num-download-log-file-http-requests | meter | calls to /download |
+| logviewer:num-file-download-exceptions | meter | errors while trying to download files. |
+| logviewer:num-file-download-exceptions | meter | number of exceptions trying to download a log file |
+| logviewer:num-file-open-exceptions | meter | errors trying to open a file (when deleting logs) |
+| logviewer:num-file-open-exceptions | meter | number of exceptions trying to open a log file for serving |
+| logviewer:num-file-read-exceptions | meter | number of exceptions trying to read from a log file for serving |
+| logviewer:num-file-removal-exceptions | meter | number of exceptions trying to cleanup files. |
+| logviewer:num-files-cleaned-up | histogram | number of files cleaned up each time through the cleanup routine. |
+| logviewer:num-files-scanned-per-deep-search | histogram | number of files scanned per deep search |
+| logviewer:num-list-dump-files-exceptions | meter | num errors in calls to /dumps/{topo-id}/{host-port} |
+| logviewer:num-list-logs-http-request | meter | calls to /listLogs |
+| logviewer:num-log-page-http-requests | meter | calls to /log |
+| logviewer:num-other-cleanup-exceptions | meter | number of exception in the cleanup loop, not directly deleting files. |
+| logviewer:num-page-read | meter | number of pages (parts of a log file) that are served up |
+| logviewer:num-read-daemon-log-exceptions | meter | num errors in calls to /daemonlog |
+| logviewer:num-read-log-exceptions | meter | num errors in calls to /log |
+| logviewer:num-search-exceptions | meter | num errors in calls to /search |
+| logviewer:num-search-log-exceptions | meter | num errors in calls to /listLogs |
+| logviewer:num-search-logs-requests | meter | calls to /search |
+| logviewer:num-search-request-no-result | meter | number of regular search results that were empty |
+| logviewer:num-set-permission-exceptions | meter | num errors running set permissions to open up files for reading. |
+| logviewer:num-shutdown-calls | meter | number of times shutdown was called on the logviewer |
+| logviewer:search-requests-duration-ms | timer | how long it takes for /search |
+| logviewer:worker-log-dir-size | gauge | size in bytes of the worker logs directory. |
+
+## Supervisor Metrics
+
+Metrics associated with the supervisor, which launches the workers for a topology. The supervisor also has a state machine for each slot. Some of the metrics are associated with that state machine and can be confusing if you do not understand the state machine.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| supervisor:blob-cache-update-duration | timer | how long it takes to update all of the blobs in the cache (frequently just check if they have changed, but may also include downloading them.) |
+| supervisor:blob-fetching-rate-MB/s | histogram | Download rate of a blob in MB/sec. Blobs are downloaded rarely so it is very bursty. |
+| supervisor:blob-localization-duration | timer | Approximately how long it takes to get the blob we want after it is requested. |
+| supervisor:current-reserved-memory-mb | gauge | total amount of memory reserved for workers on the supervisor (MB) |
+| supervisor:current-used-memory-mb | gauge | memory currently used as measured by the supervisor (this typically requires cgroups) (MB) |
+| supervisor:health-check-timeouts | meter | tracks timeouts executing health check scripts |
+| supervisor:local-resource-file-not-found-when-releasing-slot | meter | number of times file-not-found exception happens when reading local blobs upon releasing slots |
+| supervisor:num-blob-update-version-changed | meter | number of times a version of a blob changes. |
+| supervisor:num-cleanup-exceptions | meter | exceptions thrown during container cleanup. |
+| supervisor:num-force-kill-exceptions | meter | exceptions thrown during force kill. |
+| supervisor:num-kill-exceptions | meter | exceptions thrown during kill. |
+| supervisor:num-kill-worker-errors | meter | errors killing workers. |
+| supervisor:num-launched | meter | number of times the supervisor is launched. |
+| supervisor:num-shell-exceptions | meter | number of exceptions calling shell commands. |
+| supervisor:num-slots-used-gauge | gauge | number of slots used on the supervisor. |
+| supervisor:num-worker-start-timed-out | meter | number of times worker start timed out. |
+| supervisor:num-worker-transitions-into-empty | meter | number of transitions into empty state. |
+| supervisor:num-worker-transitions-into-kill | meter | number of transitions into kill state. |
+| supervisor:num-worker-transitions-into-kill-and-relaunch | meter | number of transitions into kill-and-relaunch state |
+| supervisor:num-worker-transitions-into-kill-blob-update | meter | number of transitions into kill-blob-update state |
+| supervisor:num-worker-transitions-into-running | meter | number of transitions into running state |
+| supervisor:num-worker-transitions-into-waiting-for-blob-localization | meter | number of transitions into waiting-for-blob-localization state |
+| supervisor:num-worker-transitions-into-waiting-for-blob-update | meter | number of transitions into waiting-for-blob-update state |
+| supervisor:num-worker-transitions-into-waiting-for-worker-start | meter | number of transitions into waiting-for-worker-start state |
+| supervisor:num-workers-force-kill | meter | number of times a worker was force killed. This may mean that the worker did not exit cleanly/quickly. |
+| supervisor:num-workers-killed-assignment-changed | meter | workers killed because the assignment changed. |
+| supervisor:num-workers-killed-blob-changed | meter | workers killed because the blob changed and they needed to be relaunched. |
+| supervisor:num-workers-killed-hb-null | meter | workers killed because there was no hb at all from the worker. This would typically only happen when a worker is launched for the first time. |
+| supervisor:num-workers-killed-hb-timeout | meter | workers killed because the hb from the worker was too old. This often happens because of GC issues in the worker that prevents it from sending a heartbeat, but could also mean the worker process exited, and the supervisor is not the parent of the process to know that it exited. |
+| supervisor:num-workers-killed-memory-violation | meter | workers killed because the worker was using too much memory. If the supervisor can monitor the memory usage of the worker (typically through cgroups) and the worker goes over the limit it may be shot. |
+| supervisor:num-workers-killed-process-exit | meter | workers killed because the process exited and the supervisor was the parent process |
+| supervisor:num-workers-launched | meter | number of workers launched |
+| supervisor:single-blob-localization-duration | timer | how long it takes for a blob to be updated (downloaded, unzipped, inform slots, and make the move) |
+| supervisor:time-worker-spent-in-state-empty-ms | timer | time spent in empty state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-kill-and-relaunch-ms | timer | time spent in kill-and-relaunch state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-kill-blob-update-ms | timer | time spent in kill-blob-update state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-kill-ms | timer | time spent in kill state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-running-ms | timer | time spent in running state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-waiting-for-blob-localization-ms | timer | time spent in waiting-for-blob-localization state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-waiting-for-blob-update-ms | timer | time spent in waiting-for-blob-update state as it transitions out. Not necessarily in ms. |
+| supervisor:time-worker-spent-in-state-waiting-for-worker-start-ms | timer | time spent in waiting-for-worker-start state as it transitions out. Not necessarily in ms. |
+| supervisor:update-blob-exceptions | meter | number of exceptions updating blobs. |
+| supervisor:worker-launch-duration | timer | Time taken for a worker to launch. |
+| supervisor:worker-per-call-clean-up-duration-ns | meter | how long it takes to cleanup a worker (ns). |
+| supervisor:worker-shutdown-duration-ns | meter | how long it takes to shutdown a worker (ns). |
+| supervisor:workerTokenAuthorizer-get-password-failures | meter | Failures getting password for user in WorkerTokenAuthorizer |
+
+
+## UI Metrics
+
+Metrics associated with a single UI daemon.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| ui:num-activate-topology-http-requests | meter | calls to /topology/{id}/activate |
+| ui:num-all-topologies-summary-http-requests | meter | calls to /topology/summary |
+| ui:num-build-visualization-http-requests | meter | calls to /topology/{id}/visualization |
+| ui:num-cluster-configuration-http-requests | meter | calls to /cluster/configuration |
+| ui:num-cluster-summary-http-requests | meter | calls to /cluster/summary |
+| ui:num-component-op-response-http-requests | meter | calls to /topology/{id}/component/{component}/debug/{action}/{spct} |
+| ui:num-component-page-http-requests | meter | calls to /topology/{id}/component/{component} |
+| ui:num-deactivate-topology-http-requests | meter | calls to topology/{id}/deactivate |
+| ui:num-debug-topology-http-requests | meter | calls to /topology/{id}/debug/{action}/{spct} |
+| ui:num-get-owner-resource-summaries-http-request | meter | calls to /owner-resources or /owner-resources/{id} |
+| ui:num-log-config-http-requests | meter | calls to /topology/{id}/logconfig |
+| ui:num-main-page-http-requests | meter | number of requests to /index.html |
+| ui:num-mk-visualization-data-http-requests | meter | calls to /topology/{id}/visualization-init |
+| ui:num-nimbus-summary-http-requests | meter | calls to /nimbus/summary |
+| ui:num-supervisor-http-requests | meter | calls to /supervisor |
+| ui:num-supervisor-summary-http-requests | meter | calls to /supervisor/summary |
+| ui:num-topology-lag-http-requests | meter | calls to /topology/{id}/lag |
+| ui:num-topology-metric-http-requests | meter | calls to /topology/{id}/metrics |
+| ui:num-topology-op-response-http-requests | meter | calls to /topology/{id}/logconfig or /topology/{id}/rebalance/{wait-time} or /topology/{id}/kill/{wait-time} |
+| ui:num-topology-page-http-requests | meter | calls to /topology/{id} |
+| num-web-requests | meter | nominally the total number of web requests being made. |
+
+## Pacemaker Metrics (Deprecated)
+
+The pacemaker process is deprecated and only still exists for backward compatibility.
+
+| Metric Name | Type | Description |
+|-------------|------|-------------|
+| pacemaker:get-pulse=count | meter | number of times getPulse was called. yes the = is in the name, but typically this is mapped to a '-' by the metrics reporters. |
+| pacemaker:heartbeat-size | histogram | size in bytes of heartbeats |
+| pacemaker:send-pulse-count | meter | number of times sendPulse was called |
+| pacemaker:size-total-keys | gauge | total number of keys in this pacemaker instance |
+| pacemaker:total-receive-size | meter | total size in bytes of heartbeats received |
+| pacemaker:total-sent-size | meter | total size in bytes of heartbeats read |
+
+
+## Metric Reporters
+
+For metrics to be reported, configure reporters using `storm.daemon.metrics.reporter.plugins`. The following metric reporters are supported:
+ * Console Reporter (`org.apache.storm.daemon.metrics.reporters.ConsolePreparableReporter`):
+ Reports metrics to `System.out`.
+ * CSV Reporter (`org.apache.storm.daemon.metrics.reporters.CsvPreparableReporter`):
+ Reports metrics to a CSV file.
+ * JMX Reporter (`org.apache.storm.daemon.metrics.reporters.JmxPreparableReporter`):
+ Exposes metrics via JMX.
+
+Custom reporter can be created by implementing `org.apache.storm.daemon.metrics.reporters.PreparableReporter` interface.
diff --git a/releases/2.8.8/Command-line-client.md b/releases/2.8.8/Command-line-client.md
new file mode 100644
index 0000000000..7f0b5e1054
--- /dev/null
+++ b/releases/2.8.8/Command-line-client.md
@@ -0,0 +1,358 @@
+---
+title: Command Line Client
+layout: documentation
+documentation: true
+---
+This page describes all the commands that are possible with the "storm" command line client. To learn how to set up your "storm" client to talk to a remote cluster, follow the instructions in [Setting up development environment](Setting-up-development-environment.html). See [Classpath handling](Classpath-handling.html) for details on using external libraries in these commands.
+
+These commands are:
+
+1. jar
+1. local
+1. sql
+1. kill
+1. activate
+1. deactivate
+1. rebalance
+1. repl
+1. classpath
+1. server_classpath
+1. localconfvalue
+1. remoteconfvalue
+1. nimbus
+1. supervisor
+1. ui
+1. drpc
+1. drpc-client
+1. blobstore
+1. dev-zookeeper
+1. get-errors
+1. heartbeats
+1. kill_workers
+1. list
+1. logviewer
+1. monitor
+1. node-health-check
+1. pacemaker
+1. set_log_level
+1. shell
+1. upload-credentials
+1. version
+1. admin
+1. help
+
+### jar
+
+Syntax: `storm jar topology-jar-path class ...`
+
+Runs the main method of `class` with the specified arguments. The storm jars and configs in `~/.storm` are put on the classpath. The process is configured so that [StormSubmitter](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/StormSubmitter.html) will upload the jar at `topology-jar-path` when the topology is submitted.
+
+When you want to ship other jars which is not included to application jar, you can pass them to `--jars` option with comma-separated string.
+For example, --jars "your-local-jar.jar,your-local-jar2.jar" will load your-local-jar.jar and your-local-jar2.jar.
+And when you want to ship maven artifacts and its transitive dependencies, you can pass them to `--artifacts` with comma-separated string. You can also exclude some dependencies like what you're doing in maven pom. Please add exclusion artifacts with '^' separated string after the artifact. For example, `--artifacts "redis.clients:jedis:2.9.0,org.apache.kafka:kafka_2.10:0.8.2.2^org.slf4j:slf4j-log4j12"` will load jedis and kafka artifact and all of transitive dependencies but exclude slf [...]
+
+When you need to pull the artifacts from other than Maven Central, you can pass remote repositories to --artifactRepositories option with comma-separated string. Repository format is "<name>^<url>". '^' is taken as separator because URL allows various characters. For example, --artifactRepositories "jboss-repository^http://repository.jboss.com/maven2,HDPRepo^http://repo.hortonworks.com/content/groups/public/" will add JBoss and HDP repositories for dependency resolver.
+
+Complete example of both options is here: `./bin/storm jar example/storm-starter/storm-starter-topologies-*.jar org.apache.storm.starter.RollingTopWords blobstore-remote2 remote --jars "./external/storm-redis/storm-redis-1.1.0.jar,./external/storm-kafka-client/storm-kafka-client-1.1.0.jar" --artifacts "redis.clients:jedis:2.9.0,org.apache.kafka:kafka-clients:1.0.0^org.slf4j:slf4j-api" --artifactRepositories "jboss-repository^http://repository.jboss.com/maven2,HDPRepo^http://repo.hortonwo [...]
+
+When you pass jars and/or artifacts options, StormSubmitter will upload them when the topology is submitted, and they will be included to classpath of both the process which runs the class, and also workers for that topology.
+
+### local
+
+Syntax: `storm local topology-jar-path class ...`
+
+The local command acts just like `storm jar` except instead of submitting a topology to a cluster it will run the cluster in local mode. This means an embedded version of the storm daemons will be run within the same process as your topology for 30 seconds before it shuts down automatically. As such the classpath of your topology will be extended to include everything needed to run those daemons.
+
+### sql
+
+Syntax: `storm sql sql-file topology-name`
+
+Compiles the SQL statements into a Trident topology and submits it to Storm.
+
+`--jars` and `--artifacts`, and `--artifactRepositories` options available for jar are also applied to sql command. Please refer "help jar" to see how to use --jars and --artifacts, and --artifactRepositories options. You normally want to pass these options since you need to set data source to your sql which is an external storage in many cases.
+
+### kill
+
+Syntax: `storm kill topology-name [-w wait-time-secs]`
+
+Kills the topology with the name `topology-name`. Storm will first deactivate the topology's spouts for the duration of the topology's message timeout to allow all messages currently being processed to finish processing. Storm will then shutdown the workers and clean up their state. You can override the length of time Storm waits between deactivation and shutdown with the -w flag.
+
+### activate
+
+Syntax: `storm activate topology-name`
+
+Activates the specified topology's spouts.
+
+### deactivate
+
+Syntax: `storm deactivate topology-name`
+
+Deactivates the specified topology's spouts.
+
+### rebalance
+
+Syntax: `storm rebalance topology-name [-w wait-time-secs] [-n new-num-workers] [-e component=parallelism]*`
+
+Sometimes you may wish to spread out where the workers for a topology are running. For example, let's say you have a 10 node cluster running 4 workers per node, and then let's say you add another 10 nodes to the cluster. You may wish to have Storm spread out the workers for the running topology so that each node runs 2 workers. One way to do this is to kill the topology and resubmit it, but Storm provides a "rebalance" command that provides an easier way to do this.
+
+Rebalance will first deactivate the topology for the duration of the message timeout (overridable with the -w flag) and then redistribute the workers evenly around the cluster. The topology will then return to its previous state of activation (so a deactivated topology will still be deactivated and an activated topology will go back to being activated).
+
+The rebalance command can also be used to change the parallelism of a running topology. Use the -n and -e switches to change the number of workers or number of executors of a component respectively.
+
+### repl
+
+*DEPRECATED: This subcommand may be removed in a future release.*
+
+Syntax: `storm repl`
+
+Opens up a Clojure REPL with the storm jars and configuration on the classpath. Useful for debugging.
+
+### classpath
+
+Syntax: `storm classpath`
+
+Prints the classpath used by the storm client when running commands.
+
+### server_classpath
+
+Syntax: `storm server_classpath`
+
+Prints the classpath used by the storm daemons.
+
+### localconfvalue
+
+Syntax: `storm localconfvalue conf-name`
+
+Prints out the value for `conf-name` in the local Storm configs. The local Storm configs are the ones in `~/.storm/storm.yaml` merged in with the configs in `defaults.yaml`.
+
+### remoteconfvalue
+
+Syntax: `storm remoteconfvalue conf-name`
+
+Prints out the value for `conf-name` in the cluster's Storm configs. The cluster's Storm configs are the ones in `$STORM-PATH/conf/storm.yaml` merged in with the configs in `defaults.yaml`. This command must be run on a cluster machine.
+
+### nimbus
+
+Syntax: `storm nimbus`
+
+Launches the nimbus daemon. This command should be run under supervision with a tool like [daemontools](http://cr.yp.to/daemontools.html) or [monit](http://mmonit.com/monit/). See [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html) for more information.
+
+### supervisor
+
+Syntax: `storm supervisor`
+
+Launches the supervisor daemon. This command should be run under supervision with a tool like [daemontools](http://cr.yp.to/daemontools.html) or [monit](http://mmonit.com/monit/). See [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html) for more information.
+
+### ui
+
+Syntax: `storm ui`
+
+Launches the UI daemon. The UI provides a web interface for a Storm cluster and shows detailed stats about running topologies. This command should be run under supervision with a tool like [daemontools](http://cr.yp.to/daemontools.html) or [monit](http://mmonit.com/monit/). See [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html) for more information.
+
+### drpc
+
+Syntax: `storm drpc`
+
+Launches a DRPC daemon. This command should be run under supervision with a tool like [daemontools](http://cr.yp.to/daemontools.html) or [monit](http://mmonit.com/monit/). See [Distributed RPC](Distributed-RPC.html) for more information.
+
+### drpc-client
+
+Syntax: `storm drpc-client [options] ([function argument]*)|(argument*)`
+
+Provides a very simple way to send DRPC requests. If a `-f` argument is supplied, to set the function name, all of the arguments are treated
+as arguments to the function. If no function is given the arguments must be pairs of function argument.
+
+*NOTE:* This is not really intended for production use. This is mostly because parsing out the results can be a pain.
+
+Creating an actual DRPC client only takes a few lines, so for production please go with that.
+
+```java
+Config conf = new Config();
+try (DRPCClient drpc = DRPCClient.getConfiguredClient(conf)) {
+ //User the drpc client
+ String result = drpc.execute(function, argument);
+}
+```
+
+#### Examples
+
+`storm drpc-client exclaim a exclaim b test bar`
+
+This will submit 3 separate DRPC request.
+1. function = "exclaim" args = "a"
+2. function = "exclaim" args = "b"
+3. function = "test" args = "bar"
+
+`storm drpc-client -f exclaim a b`
+
+This will submit 2 separate DRPC request.
+1. function = "exclaim" args = "a"
+2. function = "exclaim" args = "b"
+
+### blobstore
+
+Syntax: `storm blobstore cmd`
+
+list [KEY...] - lists blobs currently in the blob store
+
+cat [-f FILE] KEY - read a blob and then either write it to a file, or STDOUT (requires read access).
+
+create [-f FILE] [-a ACL ...] [--replication-factor NUMBER] KEY - create a new blob. Contents comes from a FILE or STDIN. ACL is in the form [uo]:[username]:[r-][w-][a-] can be comma separated list.
+
+update [-f FILE] KEY - update the contents of a blob. Contents comes from a FILE or STDIN (requires write access).
+
+delete KEY - delete an entry from the blob store (requires write access).
+
+set-acl [-s ACL] KEY - ACL is in the form [uo]:[username]:[r-][w-][a-] can be comma separated list (requires admin access).
+
+replication --read KEY - Used to read the replication factor of the blob.
+
+replication --update --replication-factor NUMBER KEY where NUMBER > 0. It is used to update the replication factor of a blob.
+
+For example, the following would create a mytopo:data.tgz key using the data stored in data.tgz. User alice would have full access, bob would have read/write access and everyone else would have read access.
+
+storm blobstore create mytopo:data.tgz -f data.tgz -a u:alice:rwa,u:bob:rw,o::r
+
+See [Blobstore(Distcahce)](distcache-blobstore.html) for more information.
+
+### dev-zookeeper
+
+Syntax: `storm dev-zookeeper`
+
+Launches a fresh Zookeeper server using "dev.zookeeper.path" as its local dir and "storm.zookeeper.port" as its port. This is only intended for development/testing, the Zookeeper instance launched is not configured to be used in production.
+
+### get-errors
+
+Syntax: `storm get-errors topology-name`
+
+Get the latest error from the running topology. The returned result contains the key value pairs for component-name and component-error for the components in error. The result is returned in json format.
+
+### heartbeats
+
+Syntax: `storm heartbeats [cmd]`
+
+list PATH - lists heartbeats nodes under PATH currently in the ClusterState.
+get PATH - Get the heartbeat data at PATH
+
+### kill_workers
+
+Syntax: `storm kill_workers`
+
+Kill the workers running on this supervisor. This command should be run on a supervisor node. If the cluster is running in secure mode, then user needs to have admin rights on the node to be able to successfully kill all workers.
+
+### list
+
+Syntax: `storm list`
+
+List the running topologies and their statuses.
+
+### logviewer
+
+Syntax: `storm logviewer`
+
+Launches the log viewer daemon. It provides a web interface for viewing storm log files. This command should be run under supervision with a tool like daemontools or monit.
+
+See Setting up a Storm cluster for more information.(http://storm.apache.org/documentation/Setting-up-a-Storm-cluster)
+
+### monitor
+
+Syntax: `storm monitor topology-name [-i interval-secs] [-m component-id] [-s stream-id] [-w [emitted | transferred]]`
+
+Monitor given topology's throughput interactively.
+One can specify poll-interval, component-id, stream-id, watch-item[emitted | transferred]
+ By default,
+ poll-interval is 4 seconds;
+ all component-ids will be list;
+ stream-id is 'default';
+ watch-item is 'emitted';
+
+### node-health-check
+
+Syntax: `storm node-health-check`
+
+Run health checks on the local supervisor.
+
+### pacemaker
+
+Syntax: `storm pacemaker`
+
+Launches the Pacemaker daemon. This command should be run under
+supervision with a tool like daemontools or monit.
+
+See Setting up a Storm cluster for more information.(http://storm.apache.org/documentation/Setting-up-a-Storm-cluster)
+
+### set_log_level
+
+Syntax: `storm set_log_level -l [logger name]=[log level][:optional timeout] -r [logger name] topology-name`
+
+Dynamically change topology log levels
+
+where log level is one of: ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, OFF
+and timeout is integer seconds.
+
+e.g.
+ ./bin/storm set_log_level -l ROOT=DEBUG:30 topology-name
+
+ Set the root logger's level to DEBUG for 30 seconds
+
+ ./bin/storm set_log_level -l com.myapp=WARN topology-name
+
+ Set the com.myapp logger's level to WARN for 30 seconds
+
+ ./bin/storm set_log_level -l com.myapp=WARN -l com.myOtherLogger=ERROR:123 topology-name
+
+ Set the com.myapp logger's level to WARN indefinitely, and com.myOtherLogger to ERROR for 123 seconds
+
+ ./bin/storm set_log_level -r com.myOtherLogger topology-name
+
+ Clears settings, resetting back to the original level
+
+### shell
+
+Syntax: `storm shell resourcesdir command args`
+
+Makes constructing jar and uploading to nimbus for using non JVM languages
+
+eg: `storm shell resources/ python3 topology.py arg1 arg2`
+
+### upload-credentials
+
+Syntax: `storm upload_credentials topology-name [credkey credvalue]*`
+
+Uploads a new set of credentials to a running topology
+ * `-e --exception-when-empty`: optional flag. If set, command will fail and throw exception if no credentials were uploaded.
+
+
+### version
+
+Syntax: `storm version`
+
+Prints the version number of this Storm release.
+
+### admin
+
+Syntax: `storm admin <cmd> [options]`
+
+The storm admin command provides access to several operations that can help an administrator debug or fix a cluster.
+
+`remove_corrupt_topologies` - This command should be run on a nimbus node as the same user nimbus runs as. It will go directly to zookeeper + blobstore and find topologies that appear to be corrupted because of missing blobs. It will kill those topologies.
+
+ `zk_cli [options]` - This command will launch a zookeeper cli pointing to the storm zookeeper instance logged in as the nimbus user. It should be run on a nimbus server as the user nimbus runs as.
+
+ * `-s --server <connection string>`: Set the connection string to use,
+ defaults to storm connection string.
+ * `-t --time-out <timeout>`: Set the timeout to use, defaults to storm
+ zookeeper timeout.
+ * `-w --write`: Allow for writes, defaults to read only, we don't want to
+ cause problems.
+ * `-n --no-root`: Don't include the storm root on the default connection string.
+ * `-j --jaas <jaas_file>`: Include a jaas file that should be used when
+ authenticating with ZK defaults to the
+ java.security.auth.login.config conf.
+
+`creds <topology_id>` - Print the credential keys for a topology.
+
+### help
+Syntax: `storm help [command]`
+
+Print one help message or list of available commands
diff --git a/releases/2.8.8/Common-patterns.md b/releases/2.8.8/Common-patterns.md
new file mode 100644
index 0000000000..edc2e24349
--- /dev/null
+++ b/releases/2.8.8/Common-patterns.md
@@ -0,0 +1,84 @@
+---
+title: Common Topology Patterns
+layout: documentation
+documentation: true
+---
+
+This page lists a variety of common patterns in Storm topologies.
+
+1. Batching
+2. BasicBolt
+3. In-memory caching + fields grouping combo
+4. Streaming top N
+5. TimeCacheMap for efficiently keeping a cache of things that have been recently updated
+6. CoordinatedBolt and KeyedFairBolt for Distributed RPC
+
+
+### Batching
+
+Oftentimes for efficiency reasons or otherwise, you want to process a group of tuples in batch rather than individually. For example, you may want to batch updates to a database or do a streaming aggregation of some sort.
+
+If you want reliability in your data processing, the right way to do this is to hold on to tuples in an instance variable while the bolt waits to do the batching. Once you do the batch operation, you then ack all the tuples you were holding onto.
+
+If the bolt emits tuples, then you may want to use multi-anchoring to ensure reliability. It all depends on the specific application. See [Guaranteeing message processing](Guaranteeing-message-processing.html) for more details on how reliability works.
+
+### BasicBolt
+Many bolts follow a similar pattern of reading an input tuple, emitting zero or more tuples based on that input tuple, and then acking that input tuple immediately at the end of the execute method. Bolts that match this pattern are things like functions and filters. This is such a common pattern that Storm exposes an interface called [IBasicBolt](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/IBasicBolt.html) that automates this pattern for you. See [...]
+
+### In-memory caching + fields grouping combo
+
+It's common to keep caches in-memory in Storm bolts. Caching becomes particularly powerful when you combine it with a fields grouping. For example, suppose you have a bolt that expands short URLs (like bit.ly, t.co, etc.) into long URLs. You can increase performance by keeping an LRU cache of short URL to long URL expansions to avoid doing the same HTTP requests over and over. Suppose component "urls" emits short URLS, and component "expand" expands short URLs into long URLs and keeps a [...]
+
+```java
+builder.setBolt("expand", new ExpandUrl(), parallelism)
+ .shuffleGrouping(1);
+```
+
+```java
+builder.setBolt("expand", new ExpandUrl(), parallelism)
+ .fieldsGrouping("urls", new Fields("url"));
+```
+
+The second approach will have vastly more effective caches since the same URL will always go to the same task. This avoids having duplication across any of the caches in the tasks and makes it much more likely that a short URL will hit the cache.
+
+### Streaming top N
+
+A common continuous computation done on Storm is a "streaming top N" of some sort. Suppose you have a bolt that emits tuples of the form ["value", "count"] and you want a bolt that emits the top N tuples based on the count. The simplest way to do this is to have a bolt that does a global grouping on the stream and maintains a list in memory of the top N items.
+
+This approach obviously doesn't scale to large streams since the entire stream has to go through one task. A better way to do the computation is to do many top N's in parallel across partitions of the stream, and then merge those top N's together to get the global top N. The pattern looks like this:
+
+```java
+builder.setBolt("rank", new RankObjects(), parallelism)
+ .fieldsGrouping("objects", new Fields("value"));
+builder.setBolt("merge", new MergeObjects())
+ .globalGrouping("rank");
+```
+
+This pattern works because of the fields grouping done by the first bolt which gives the partitioning you need for this to be semantically correct. You can see an example of this pattern in storm-starter [here]({{page.git-blob-base}}/examples/storm-starter/src/jvm/org/apache/storm/starter/RollingTopWords.java).
+
+If however, you have a known skew in the data being processed it can be advantageous to use partialKeyGrouping instead of fieldsGrouping. This will distribute the load for each key between two downstream bolts instead of a single one.
+
+```java
+builder.setBolt("count", new CountObjects(), parallelism)
+ .partialKeyGrouping("objects", new Fields("value"));
+builder.setBolt("rank" new AggregateCountsAndRank(), parallelism)
+ .fieldsGrouping("count", new Fields("key"))
+builder.setBolt("merge", new MergeRanksObjects())
+ .globalGrouping("rank");
+```
+
+The topology needs an extra layer of processing to aggregate the partial counts from the upstream bolts but this only processes aggregated values now so the bolt is not subject to the load caused by the skewed data. You can see an example of this pattern in storm-starter [here]({{page.git-blob-base}}/examples/storm-starter/src/jvm/org/apache/storm/starter/SkewedRollingTopWords.java).
+
+### TimeCacheMap for efficiently keeping a cache of things that have been recently updated
+
+You sometimes want to keep a cache in memory of items that have been recently "active" and have items that have been inactive for some time automatically expire. [TimeCacheMap](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/utils/TimeCacheMap.html) is an efficient data structure for doing this and provides hooks so you can insert callbacks whenever an item is expired.
+
+### CoordinatedBolt and KeyedFairBolt for Distributed RPC
+
+When building distributed RPC applications on top of Storm, there are two common patterns that are usually needed. These are encapsulated by [CoordinatedBolt](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/CoordinatedBolt.html) and [KeyedFairBolt](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/KeyedFairBolt.html) which are part of the "standard library" that ships with the Storm codebase.
+
+`CoordinatedBolt` wraps the bolt containing your logic and figures out when your bolt has received all the tuples for any given request. It makes heavy use of direct streams to do this.
+
+`KeyedFairBolt` also wraps the bolt containing your logic and makes sure your topology processes multiple DRPC invocations at the same time, instead of doing them serially one at a time.
+
+See [Distributed RPC](Distributed-RPC.html) for more details.
diff --git a/releases/2.8.8/Concepts.md b/releases/2.8.8/Concepts.md
new file mode 100644
index 0000000000..a2b0218983
--- /dev/null
+++ b/releases/2.8.8/Concepts.md
@@ -0,0 +1,120 @@
+---
+title: Concepts
+layout: documentation
+documentation: true
+---
+
+This page lists the main concepts of Storm and links to resources where you can find more information. The concepts discussed are:
+
+1. Topologies
+2. Streams
+3. Spouts
+4. Bolts
+5. Stream groupings
+6. Reliability
+7. Tasks
+8. Workers
+
+### Topologies
+
+The logic for a realtime application is packaged into a Storm topology. A Storm topology is analogous to a MapReduce job. One key difference is that a MapReduce job eventually finishes, whereas a topology runs forever (or until you kill it, of course). A topology is a graph of spouts and bolts that are connected with stream groupings. These concepts are described below.
+
+**Resources:**
+
+* [TopologyBuilder](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/TopologyBuilder.html): use this class to construct topologies in Java
+* [Running topologies on a production cluster](Running-topologies-on-a-production-cluster.html)
+* [Local mode](Local-mode.html): Read this to learn how to develop and test topologies in local mode.
+
+### Streams
+
+The stream is the core abstraction in Storm. A stream is an unbounded sequence of tuples that is processed and created in parallel in a distributed fashion. Streams are defined with a schema that names the fields in the stream's tuples. By default, tuples can contain integers, longs, shorts, bytes, strings, doubles, floats, booleans, and byte arrays. You can also define your own serializers so that custom types can be used natively within tuples.
+
+Every stream is given an id when declared. Since single-stream spouts and bolts are so common, [OutputFieldsDeclarer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/OutputFieldsDeclarer.html) has convenience methods for declaring a single stream without specifying an id. In this case, the stream is given the default id of "default".
+
+
+**Resources:**
+
+* [Tuple](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/tuple/Tuple.html): streams are composed of tuples
+* [OutputFieldsDeclarer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/OutputFieldsDeclarer.html): used to declare streams and their schemas
+* [Serialization](Serialization.html): Information about Storm's dynamic typing of tuples and declaring custom serializations
+
+### Spouts
+
+A spout is a source of streams in a topology. Generally spouts will read tuples from an external source and emit them into the topology (e.g. a Kestrel queue or the Twitter API). Spouts can either be __reliable__ or __unreliable__. A reliable spout is capable of replaying a tuple if it failed to be processed by Storm, whereas an unreliable spout forgets about the tuple as soon as it is emitted.
+
+Spouts can emit more than one stream. To do so, declare multiple streams using the `declareStream` method of [OutputFieldsDeclarer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/OutputFieldsDeclarer.html) and specify the stream to emit to when using the `emit` method on [SpoutOutputCollector](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/spout/SpoutOutputCollector.html).
+
+The main method on spouts is `nextTuple`. `nextTuple` either emits a new tuple into the topology or simply returns if there are no new tuples to emit. It is imperative that `nextTuple` does not block for any spout implementation, because Storm calls all the spout methods on the same thread.
+
+The other main methods on spouts are `ack` and `fail`. These are called when Storm detects that a tuple emitted from the spout either successfully completed through the topology or failed to be completed. `ack` and `fail` are only called for reliable spouts. See [the Javadoc](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/spout/ISpout.html) for more information.
+
+**Resources:**
+
+* [IRichSpout](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/IRichSpout.html): this is the interface that spouts must implement.
+* [Guaranteeing message processing](Guaranteeing-message-processing.html)
+
+### Bolts
+
+All processing in topologies is done in bolts. Bolts can do anything from filtering, functions, aggregations, joins, talking to databases, and more.
+
+Bolts can do simple stream transformations. Doing complex stream transformations often requires multiple steps and thus multiple bolts. For example, transforming a stream of tweets into a stream of trending images requires at least two steps: a bolt to do a rolling count of retweets for each image, and one or more bolts to stream out the top X images (you can do this particular stream transformation in a more scalable way with three bolts than with two).
+
+Bolts can emit more than one stream. To do so, declare multiple streams using the `declareStream` method of [OutputFieldsDeclarer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/OutputFieldsDeclarer.html) and specify the stream to emit to when using the `emit` method on [OutputCollector](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/OutputCollector.html).
+
+When you declare a bolt's input streams, you always subscribe to specific streams of another component. If you want to subscribe to all the streams of another component, you have to subscribe to each one individually. [InputDeclarer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/InputDeclarer.html) has syntactic sugar for subscribing to streams declared on the default stream id. Saying `declarer.shuffleGrouping("1")` subscribes to the default stream [...]
+
+The main method in bolts is the `execute` method which takes in as input a new tuple. Bolts emit new tuples using the [OutputCollector](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/OutputCollector.html) object. Bolts must call the `ack` method on the `OutputCollector` for every tuple they process so that Storm knows when tuples are completed (and can eventually determine that its safe to ack the original spout tuples). For the common case of processing [...]
+
+Its perfectly fine to launch new threads in bolts that do processing asynchronously. [OutputCollector](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/OutputCollector.html) is thread-safe and can be called at any time.
+
+**Resources:**
+
+* [IRichBolt](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/IRichBolt.html): this is general interface for bolts.
+* [IBasicBolt](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/IBasicBolt.html): this is a convenience interface for defining bolts that do filtering or simple functions.
+* [OutputCollector](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/OutputCollector.html): bolts emit tuples to their output streams using an instance of this class
+* [Guaranteeing message processing](Guaranteeing-message-processing.html)
+
+### Stream groupings
+
+Part of defining a topology is specifying for each bolt which streams it should receive as input. A stream grouping defines how that stream should be partitioned among the bolt's tasks.
+
+There are eight built-in stream groupings in Storm, and you can implement a custom stream grouping by implementing the [CustomStreamGrouping](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/grouping/CustomStreamGrouping.html) interface:
+
+1. **Shuffle grouping**: Tuples are randomly distributed across the bolt's tasks in a way such that each bolt is guaranteed to get an equal number of tuples.
+2. **Fields grouping**: The stream is partitioned by the fields specified in the grouping. For example, if the stream is grouped by the "user-id" field, tuples with the same "user-id" will always go to the same task, but tuples with different "user-id"'s may go to different tasks.
+3. **Partial Key grouping**: The stream is partitioned by the fields specified in the grouping, like the Fields grouping, but are load balanced between two downstream bolts, which provides better utilization of resources when the incoming data is skewed. [This paper](https://melmeric.files.wordpress.com/2014/11/the-power-of-both-choices-practical-load-balancing-for-distributed-stream-processing-engines.pdf) provides a good explanation of how it works and the advantages it provides.
+4. **All grouping**: The stream is replicated across all the bolt's tasks. Use this grouping with care.
+5. **Global grouping**: The entire stream goes to a single one of the bolt's tasks. Specifically, it goes to the task with the lowest id.
+6. **None grouping**: This grouping specifies that you don't care how the stream is grouped. Currently, none groupings are equivalent to shuffle groupings. Eventually though, Storm will push down bolts with none groupings to execute in the same thread as the bolt or spout they subscribe from (when possible).
+7. **Direct grouping**: This is a special kind of grouping. A stream grouped this way means that the __producer__ of the tuple decides which task of the consumer will receive this tuple. Direct groupings can only be declared on streams that have been declared as direct streams. Tuples emitted to a direct stream must be emitted using one of the [emitDirect](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/OutputCollector.html#emitDirect-int-java.util.Collec [...]
+8. **Local or shuffle grouping**: If the target bolt has one or more tasks in the same worker process, tuples will be shuffled to just those in-process tasks. Otherwise, this acts like a normal shuffle grouping.
+
+**Resources:**
+
+* [TopologyBuilder](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/TopologyBuilder.html): use this class to define topologies
+* [InputDeclarer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/InputDeclarer.html): this object is returned whenever `setBolt` is called on `TopologyBuilder` and is used for declaring a bolt's input streams and how those streams should be grouped
+
+### Reliability
+
+Storm guarantees that every spout tuple will be fully processed by the topology. It does this by tracking the tree of tuples triggered by every spout tuple and determining when that tree of tuples has been successfully completed. Every topology has a "message timeout" associated with it. If Storm fails to detect that a spout tuple has been completed within that timeout, then it fails the tuple and replays it later.
+
+To take advantage of Storm's reliability capabilities, you must tell Storm when new edges in a tuple tree are being created and tell Storm whenever you've finished processing an individual tuple. These are done using the [OutputCollector](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/OutputCollector.html) object that bolts use to emit tuples. Anchoring is done in the `emit` method, and you declare that you're finished with a tuple using the `ack` method.
+
+This is all explained in much more detail in [Guaranteeing message processing](Guaranteeing-message-processing.html).
+
+### Tasks
+
+Each spout or bolt executes as many tasks across the cluster. Each task corresponds to one thread of execution, and stream groupings define how to send tuples from one set of tasks to another set of tasks. You set the parallelism for each spout or bolt in the `setSpout` and `setBolt` methods of [TopologyBuilder](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/TopologyBuilder.html).
+
+### Workers
+
+Topologies execute across one or more worker processes. Each worker process is a physical JVM and executes a subset of all the tasks for the topology. For example, if the combined parallelism of the topology is 300 and 50 workers are allocated, then each worker will execute 6 tasks (as threads within the worker). Storm tries to spread the tasks evenly across all the workers.
+
+**Resources:**
+
+* [Config.TOPOLOGY_WORKERS](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#TOPOLOGY_WORKERS): this config sets the number of workers to allocate for executing the topology
+
+### Performance Tuning
+
+Refer to [performance tuning guide](Performance.html).
+
diff --git a/releases/2.8.8/Configuration.md b/releases/2.8.8/Configuration.md
new file mode 100644
index 0000000000..57b090d052
--- /dev/null
+++ b/releases/2.8.8/Configuration.md
@@ -0,0 +1,41 @@
+---
+title: Configuration
+layout: documentation
+documentation: true
+---
+Storm has a variety of configurations for tweaking the behavior of nimbus, supervisors, and running topologies. Some configurations are system configurations and cannot be modified on topology by topology basis, whereas other configurations can be modified per topology.
+
+Every configuration has a default value defined in [defaults.yaml]({{page.git-blob-base}}/conf/defaults.yaml) in the Storm codebase. You can override these configurations by defining a storm.yaml in the classpath of Nimbus and the supervisors. Finally, you can define a topology-specific configuration that you submit along with your topology when using [StormSubmitter](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/StormSubmitter.html). However, the topology-s [...]
+
+Storm 0.7.0 and onwards lets you override configuration on a per-bolt/per-spout basis. The only configurations that can be overridden this way are:
+
+1. "topology.debug"
+2. "topology.max.spout.pending"
+3. "topology.max.task.parallelism"
+4. "topology.kryo.register": This works a little bit differently than the other ones, since the serializations will be available to all components in the topology. More details on [Serialization](Serialization.html).
+
+The Java API lets you specify component specific configurations in two ways:
+
+1. *Internally:* Override `getComponentConfiguration` in any spout or bolt and return the component-specific configuration map.
+2. *Externally:* `setSpout` and `setBolt` in `TopologyBuilder` return an object with methods `addConfiguration` and `addConfigurations` that can be used to override the configurations for the component.
+
+The preference order for configuration values is defaults.yaml < storm.yaml < topology specific configuration < internal component specific configuration < external component specific configuration.
+
+# Bolts, Spouts, and Plugins
+In almost all cases configuration for a bolt or a spout should be done through setters on the bolt or spout implementation and not the topology conf. In some rare cases, it may make sense to
+expose topology wide configurations that are not currently a part of [Config](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html) or [DaemonConfig](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/DaemonConfig.html) such as
+when writing a custom scheduler or a plugin to some part of storm. In those
+cases you can create your own class like Config but implements [Validated](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/validation/Validated.html). Any `public static final String` field declared in this
+class will be treated as a config and annotations from the `org.apache.storm.validation.ConfigValidationAnnotations` class can be used to enforce what is stored in that config.
+To let the validator know about this class you need to treat the class
+like a service that will be loaded through a ServiceLoader for the Validated class and include a `META-INF/services/org.apache.storm.validation.Validated` file in your jar that holds
+the name of your Config class.
+
+**Resources:**
+
+* [Config](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html): a listing of client configurations as well as a helper class for creating topology specific configurations
+* [DaemonConfig](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/DaemonConfig.html): a listing of Storm Daemon configurations.
+* [defaults.yaml]({{page.git-blob-base}}/conf/defaults.yaml): the default values for all configurations
+* [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html): explains how to create and configure a Storm cluster
+* [Running topologies on a production cluster](Running-topologies-on-a-production-cluster.html): lists useful configurations when running topologies on a cluster
+* [Local mode](Local-mode.html): lists useful configurations when using local mode
diff --git a/releases/2.8.8/Contributing-to-Storm.md b/releases/2.8.8/Contributing-to-Storm.md
new file mode 100644
index 0000000000..2c6ad1f2d9
--- /dev/null
+++ b/releases/2.8.8/Contributing-to-Storm.md
@@ -0,0 +1,33 @@
+---
+title: Contributing
+layout: documentation
+documentation: true
+---
+
+### Getting started with contributing
+
+Some of the issues on the [issue tracker](https://issues.apache.org/jira/browse/STORM) are marked with the ["Newbie"](https://issues.apache.org/jira/browse/STORM-2891?jql=project%20%3D%20STORM%20AND%20status%20%3D%20Open%20AND%20labels%20in%20(newbie%2C%20%22newbie%2B%2B%22)) label. If you're interested in contributing to Storm but don't know where to begin, these are good issues to start with. These issues are a great way to get your feet wet with learning the codebase because they requ [...]
+
+### Learning the codebase
+
+The [Implementation docs](Implementation-docs.html) section of the wiki gives detailed walkthroughs of the codebase. Reading through these docs is highly recommended to understand the codebase.
+
+### Contribution process
+
+Contributions to the Storm codebase should be sent as [GitHub](https://github.com/apache/storm) pull requests. If there's any problems to the pull request we can iterate on it using GitHub's commenting features.
+
+For small patches, feel free to submit pull requests directly for them. For larger contributions, please use the following process. The idea behind this process is to prevent any wasted work and catch design issues early on:
+
+1. Open an issue on the [issue tracker](https://issues.apache.org/jira/browse/STORM) if one doesn't exist already
+2. Comment on the issue with your plan for implementing the issue. Explain what pieces of the codebase you're going to touch and how everything is going to fit together.
+3. Storm committers will iterate with you on the design to make sure you're on the right track
+4. Implement your issue, submit a pull request, and iterate from there.
+
+### Modules built on top of Storm
+
+Modules built on top of Storm (like spouts, bolts, etc) that aren't appropriate for Storm core can be done as your own project or as part of [@stormprocessor](https://github.com/stormprocessor). To be part of @stormprocessor put your project on your own Github and then send an email to the mailing list proposing to make it part of @stormprocessor. Then the community can discuss whether it's useful enough to be part of @stormprocessor. Then you'll be added to the @stormprocessor organizat [...]
+
+### Contributing documentation
+
+Documentation contributions are very welcome! The best way to send contributions is as emails through the mailing list.
+
diff --git a/releases/2.8.8/Creating-a-new-Storm-project.md b/releases/2.8.8/Creating-a-new-Storm-project.md
new file mode 100644
index 0000000000..35ab1eba1e
--- /dev/null
+++ b/releases/2.8.8/Creating-a-new-Storm-project.md
@@ -0,0 +1,25 @@
+---
+title: Creating a New Storm Project
+layout: documentation
+documentation: true
+---
+This page outlines how to set up a Storm project for development. The steps are:
+
+1. Add Storm jars to classpath
+2. If using multilang, add multilang dir to classpath
+
+Follow along to see how to set up the [storm-starter]({{page.git-blob-base}}/examples/storm-starter) project in Eclipse.
+
+### Add Storm jars to classpath
+
+You'll need the Storm jars on your classpath to develop Storm topologies. Using [Maven](Maven.html) is highly recommended. [Here's an example]({{page.git-blob-base}}/examples/storm-starter/pom.xml) of how to setup your pom.xml for a Storm project. If you don't want to use Maven, you can include the jars from the Storm release on your classpath.
+
+To set up the classpath in Eclipse, create a new Java project, include `src/jvm/` as a source path, and make sure all the jars in `lib/` and `lib/dev/` are in the `Referenced Libraries` section of the project.
+
+### If using multilang, add multilang dir to classpath
+
+If you implement spouts or bolts in languages other than Java, then those implementations should be under the `multilang/resources/` directory of the project. For Storm to find these files in local mode, the `resources/` dir needs to be on the classpath. You can do this in Eclipse by adding `multilang/` as a source folder. You may also need to add multilang/resources as a source directory.
+
+For more information on writing topologies in other languages, see [Using non-JVM languages with Storm](Using-non-JVM-languages-with-Storm.html).
+
+To test that everything is working in Eclipse, you should now be able to `Run` the `WordCountTopology.java` file. You will see messages being emitted at the console for 10 seconds.
diff --git a/releases/2.8.8/DSLs-and-multilang-adapters.md b/releases/2.8.8/DSLs-and-multilang-adapters.md
new file mode 100644
index 0000000000..917b41905a
--- /dev/null
+++ b/releases/2.8.8/DSLs-and-multilang-adapters.md
@@ -0,0 +1,11 @@
+---
+title: Storm DSLs and Multi-Lang Adapters
+layout: documentation
+documentation: true
+---
+* [Clojure DSL](Clojure-DSL.html)
+* [Scala DSL](https://github.com/velvia/ScalaStorm)
+* [JRuby DSL](https://github.com/colinsurprenant/redstorm)
+* [Storm/Esper integration](https://github.com/tomdz/storm-esper): Streaming SQL on top of Storm
+* [io-storm](https://github.com/dan-blanchard/io-storm): Perl multilang adapter
+* [FsShelter](https://github.com/Prolucid/FsShelter): F# DSL and runtime with protobuf multilang
diff --git a/releases/2.8.8/Daemon-Fault-Tolerance.md b/releases/2.8.8/Daemon-Fault-Tolerance.md
new file mode 100644
index 0000000000..8dce601a8b
--- /dev/null
+++ b/releases/2.8.8/Daemon-Fault-Tolerance.md
@@ -0,0 +1,30 @@
+---
+title: Daemon Fault Tolerance
+layout: documentation
+documentation: true
+---
+Storm has several different daemon processes. Nimbus that schedules workers, supervisors that launch and kill workers, the log viewer that gives access to logs, and the UI that shows the status of a cluster.
+
+## What happens when a worker dies?
+
+When a worker dies, the supervisor will restart it. If it continuously fails on startup and is unable to heartbeat to Nimbus, Nimbus will reschedule the worker.
+
+## What happens when a node dies?
+
+The tasks assigned to that machine will time-out and Nimbus will reassign those tasks to other machines.
+
+## What happens when Nimbus or Supervisor daemons die?
+
+The Nimbus and Supervisor daemons are designed to be fail-fast (process self-destructs whenever any unexpected situation is encountered) and stateless (all state is kept in Zookeeper or on disk). As described in [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html), the Nimbus and Supervisor daemons must be run under supervision using a tool like daemontools or monit. So if the Nimbus or Supervisor daemons die, they restart like nothing happened.
+
+Most notably, no worker processes are affected by the death of Nimbus or the Supervisors. This is in contrast to Hadoop, where if the JobTracker dies, all the running jobs are lost.
+
+## Is Nimbus a single point of failure?
+
+If you lose the Nimbus node, the workers will still continue to function. Additionally, supervisors will continue to restart workers if they die. However, without Nimbus, workers won't be reassigned to other machines when necessary (like if you lose a worker machine).
+
+Storm Nimbus is highly available since 1.0.0. More information please refer to [Nimbus HA Design](nimbus-ha-design.html) document.
+
+## How does Storm guarantee data processing?
+
+Storm provides mechanisms to guarantee data processing even if nodes die or messages are lost. See [Guaranteeing message processing](Guaranteeing-message-processing.html) for the details.
diff --git a/releases/2.8.8/Defining-a-non-jvm-language-dsl-for-storm.md b/releases/2.8.8/Defining-a-non-jvm-language-dsl-for-storm.md
new file mode 100644
index 0000000000..aad5525085
--- /dev/null
+++ b/releases/2.8.8/Defining-a-non-jvm-language-dsl-for-storm.md
@@ -0,0 +1,45 @@
+---
+title: Defining a Non-JVM DSL for Storm
+layout: documentation
+documentation: true
+---
+The right place to start to learn how to make a non-JVM DSL for Storm is [storm-client/src/storm.thrift]({{page.git-blob-base}}/storm-client/src/storm.thrift). Since Storm topologies are just Thrift structures, and Nimbus is a Thrift daemon, you can create and submit topologies in any language.
+
+When you create the Thrift structs for spouts and bolts, the code for the spout or bolt is specified in the ComponentObject struct:
+
+```
+union ComponentObject {
+ 1: binary serialized_java;
+ 2: ShellComponent shell;
+ 3: JavaObject java_object;
+}
+```
+
+For a Python DSL, you would want to make use of "2" and "3". ShellComponent lets you specify a script to run that component (e.g., your python code). And JavaObject lets you specify native java spouts and bolts for the component (and Storm will use reflection to create that spout or bolt).
+
+There's a "storm shell" command that will help with submitting a topology. Its usage is like this:
+
+```
+storm shell resources/ python3 topology.py arg1 arg2
+```
+
+storm shell will then package resources/ into a jar, upload the jar to Nimbus, and call your topology.py script like this:
+
+```
+python3 topology.py arg1 arg2 {nimbus-host} {nimbus-port} {uploaded-jar-location}
+```
+
+Then you can connect to Nimbus using the Thrift API and submit the topology, passing {uploaded-jar-location} into the submitTopology method. For reference, here's the submitTopology definition:
+
+```java
+void submitTopology(
+ 1: string name,
+ 2: string uploadedJarLocation,
+ 3: string jsonConf,
+ 4: StormTopology topology)
+ throws (
+ 1: AlreadyAliveException e,
+ 2: InvalidTopologyException ite);
+```
+
+Finally, one of the key things to do in a non-JVM DSL is make it easy to define the entire topology in one file (the bolts, spouts, and the definition of the topology).
diff --git a/releases/2.8.8/Distributed-RPC.md b/releases/2.8.8/Distributed-RPC.md
new file mode 100644
index 0000000000..e92f18df90
--- /dev/null
+++ b/releases/2.8.8/Distributed-RPC.md
@@ -0,0 +1,216 @@
+---
+title: Distributed RPC
+layout: documentation
+documentation: true
+---
+The idea behind distributed RPC (DRPC) is to parallelize the computation of really intense functions on the fly using Storm. The Storm topology takes in as input a stream of function arguments, and it emits an output stream of the results for each of those function calls.
+
+DRPC is not so much a feature of Storm as it is a pattern expressed from Storm's primitives of streams, spouts, bolts, and topologies. DRPC could have been packaged as a separate library from Storm, but it's so useful that it's bundled with Storm.
+
+### High level overview
+
+Distributed RPC is coordinated by a "DRPC server" (Storm comes packaged with an implementation of this). The DRPC server coordinates receiving an RPC request, sending the request to the Storm topology, receiving the results from the Storm topology, and sending the results back to the waiting client. From a client's perspective, a distributed RPC call looks just like a regular RPC call. For example, here's how a client would compute the results for the "reach" function with the argument " [...]
+
+```java
+Config conf = new Config();
+conf.put("storm.thrift.transport", "org.apache.storm.security.auth.plain.PlainSaslTransportPlugin");
+conf.put(Config.STORM_NIMBUS_RETRY_TIMES, 3);
+conf.put(Config.STORM_NIMBUS_RETRY_INTERVAL, 10);
+conf.put(Config.STORM_NIMBUS_RETRY_INTERVAL_CEILING, 20);
+DRPCClient client = new DRPCClient(conf, "drpc-host", 3772);
+String result = client.execute("reach", "http://twitter.com");
+```
+
+or if you just want to use a preconfigured client you can call. The exact host will be selected randomly from the configured set of hosts, if the host appears to be down it will loop through all configured hosts looking for one that works.
+
+```java
+DRPCClient client = DRPCClient.getConfiguredClient(conf);
+String result = client.execute("reach", "http://twitter.com");
+```
+
+The distributed RPC workflow looks like this:
+
+
+
+A client sends the DRPC server the name of the function to execute and the arguments to that function. The topology implementing that function uses a `DRPCSpout` to receive a function invocation stream from the DRPC server. Each function invocation is tagged with a unique id by the DRPC server. The topology then computes the result and at the end of the topology a bolt called `ReturnResults` connects to the DRPC server and gives it the result for the function invocation id. The DRPC serv [...]
+
+### LinearDRPCTopologyBuilder
+
+Storm comes with a topology builder called [LinearDRPCTopologyBuilder](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/drpc/LinearDRPCTopologyBuilder.html) that automates almost all the steps involved for doing DRPC. These include:
+
+1. Setting up the spout
+2. Returning the results to the DRPC server
+3. Providing functionality to bolts for doing finite aggregations over groups of tuples
+
+Let's look at a simple example. Here's the implementation of a DRPC topology that returns its input argument with a "!" appended:
+
+```java
+public static class ExclaimBolt extends BaseBasicBolt {
+ public void execute(Tuple tuple, BasicOutputCollector collector) {
+ String input = tuple.getString(1);
+ collector.emit(new Values(tuple.getValue(0), input + "!"));
+ }
+
+ public void declareOutputFields(OutputFieldsDeclarer declarer) {
+ declarer.declare(new Fields("id", "result"));
+ }
+}
+
+public static void main(String[] args) throws Exception {
+ LinearDRPCTopologyBuilder builder = new LinearDRPCTopologyBuilder("exclamation");
+ builder.addBolt(new ExclaimBolt(), 3);
+ // ...
+}
+```
+
+As you can see, there's very little to it. When creating the `LinearDRPCTopologyBuilder`, you tell it the name of the DRPC function for the topology. A single DRPC server can coordinate many functions, and the function name distinguishes the functions from one another. The first bolt you declare will take in as input 2-tuples, where the first field is the request id and the second field is the arguments for that request. `LinearDRPCTopologyBuilder` expects the last bolt to emit an output [...]
+
+In this example, `ExclaimBolt` simply appends a "!" to the second field of the tuple. `LinearDRPCTopologyBuilder` handles the rest of the coordination of connecting to the DRPC server and sending results back.
+
+### Local mode DRPC
+
+In the past to use DRPC in local mode it took creating a special LocalDRPC instance. This can still be used when writing tests for your code, but in the current version of storm when you run in local mode a LocalDRPC
+instance is also created, and any DRPCClient created will link to it instead of the outside world. This means that any interaction you want to test needs to be a part of the script that launches the topology, just like
+with LocalDRPC.
+
+### Remote mode DRPC
+
+Using DRPC on an actual cluster is also straightforward. There's three steps:
+
+1. Launch DRPC server(s)
+2. Configure the locations of the DRPC servers
+3. Submit DRPC topologies to Storm cluster
+
+Launching a DRPC server can be done with the `storm` script and is just like launching Nimbus or the UI:
+
+```
+bin/storm drpc
+```
+
+Next, you need to configure your Storm cluster to know the locations of the DRPC server(s). This is how `DRPCSpout` knows from where to read function invocations. This can be done through the `storm.yaml` file or the topology configurations. You should also specify storm.thrift.transport property to match DRPCClient settings. Configuring this through the `storm.yaml` looks something like this:
+
+```yaml
+drpc.servers:
+ - "drpc1.foo.com"
+ - "drpc2.foo.com"
+drpc.http.port: 8081
+storm.thrift.transport: "org.apache.storm.security.auth.plain.PlainSaslTransportPlugin"
+```
+
+Finally, you launch DRPC topologies using `StormSubmitter` just like you launch any other topology. To run the above example in remote mode, you do something like this:
+
+```java
+StormSubmitter.submitTopology("exclamation-drpc", conf, builder.createRemoteTopology());
+```
+
+`createRemoteTopology` is used to create topologies suitable for Storm clusters.
+
+Assuming that the topology is listening on the `exclaim` function you can execute something several differnt ways.
+
+Programatically:
+```java
+Config conf = new Config();
+try (DRPCClient drpc = DRPCClient.getConfiguredClient(conf)) {
+ //User the drpc client
+ String result = drpc.execute("exclaim", "argument");
+}
+```
+
+through curl:
+```curl http://hostname:8081/drpc/exclaim/argument```
+
+Through the command line:
+```bin/storm drpc-client exclaim argument```
+
+### A more complex example
+
+The exclamation DRPC example was a toy example for illustrating the concepts of DRPC. Let's look at a more complex example which really needs the parallelism a Storm cluster provides for computing the DRPC function. The example we'll look at is computing the reach of a URL on Twitter.
+
+The reach of a URL is the number of unique people exposed to a URL on Twitter. To compute reach, you need to:
+
+1. Get all the people who tweeted the URL
+2. Get all the followers of all those people
+3. Unique the set of followers
+4. Count the unique set of followers
+
+A single reach computation can involve thousands of database calls and tens of millions of follower records during the computation. It's a really, really intense computation. As you're about to see, implementing this function on top of Storm is dead simple. On a single machine, reach can take minutes to compute; on a Storm cluster, you can compute reach for even the hardest URLs in a couple seconds.
+
+A sample reach topology is defined in storm-starter [here]({{page.git-blob-base}}/examples/storm-starter/src/jvm/org/apache/storm/starter/ReachTopology.java). Here's how you define the reach topology:
+
+```java
+LinearDRPCTopologyBuilder builder = new LinearDRPCTopologyBuilder("reach");
+builder.addBolt(new GetTweeters(), 3);
+builder.addBolt(new GetFollowers(), 12)
+ .shuffleGrouping();
+builder.addBolt(new PartialUniquer(), 6)
+ .fieldsGrouping(new Fields("id", "follower"));
+builder.addBolt(new CountAggregator(), 2)
+ .fieldsGrouping(new Fields("id"));
+```
+
+The topology executes as four steps:
+
+1. `GetTweeters` gets the users who tweeted the URL. It transforms an input stream of `[id, url]` into an output stream of `[id, tweeter]`. Each `url` tuple will map to many `tweeter` tuples.
+2. `GetFollowers` gets the followers for the tweeters. It transforms an input stream of `[id, tweeter]` into an output stream of `[id, follower]`. Across all the tasks, there may of course be duplication of follower tuples when someone follows multiple people who tweeted the same URL.
+3. `PartialUniquer` groups the followers stream by the follower id. This has the effect of the same follower going to the same task. So each task of `PartialUniquer` will receive mutually independent sets of followers. Once `PartialUniquer` receives all the follower tuples directed at it for the request id, it emits the unique count of its subset of followers.
+4. Finally, `CountAggregator` receives the partial counts from each of the `PartialUniquer` tasks and sums them up to complete the reach computation.
+
+Let's take a look at the `PartialUniquer` bolt:
+
+```java
+public class PartialUniquer extends BaseBatchBolt {
+ BatchOutputCollector _collector;
+ Object _id;
+ Set<String> _followers = new HashSet<String>();
+
+ @Override
+ public void prepare(Map conf, TopologyContext context, BatchOutputCollector collector, Object id) {
+ _collector = collector;
+ _id = id;
+ }
+
+ @Override
+ public void execute(Tuple tuple) {
+ _followers.add(tuple.getString(1));
+ }
+
+ @Override
+ public void finishBatch() {
+ _collector.emit(new Values(_id, _followers.size()));
+ }
+
+ @Override
+ public void declareOutputFields(OutputFieldsDeclarer declarer) {
+ declarer.declare(new Fields("id", "partial-count"));
+ }
+}
+```
+
+`PartialUniquer` implements `IBatchBolt` by extending `BaseBatchBolt`. A batch bolt provides a first class API to processing a batch of tuples as a concrete unit. A new instance of the batch bolt is created for each request id, and Storm takes care of cleaning up the instances when appropriate.
+
+When `PartialUniquer` receives a follower tuple in the `execute` method, it adds it to the set for the request id in an internal `HashSet`.
+
+Batch bolts provide the `finishBatch` method which is called after all the tuples for this batch targeted at this task have been processed. In the callback, `PartialUniquer` emits a single tuple containing the unique count for its subset of follower ids.
+
+Under the hood, `CoordinatedBolt` is used to detect when a given bolt has received all of the tuples for any given request id. `CoordinatedBolt` makes use of direct streams to manage this coordination.
+
+The rest of the topology should be self-explanatory. As you can see, every single step of the reach computation is done in parallel, and defining the DRPC topology was extremely simple.
+
+### Non-linear DRPC topologies
+
+`LinearDRPCTopologyBuilder` only handles "linear" DRPC topologies, where the computation is expressed as a sequence of steps (like reach). It's not hard to imagine functions that would require a more complicated topology with branching and merging of the bolts. For now, to do this you'll need to drop down into using `CoordinatedBolt` directly. Be sure to talk about your use case for non-linear DRPC topologies on the mailing list to inform the construction of more general abstractions for [...]
+
+### How LinearDRPCTopologyBuilder works
+
+* DRPCSpout emits [args, return-info]. return-info is the host and port of the DRPC server as well as the id generated by the DRPC server
+* constructs a topology comprising of:
+ * DRPCSpout
+ * PrepareRequest (generates a request id and creates a stream for the return info and a stream for the args)
+ * CoordinatedBolt wrappers and direct groupings
+ * JoinResult (joins the result with the return info)
+ * ReturnResult (connects to the DRPC server and returns the result)
+* LinearDRPCTopologyBuilder is a good example of a higher level abstraction built on top of Storm's primitives
+
+### Advanced
+* KeyedFairBolt for weaving the processing of multiple requests at the same time
+* How to use `CoordinatedBolt` directly
diff --git a/releases/2.8.8/Docker-support.md b/releases/2.8.8/Docker-support.md
new file mode 100644
index 0000000000..1d7c754a70
--- /dev/null
+++ b/releases/2.8.8/Docker-support.md
@@ -0,0 +1,135 @@
+---
+title: Docker Support
+layout: documentation
+documentation: true
+---
+
+# Docker Support
+
+This page describes how storm supervisor launches the worker in a docker container.
+
+Note: This has only been tested on RHEL7.
+
+## Motivation
+
+This feature is mostly about security and portability. With workers running inside of docker containers, we isolate running user code from each other and from the hosted machine so that the whole system is less vulnerable to attack.
+It also allows users to run their topologies on different os versions using different docker images.
+
+## Implementation
+
+Essentially, `DockerManager` composes a docker-run command and uses `worker-launcher` executable to execute the command
+to launch a container. The `storm-worker-script.sh` script is the actual command to launch the worker process and logviewer in the container.
+One container ID is mapped to one worker ID conceptually. When the worker process dies, the container exits.
+
+For security, when the supervisor launches the docker container, it makes the whole container read-only except some explicit bind mount locations.
+It also drops all the kernel capabilities and disables container processes from gaining new privileges.
+
+For security reasons, we can drop privileges of containers like PTRACE. Consequently, `jmap`, `strace` and some other debugging tools cannot be used directly in the container when entered with docker-exec command.
+We need to install `nscd` and have it running in the system. Storm will bind mount nscd directory when it launches the container.
+And `nsenter` will be used to enter the docker container without losing privileges. This functionality is also implemented in `worker-launcher` executable.
+
+The command that will be run by `worker-launcher` executable to launch a container will be something like:
+
+```bash
+run --name=8198e1f0-f323-4b9d-8625-e4fd640cd058 \
+--user=<uid>:<gid> \
+-d \
+--net=host \
+--read-only \
+-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
+-v /usr/share/apache-storm-2.3.0:/usr/share/apache-storm-2.3.0:ro \
+-v /<storm-local-dir>/supervisor:/<storm-local-dir>/supervisor:ro \
+-v /<storm-local-dir>/workers/8198e1f0-f323-4b9d-8625-e4fd640cd058:/<storm-local-dir>/workers/8198e1f0-f323-4b9d-8625-e4fd640cd058 \
+-v /<workers-artifacts-dir>/workers-artifacts/word-count-1-1591895933/6703:/<workers-artifacts-dir>/workers-artifacts/word-count-1-1591895933/6703 \
+-v /<storm-local-dir>/workers-users/8198e1f0-f323-4b9d-8625-e4fd640cd058:/<storm-local-dir>/workers-users/8198e1f0-f323-4b9d-8625-e4fd640cd058 \
+-v /var/run/nscd:/var/run/nscd \
+-v /<storm-local-dir>/supervisor/stormdist/word-count-1-1591895933/shared_by_topology:/<storm-local-dir>/supervisor/stormdist/word-count-1-1591895933/shared_by_topology \
+-v /<storm-local-dir>/workers/8198e1f0-f323-4b9d-8625-e4fd640cd058/tmp:/tmp \
+-v /etc/storm:/etc/storm:ro \
+--cgroup-parent=/storm \
+--group-add <gid> \
+--workdir=/<storm-local-dir>/workers/8198e1f0-f323-4b9d-8625-e4fd640cd058 \
+--cidfile=/<storm-local-dir>/workers/8198e1f0-f323-4b9d-8625-e4fd640cd058/container.cid \
+--cap-drop=ALL \
+--security-opt no-new-privileges \
+--security-opt seccomp=/usr/share/apache-storm-2.3.0/conf/seccomp.json \
+--cpus=2.6 xxx.xxx.com:8080/storm/storm/rhel7:latest \
+bash /<storm-local-dir>/workers/8198e1f0-f323-4b9d-8625-e4fd640cd058/storm-worker-script.sh
+```
+
+
+## Setup
+
+To make supervisor work with docker, you need to configure related settings correctly following the instructions below.
+
+### Settings Related To Docker Support in Storm
+
+| Setting | Description [...]
+|-------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| `storm.resource.isolation.plugin.enable` | set to `true` to enable isolation plugin. `storm.resource.isolation.plugin` determines which plugin to use. If this is set to `false`, `org.apache.storm.container.DefaultResourceIsolationManager` will be used. |
+| `storm.resource.isolation.plugin` | set to `"org.apache.storm.container.docker.DockerManager"` to enable docker support [...]
+| `storm.oci.allowed.images` | An allowlist of docker images that can be used. Users can only choose a docker image from the list.
+| `storm.oci.image` | The default docker image to be used if user doesn't specify which image to use. And it must belong to the `storm.oci.allowed.images`
+| `topology.oci.image` | Topologies can specify which image to use. It must belong to the `storm.oci.allowed.images` |
+| `storm.oci.cgroup.root` | The root path of cgroup for docker to use. On RHEL7, it should be "/sys/fs/cgroup".
+| `storm.oci.cgroup.parent` | --cgroup-parent config for docker command. It must follow the constraints of docker commands. The path will be made as absolute path if it's a relative path because we saw some weird bugs ((the cgroup memory directory disappears after a while) when a relative path is used.
+| `storm.oci.readonly.bindmounts` | A list of read only bind mounted directories.
+| `storm.oci.readwrite.bindmounts` | A list of read write bind mounted directories.
+| `storm.oci.nscd.dir` | The directory of nscd (name service cache daemon), e.g. "/var/run/nscd/". nscd must be running so that profiling can work properly.
+| `storm.oci.seccomp.profile` | Specify the seccomp Json file to be used as a seccomp filter
+| `supervisor.worker.launcher` | Full path to the worker-launcher executable. Details explained at [How to set up worker-launcher](#how-to-set-up-worker-launcher)
+
+Note that we only support cgroupfs cgroup driver because of some issues with `systemd` cgroup driver; restricting to `cgroupfs` also makes cgroup paths simpler. Please make sure to use `cgroupfs` before setting up docker support.
+
+#### Example
+
+Below is a simple configuration example for storm on Rhel7. In this example, storm is deployed at `/usr/share/apache-storm-2.3.0`.
+
+```bash
+storm.resource.isolation.plugin.enable: true
+storm.resource.isolation.plugin: "org.apache.storm.container.docker.DockerManager"
+storm.oci.allowed.images: ["xxx.xxx.com:8080/storm/storm/rhel7:latest"]
+storm.oci.image: "xxx.xxx.com:8080/storm/storm/rhel7:latest"
+storm.oci.cgroup.root: "/storm"
+storm.oci.cgroup.parent: "/sys/fs/cgroup"
+storm.oci.readonly.bindmounts:
+ - "/etc/storm"
+storm.oci.nscd.dir: "/var/run/nscd"
+supervisor.worker.launcher: "/usr/share/apache-storm-2.3.0/bin/worker-launcher"
+```
+
+### How to set up worker-launcher
+
+The `worker-launcher` executable is a special program that is used to launch docker containers, run `docker` and `nsenter` commands.
+For this to work, `worker-launcher` needs to be owned by root, but with the group set to be a group that only the supervisor headless user is a part of.
+`worker-launcher` also needs to have `6550` octal permissions. There is also a `worker-launcher.cfg` file, usually located under `/etc/storm`, that should look something like the following:
+```
+storm.worker-launcher.group=$(worker_launcher_group)
+min.user.id=$(min_user_id)
+worker.profiler.script.path=$(profiler_script_path)
+```
+where `storm.worker-launcher.group` is the same group the supervisor user is a part of, and `min.user.id` is set to the first real user id on the system. This config file also needs to be owned by root and not have world nor group write permissions.
+`worker.profiler.script.path` points to the profiler script. For security, the script should be only writable by root. Note that it's the only profiler script that will be used and `DaemonConfig.WORKER_PROFILER_COMMAND` will be ignored.
+
+There are two optional configs that will be used by docker support: `docker.binary` and `nsenter.binary`. By default, they are set to
+```
+docker.binary=/usr/bin/docker
+nsenter.binary=/usr/bin/nsenter
+```
+and you don't need to set them in the worker-launcher.cfg unless you need to change them.
+
+## Profile the processes inside the container
+You can profile your worker processes by clicking on the profiling buttons (jstack, heap, etc) on storm UI.
+If you have sudo permission, you can also run `sudo nsenter --target <container-pid> --pid --mount --setuid <uid> --setgid <gid>` to enter the container.
+Then you can run `jstack`, `jmap` etc inside the container. `<container-pid>` is the pid of the container process on the host.
+`<container-pid>` can be obtained by running `sudo docker inspect --format '{% raw %}{{.State.Pid}}{% endraw %}' <container-id>` command.
+`<uid>` and `<gid>` are the user id and group id of the container owner, respectively.
+
+## Seccomp security profiles
+
+You can set `storm.oci.seccomp.profile` to restrict the actions available within the container. If it's not set, the [default docker seccomp profile](https://github.com/moby/moby/blob/master/profiles/seccomp/default.json)
+is used. You can use `conf/seccomp.json.example` provided or you can specify our own `seccomp.json` file.
+
+## CGroup Metrics
+
+Docker internally uses cgroups to control resources for containers. The CGroup Metrics described at [cgroups_in_storm.md](cgroups_in_storm.md#CGroup-Metrics) still apply except CGroupCpuGuarantee. To get CGroup cpu guarantee, use CGroupCpuGuaranteeByCfsQuota instead.
\ No newline at end of file
diff --git a/releases/2.8.8/Eventlogging.md b/releases/2.8.8/Eventlogging.md
new file mode 100644
index 0000000000..0502f57857
--- /dev/null
+++ b/releases/2.8.8/Eventlogging.md
@@ -0,0 +1,125 @@
+---
+title: Topology event inspector
+layout: documentation
+documentation: true
+---
+
+# Introduction
+
+Topology event inspector provides the ability to view the tuples as it flows through different stages in a storm topology.
+This could be useful for inspecting the tuples emitted at a spout or a bolt in the topology pipeline while the topology is running, without stopping or redeploying the topology. The normal flow of tuples from the spouts to the bolts is not affected by turning on event logging.
+
+## Enabling event logging
+
+Note: Event logging needs to be enabled first by setting the storm config "topology.eventlogger.executors" to a non zero value. Please see
+the [Configuration](#config) section for more details.
+
+Events can be logged by clicking the "Debug" button under the topology actions in the topology view. This logs the
+tuples from all the spouts and bolts in a topology at the specified sampling percentage.
+
+<div align="center">
+<img title="Enable Eventlogging" src="images/enable-event-logging-topology.png" style="max-width: 80rem"/>
+
+<p>Figure 1: Enable event logging at topology level.</p>
+</div>
+
+You could also enable event logging at a specific spout or bolt level by going to the corresponding component page and
+clicking "Debug" under component actions.
+
+<div align="center">
+<img title="Enable Eventlogging at component level" src="images/enable-event-logging-spout.png" style="max-width: 80rem"/>
+
+<p>Figure 2: Enable event logging at component level.</p>
+</div>
+
+## Viewing the event logs
+The Storm "logviewer" should be running for viewing the logged tuples. If not already running log viewer can be started by running the "bin/storm logviewer" command from the storm installation directory. For viewing the tuples, go to the specific spout or bolt component page from storm UI and click on the "events" link under the component summary (as highlighted in Figure 2 above).
+
+This would open up a view like below where you can navigate between different pages and view the logged tuples.
+
+<div align="center">
+<img title="Viewing logged tuples" src="images/event-logs-view.png" style="max-width: 80rem"/>
+
+<p>Figure 3: Viewing the logged events.</p>
+</div>
+
+Each line in the event log contains an entry corresponding to a tuple emitted from a specific spout/bolt in a comma separated format.
+
+`Timestamp, Component name, Component task-id, MessageId (in case of anchoring), List of emitted values`
+
+## Disabling the event logs
+
+Event logging can be disabled at a specific component or at the topology level by clicking the "Stop Debug" under the topology or component actions in the Storm UI.
+
+<div align="center">
+<img title="Disable Eventlogging at topology level" src="images/disable-event-logging-topology.png" style="max-width: 80rem"/>
+
+<p>Figure 4: Disable event logging at topology level.</p>
+</div>
+
+## <a name="config"></a>Configuration
+Eventlogging works by sending the events (tuples) from each component to an internal eventlogger bolt. By default Storm does not start any event logger tasks, but this can be easily changed by setting the below parameter while running your topology (by setting it in storm.yaml or passing options via command line).
+
+| Parameter | Meaning |
+| -------------------------------------------|-----------------------|
+| "topology.eventlogger.executors": 0 | No event logger tasks are created (default). |
+| "topology.eventlogger.executors": 1 | One event logger task for the topology. |
+| "topology.eventlogger.executors": nil | One event logger task per worker. |
+
+
+## Extending eventlogging
+
+Storm provides an `IEventLogger` interface which is used by the event logger bolt to log the events.
+
+```java
+/**
+ * EventLogger interface for logging the event info to a sink like log file or db
+ * for inspecting the events via UI for debugging.
+ */
+public interface IEventLogger {
+ /**
+ * Invoked during eventlogger bolt prepare.
+ */
+ void prepare(Map stormConf, Map<String, Object> arguments, TopologyContext context);
+
+ /**
+ * Invoked when the {@link EventLoggerBolt} receives a tuple from the spouts or bolts that has event logging enabled.
+ *
+ * @param e the event
+ */
+ void log(EventInfo e);
+
+ /**
+ * Invoked when the event logger bolt is cleaned up
+ */
+ void close();
+}
+```
+
+The default implementation for this is a FileBasedEventLogger which logs the events to an events.log file ( `logs/workers-artifacts/<topology-id>/<worker-port>/events.log`).
+Alternate implementations of the `IEventLogger` interface can be added to extend the event logging functionality (say build a search index or log the events in a database etc)
+
+If you just want to use FileBasedEventLogger but with changing the log format, you can simply implement your own by extending FileBasedEventLogger and override `buildLogMessage(EventInfo)` to provide log line explicitly.
+
+To register event logger to your topology, add to your topology's configuration like:
+
+```java
+conf.registerEventLogger(org.apache.storm.metric.FileBasedEventLogger.class);
+```
+
+You can refer [Config#registerEventLogger](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#registerEventLogger-java.lang.Class-) and overloaded methods from javadoc.
+
+Otherwise edit the storm.yaml config file:
+
+```yaml
+topology.event.logger.register:
+ - class: "org.apache.storm.metric.FileBasedEventLogger"
+ - class: "org.mycompany.MyEventLogger"
+ arguments:
+ endpoint: "event-logger.mycompany.org"
+```
+
+When you implement your own event logger, `arguments` is passed to Map<String, Object> when [IEventLogger#prepare](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/metric/IEventLogger.html#prepare-java.util.Map-java.lang.Map-org.apache.storm.task.TopologyContext-) is called.
+
+Please keep in mind that EventLoggerBolt is just a kind of Bolt, so whole throughput of the topology will go down when registered event loggers cannot keep up handling incoming events, so you may want to take care of the Bolt like normal Bolt.
+One of idea to avoid this is making your implementation of IEventLogger as `non-blocking` fashion.
diff --git a/releases/2.8.8/FAQ.md b/releases/2.8.8/FAQ.md
new file mode 100644
index 0000000000..f59d45fad1
--- /dev/null
+++ b/releases/2.8.8/FAQ.md
@@ -0,0 +1,127 @@
+---
+title: FAQ
+layout: documentation
+documentation: true
+---
+
+## Best Practices
+
+### What rules of thumb can you give me for configuring Storm+Trident?
+
+* number of workers a multiple of number of machines; parallelism a multiple of number of workers; number of kafka partitions a multiple of number of spout parallelism
+* Use one worker per topology per machine
+* Start with fewer, larger aggregators, one per machine with workers on it
+* Use the isolation scheduler
+* Use one acker per worker -- 0.9 makes that the default, but earlier versions do not.
+* enable GC logging; you should see very few major GCs if things are in reasonable shape.
+* set the trident batch millis to about 50% of your typical end-to-end latency.
+* Start with a max spout pending that is for sure too small -- one for trident, or the number of executors for storm -- and increase it until you stop seeing changes in the flow. You'll probably end up with something near `2*(throughput in recs/sec)*(end-to-end latency)` (2x the Little's law capacity).
+
+### What are some of the best ways to get a worker to mysteriously and bafflingly die?
+
+* Do you have write access to the log directory
+* Are you blowing out your heap?
+* Are all the right libraries installed on all of the workers?
+* Is the zookeeper hostname still set to localhost?
+* Did you supply a correct, unique hostname -- one that resolves back to the machine -- to each worker, and put it in the storm conf file?
+* Have you opened firewall/securitygroup permissions _bidirectionally_ among a) all the workers, b) the storm master, c) zookeeper? Also, from the workers to any kafka/kestrel/database/etc that your topology accesses? Use netcat to poke the appropriate ports and be sure.
+
+### Halp! I cannot see:
+
+* **my logs** Logs by default go to $STORM_HOME/logs. Check that you have write permissions to that directory. They are configured in log4j2/{cluster, worker}.xml.
+* **final JVM settings** Add the `-XX+PrintFlagsFinal` commandline option in the childopts (see the conf file)
+* **final Java system properties** Add `Properties props = System.getProperties(); props.list(System.out);` near where you build your topology.
+
+### How many Workers should I use?
+
+The total number of workers is set by the supervisors -- there's some number of JVM slots each supervisor will superintend. The thing you set on the topology is how many worker slots it will try to claim.
+
+There's no great reason to use more than one worker per topology per machine.
+
+With one topology running on three 8-core nodes, and parallelism hint 24, each bolt gets 8 executors per machine, i.e. one for each core. There are three big benefits to running three workers (with 8 assigned executors each) compare to running say 24 workers (one assigned executor each).
+
+First, data that is repartitioned (shuffles or group-bys) to executors in the same worker will not have to hit the transfer buffer. Instead, tuples are deposited directly from send to receive buffer. That's a big win. By contrast, if the destination executor were on the same machine in a different worker, it would have to go send -> worker transfer -> local socket -> worker recv -> exec recv buffer. It doesn't hit the network card, but it's not as big a win as when executors are in the s [...]
+
+Second, you're typically better off with three aggregators having very large backing cache than having twenty-four aggregators having small backing caches. This reduces the effect of skew, and improves LRU efficiency.
+
+Lastly, fewer workers reduces control flow chatter.
+
+## Topology
+
+### Can a Trident topology have Multiple Streams?
+
+> Can a Trident Topology work like a workflow with conditional paths (if-else)? e.g. A Spout (S1) connects to a bolt (B0) which based on certain values in the incoming tuple routes them to either bolt (B1) or bolt (B2) but not both.
+
+A Trident "each" operator returns a Stream object, which you can store in a variable. You can then run multiple eaches on the same Stream to split it, e.g.:
+
+ Stream s = topology.each(...).groupBy(...).aggregate(...)
+ Stream branch1 = s.each(..., FilterA)
+ Stream branch2 = s.each(..., FilterB)
+
+You can join streams with join, merge or multiReduce.
+
+At time of writing, you can't emit to multiple output streams from Trident -- see [STORM-68](https://issues.apache.org/jira/browse/STORM-68)
+
+### Why am I getting a NotSerializableException/IllegalStateException when my topology is being started up?
+
+Within the Storm lifecycle, the topology is instantiated and then serialized to byte format to be stored in ZooKeeper, prior to the topology being executed. Within this step, if a spout or bolt within the topology has an initialized unserializable property, serialization will fail. If there is a need for a field that is unserializable, initialize it within the bolt's `prepare` or spout's `open` method, which is run after the topology is delivered to the worker.
+
+## Spouts
+
+### What is a coordinator, and why are there several?
+
+A trident-spout is actually run within a storm _bolt_. The storm-spout of a trident topology is the MasterBatchCoordinator -- it coordinates trident batches and is the same no matter what spouts you use. A batch is born when the MBC dispenses a seed tuple to each of the spout-coordinators. The spout-coordinator bolts know how your particular spouts should cooperate -- so in the kafka case, it's what helps figure out what partition and offset range each spout should pull from.
+
+### What can I store into the spout's metadata record?
+
+You should only store static data, and as little of it as possible, into the metadata record (note: maybe you _can_ store more interesting things; you shouldn't, though)
+
+### How often is the 'emitBatchNew' function called?
+
+Since the MBC is the actual spout, all the tuples in a batch are just members of its tupletree. That means storm's "max spout pending" config effectively defines the number of concurrent batches trident runs. The MBC emits a new batch if it has fewer than max-spending tuples pending and if at least one [trident batch interval]({{page.git-blob-base}}/conf/defaults.yaml#L115)'s worth of seconds has passed since the last batch.
+
+### If nothing was emitted does Trident slow down the calls?
+
+Yes, there's a pluggable "spout wait strategy"; the default is to sleep for a [configurable amount of time]({{page.git-blob-base}}/conf/defaults.yaml#L110)
+
+### OK, then what is the trident batch interval for?
+
+You know how computers of the 486 era had a [turbo button](http://en.wikipedia.org/wiki/Turbo_button) on them? It's like that.
+
+Actually, it has two practical uses. One is to throttle spouts that poll a remote source without throttling processing. For example, we have a spout that looks in a given S3 bucket for a new batch-uploaded file to read, linebreak and emit. We don't want it hitting S3 more than every few seconds: files don't show up more than once every few minutes, and a batch takes a few seconds to process.
+
+The other is to limit overpressure on the internal queues during startup or under a heavy burst load -- if the spouts spring to life and suddenly jam ten batches' worth of records into the system, you could have a mass of less-urgent tuples from batch 7 clog up the transfer buffer and prevent the $commit tuple from batch 3 to get through (or even just the regular old tuples from batch 3). What we do is set the trident batch interval to about half the typical end-to-end processing latency [...]
+
+Note that this is a cap, not an additional delay -- with a period of 300ms, if your batch takes 258ms Trident will only delay an additional 42ms.
+
+### How do you set the batch size?
+
+Trident doesn't place its own limits on the batch count. In the case of the Kafka spout, the max fetch bytes size divided by the average record size defines an effective records per subbatch partition.
+
+### How do I resize a batch?
+
+The trident batch is a somewhat overloaded facility. Together with the number of partitions, the batch size is constrained by or serves to define
+
+1. the unit of transactional safety (tuples at risk vs time)
+2. per partition, an effective windowing mechanism for windowed stream analytics
+3. per partition, the number of simultaneous queries that will be made by a partitionQuery, partitionPersist, etc;
+4. per partition, the number of records convenient for the spout to dispatch at the same time;
+
+You can't change the overall batch size once generated, but you can change the number of partitions -- do a shuffle and then change the parallelism hint
+
+## Time Series
+
+### How do I aggregate events by time?
+
+If you have records with an immutable timestamp, and you would like to count, average or otherwise aggregate them into discrete time buckets, Trident is an excellent and scalable solution.
+
+Write an `Each` function that turns the timestamp into a time bucket: if the bucket size was "by hour", then the timestamp `2013-08-08 12:34:56` would be mapped to the `2013-08-08 12:00:00` time bucket, and so would everything else in the twelve o'clock hour. Then group on that timebucket and use a grouped persistentAggregate. The persistentAggregate uses a local cacheMap backed by a data store. Groups with many records require very few reads from the data store, and use efficient bulk r [...]
+
+### How can I know that all records for a time bucket have been received?
+
+You cannot know that all events are collected -- this is an epistemological challenge, not a distributed systems challenge. You can:
+
+* Set a time limit using domain knowledge
+* Introduce a _punctuation_: a record known to come after all records in the given time bucket. Trident uses this scheme to know when a batch is complete. If you for instance receive records from a set of sensors, each in order for that sensor, then once all sensors have sent you a 3:02:xx or later timestamp lets you know you can commit.
+* When possible, make your process incremental: each value that comes in makes the answer more and more true. A Trident ReducerAggregator is an operator that takes a prior result and a set of new records and returns a new result. This lets the result be cached and serialized to a datastore; if a server drops off line for a day and then comes back with a full day's worth of data in a rush, the old results will be calmly retrieved and updated.
+* Lambda architecture: Record all events into an archival store (S3, HBase, HDFS) on receipt. in the fast layer, once the time window is clear, process the bucket to get an actionable answer, and ignore everything older than the time window. Periodically run a global aggregation to calculate a "correct" answer.
diff --git a/releases/2.8.8/Fault-tolerance.md b/releases/2.8.8/Fault-tolerance.md
new file mode 100644
index 0000000000..9a7a349f5b
--- /dev/null
+++ b/releases/2.8.8/Fault-tolerance.md
@@ -0,0 +1,28 @@
+---
+layout: documentation
+---
+This page explains the design details of Storm that make it a fault-tolerant system.
+
+## What happens when a worker dies?
+
+When a worker dies, the supervisor will restart it. If it continuously fails on startup and is unable to heartbeat to Nimbus, Nimbus will reassign the worker to another machine.
+
+## What happens when a node dies?
+
+The tasks assigned to that machine will time-out and Nimbus will reassign those tasks to other machines.
+
+## What happens when Nimbus or Supervisor daemons die?
+
+The Nimbus and Supervisor daemons are designed to be fail-fast (process self-destructs whenever any unexpected situation is encountered) and stateless (all state is kept in Zookeeper or on disk). As described in [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html), the Nimbus and Supervisor daemons must be run under supervision using a tool like daemontools or monit. So if the Nimbus or Supervisor daemons die, they restart like nothing happened.
+
+Most notably, no worker processes are affected by the death of Nimbus or the Supervisors. This is in contrast to Hadoop, where if the JobTracker dies, all the running jobs are lost.
+
+## Is Nimbus a single point of failure?
+
+If you lose the Nimbus node, the workers will still continue to function. Additionally, supervisors will continue to restart workers if they die. However, without Nimbus, workers won't be reassigned to other machines when necessary (like if you lose a worker machine).
+
+So the answer is that Nimbus is "sort of" a SPOF. In practice, it's not a big deal since nothing catastrophic happens when the Nimbus daemon dies. There are plans to make Nimbus highly available in the future.
+
+## How does Storm guarantee data processing?
+
+Storm provides mechanisms to guarantee data processing even if nodes die or messages are lost. See [Guaranteeing message processing](Guaranteeing-message-processing.html) for the details.
diff --git a/releases/2.8.8/Generic-resources.md b/releases/2.8.8/Generic-resources.md
new file mode 100644
index 0000000000..f3bfe3e98c
--- /dev/null
+++ b/releases/2.8.8/Generic-resources.md
@@ -0,0 +1,39 @@
+---
+title: Generic Resources
+layout: documentation
+documentation: true
+---
+
+### Generic Resources
+Generic Resources allow Storm to reference arbitrary resource types. Generic Resources may be considered an extension of the resources enumerated by the [Resource Aware Scheduler](Resource_Aware_Scheduler_overview.html), which accounts for CPU and memory.
+
+### API Overview
+For a Storm Topology, the user can now specify the amount of generic resources a topology component (i.e. Spout or Bolt) is required to run a single instance of the component. The user can specify the resource requirement for a topology component by using the following API call.
+```
+ public T addResource(String resourceName, Number resourceValue)
+```
+Parameters:
+- resourceName – The name of the generic resource
+- resourceValue – The amount of the generic resource
+
+Example of Usage:
+```
+ SpoutDeclarer s1 = builder.setSpout("word", new TestWordSpout(), 10);
+ s1.addResouce("gpu.count", 1.0);
+```
+
+### Specifying Generic Cluster Resources
+
+A storm administrator can specify node resource availability by modifying the _conf/storm.yaml_ file located in the storm home directory of that node.
+```
+ supervisor.resources.map: {[type<String>] : [amount<Double>]}
+```
+Example of Usage:
+```
+ supervisor.resources.map: {"gpu.count" : 2.0}
+```
+
+
+### Generic Resources in UI
+
+
diff --git a/releases/2.8.8/Guaranteeing-message-processing.md b/releases/2.8.8/Guaranteeing-message-processing.md
new file mode 100644
index 0000000000..1cdc900de2
--- /dev/null
+++ b/releases/2.8.8/Guaranteeing-message-processing.md
@@ -0,0 +1,181 @@
+---
+title: Guaranteeing Message Processing
+layout: documentation
+documentation: true
+---
+Storm offers several different levels of guaranteed message processing, including best effort, at least once, and exactly once through [Trident](Trident-tutorial.html).
+This page describes how Storm can guarantee at least once processing.
+
+### What does it mean for a message to be "fully processed"?
+
+A tuple coming off a spout can trigger thousands of tuples to be created based on it. Consider, for example, the streaming word count topology:
+
+```java
+TopologyBuilder builder = new TopologyBuilder();
+builder.setSpout("sentences", new KestrelSpout("kestrel.backtype.com",
+ 22133,
+ "sentence_queue",
+ new StringScheme()));
+builder.setBolt("split", new SplitSentence(), 10)
+ .shuffleGrouping("sentences");
+builder.setBolt("count", new WordCount(), 20)
+ .fieldsGrouping("split", new Fields("word"));
+```
+
+This topology reads sentences off of a Kestrel queue, splits the sentences into its constituent words, and then emits for each word the number of times it has seen that word before. A tuple coming off the spout triggers many tuples being created based on it: a tuple for each word in the sentence and a tuple for the updated count for each word. The tree of messages looks something like this:
+
+
+
+Storm considers a tuple coming off a spout "fully processed" when the tuple tree has been exhausted and every message in the tree has been processed. A tuple is considered failed when its tree of messages fails to be fully processed within a specified timeout. This timeout can be configured on a topology-specific basis using the [Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#TOPOLOGY_MESSAGE_TIMEOUT_SECS) con [...]
+
+### What happens if a message is fully processed or fails to be fully processed?
+
+To understand this question, let's take a look at the lifecycle of a tuple coming off of a spout. For reference, here is the interface that spouts implement (see the [Javadoc](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/spout/ISpout.html) for more information):
+
+```java
+public interface ISpout extends Serializable {
+ void open(Map conf, TopologyContext context, SpoutOutputCollector collector);
+ void close();
+ void nextTuple();
+ void ack(Object msgId);
+ void fail(Object msgId);
+}
+```
+
+First, Storm requests a tuple from the `Spout` by calling the `nextTuple` method on the `Spout`. The `Spout` uses the `SpoutOutputCollector` provided in the `open` method to emit a tuple to one of its output streams. When emitting a tuple, the `Spout` provides a "message id" that will be used to identify the tuple later. For example, the `KestrelSpout` reads a message off of the kestrel queue and emits as the "message id" the id provided by Kestrel for the message. Emitting a message to [...]
+
+```java
+_collector.emit(new Values("field1", "field2", 3) , msgId);
+```
+
+Next, the tuple gets sent to consuming bolts and Storm takes care of tracking the tree of messages that is created. If Storm detects that a tuple is fully processed, Storm will call the `ack` method on the originating `Spout` task with the message id that the `Spout` provided to Storm. Likewise, if the tuple times-out Storm will call the `fail` method on the `Spout`. Note that a tuple will be acked or failed by the exact same `Spout` task that created it. So if a `Spout` is executing as [...]
+
+Let's use `KestrelSpout` again to see what a `Spout` needs to do to guarantee message processing. When `KestrelSpout` takes a message off the Kestrel queue, it "opens" the message. This means the message is not actually taken off the queue yet, but instead placed in a "pending" state waiting for acknowledgement that the message is completed. While in the pending state, a message will not be sent to other consumers of the queue. Additionally, if a client disconnects all pending messages f [...]
+
+### What is Storm's reliability API?
+
+There are two things you have to do as a user to benefit from Storm's reliability capabilities. First, you need to tell Storm whenever you're creating a new link in the tree of tuples. Second, you need to tell Storm when you have finished processing an individual tuple. By doing both these things, Storm can detect when the tree of tuples is fully processed and can ack or fail the spout tuple appropriately. Storm's API provides a concise way of doing both of these tasks.
+
+Specifying a link in the tuple tree is called _anchoring_. Anchoring is done at the same time you emit a new tuple. Let's use the following bolt as an example. This bolt splits a tuple containing a sentence into a tuple for each word:
+
+```java
+public class SplitSentence extends BaseRichBolt {
+ OutputCollector _collector;
+
+ public void prepare(Map conf, TopologyContext context, OutputCollector collector) {
+ _collector = collector;
+ }
+
+ public void execute(Tuple tuple) {
+ String sentence = tuple.getString(0);
+ for(String word: sentence.split(" ")) {
+ _collector.emit(tuple, new Values(word));
+ }
+ _collector.ack(tuple);
+ }
+
+ public void declareOutputFields(OutputFieldsDeclarer declarer) {
+ declarer.declare(new Fields("word"));
+ }
+ }
+```
+
+Each word tuple is _anchored_ by specifying the input tuple as the first argument to `emit`. Since the word tuple is anchored, the spout tuple at the root of the tree will be replayed later on if the word tuple failed to be processed downstream. In contrast, let's look at what happens if the word tuple is emitted like this:
+
+```java
+_collector.emit(new Values(word));
+```
+
+Emitting the word tuple this way causes it to be _unanchored_. If the tuple fails be processed downstream, the root tuple will not be replayed. Depending on the fault-tolerance guarantees you need in your topology, sometimes it's appropriate to emit an unanchored tuple.
+
+An output tuple can be anchored to more than one input tuple. This is useful when doing streaming joins or aggregations. A multi-anchored tuple failing to be processed will cause multiple tuples to be replayed from the spouts. Multi-anchoring is done by specifying a list of tuples rather than just a single tuple. For example:
+
+```java
+List<Tuple> anchors = new ArrayList<Tuple>();
+anchors.add(tuple1);
+anchors.add(tuple2);
+_collector.emit(anchors, new Values(1, 2, 3));
+```
+
+Multi-anchoring adds the output tuple into multiple tuple trees. Note that it's also possible for multi-anchoring to break the tree structure and create tuple DAGs, like so:
+
+
+
+Storm's implementation works for DAGs as well as trees (pre-release it only worked for trees, and the name "tuple tree" stuck).
+
+Anchoring is how you specify the tuple tree -- the next and final piece to Storm's reliability API is specifying when you've finished processing an individual tuple in the tuple tree. This is done by using the `ack` and `fail` methods on the `OutputCollector`. If you look back at the `SplitSentence` example, you can see that the input tuple is acked after all the word tuples are emitted.
+
+You can use the `fail` method on the `OutputCollector` to immediately fail the spout tuple at the root of the tuple tree. For example, your application may choose to catch an exception from a database client and explicitly fail the input tuple. By failing the tuple explicitly, the spout tuple can be replayed faster than if you waited for the tuple to time-out.
+
+Every tuple you process must be acked or failed. Storm uses memory to track each tuple, so if you don't ack/fail every tuple, the task will eventually run out of memory.
+
+A lot of bolts follow a common pattern of reading an input tuple, emitting tuples based on it, and then acking the tuple at the end of the `execute` method. These bolts fall into the categories of filters and simple functions. Storm has an interface called `BasicBolt` that encapsulates this pattern for you. The `SplitSentence` example can be written as a `BasicBolt` like follows:
+
+```java
+public class SplitSentence extends BaseBasicBolt {
+ public void execute(Tuple tuple, BasicOutputCollector collector) {
+ String sentence = tuple.getString(0);
+ for(String word: sentence.split(" ")) {
+ collector.emit(new Values(word));
+ }
+ }
+
+ public void declareOutputFields(OutputFieldsDeclarer declarer) {
+ declarer.declare(new Fields("word"));
+ }
+ }
+```
+
+This implementation is simpler than the implementation from before and is semantically identical. Tuples emitted to `BasicOutputCollector` are automatically anchored to the input tuple, and the input tuple is acked for you automatically when the execute method completes.
+
+In contrast, bolts that do aggregations or joins may delay acking a tuple until after it has computed a result based on a bunch of tuples. Aggregations and joins will commonly multi-anchor their output tuples as well. These things fall outside the simpler pattern of `IBasicBolt`.
+
+### How do I make my applications work correctly given that tuples can be replayed?
+
+As always in software design, the answer is "it depends." If you really want exactly once semantics use the [Trident](Trident-tutorial.html) API. In some cases, like with a lot of analytics, dropping data is OK so disabling the fault tolerance by setting the number of acker bolts to 0 [Config.TOPOLOGY_ACKERS](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#TOPOLOGY_ACKERS). But in some cases you want to be sure that everything was processed at lea [...]
+
+### How does Storm implement reliability in an efficient way?
+
+A Storm topology has a set of special "acker" tasks that track the DAG of tuples for every spout tuple. When an acker sees that a DAG is complete, it sends a message to the spout task that created the spout tuple to ack the message. You can set the number of acker tasks for a topology in the topology configuration using [Config.TOPOLOGY_ACKERS](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#TOPOLOGY_ACKERS). Storm defaults TOPOLOGY_ACKERS to one t [...]
+
+The best way to understand Storm's reliability implementation is to look at the lifecycle of tuples and tuple DAGs. When a tuple is created in a topology, whether in a spout or a bolt, it is given a random 64 bit id. These ids are used by ackers to track the tuple DAG for every spout tuple.
+
+Every tuple knows the ids of all the spout tuples for which it exists in their tuple trees. When you emit a new tuple in a bolt, the spout tuple ids from the tuple's anchors are copied into the new tuple. When a tuple is acked, it sends a message to the appropriate acker tasks with information about how the tuple tree changed. In particular it tells the acker "I am now completed within the tree for this spout tuple, and here are the new tuples in the tree that were anchored to me".
+
+For example, if tuples "D" and "E" were created based on tuple "C", here's how the tuple tree changes when "C" is acked:
+
+
+
+Since "C" is removed from the tree at the same time that "D" and "E" are added to it, the tree can never be prematurely completed.
+
+There are a few more details to how Storm tracks tuple trees. As mentioned already, you can have an arbitrary number of acker tasks in a topology. This leads to the following question: when a tuple is acked in the topology, how does it know to which acker task to send that information?
+
+Storm uses mod hashing to map a spout tuple id to an acker task. Since every tuple carries with it the spout tuple ids of all the trees they exist within, they know which acker tasks to communicate with.
+
+Another detail of Storm is how the acker tasks track which spout tasks are responsible for each spout tuple they're tracking. When a spout task emits a new tuple, it simply sends a message to the appropriate acker telling it that its task id is responsible for that spout tuple. Then when an acker sees a tree has been completed, it knows to which task id to send the completion message.
+
+Acker tasks do not track the tree of tuples explicitly. For large tuple trees with tens of thousands of nodes (or more), tracking all the tuple trees could overwhelm the memory used by the ackers. Instead, the ackers take a different strategy that only requires a fixed amount of space per spout tuple (about 20 bytes). This tracking algorithm is the key to how Storm works and is one of its major breakthroughs.
+
+An acker task stores a map from a spout tuple id to a pair of values. The first value is the task id that created the spout tuple which is used later on to send completion messages. The second value is a 64 bit number called the "ack val". The ack val is a representation of the state of the entire tuple tree, no matter how big or how small. It is simply the xor of all tuple ids that have been created and/or acked in the tree.
+
+When an acker task sees that an "ack val" has become 0, then it knows that the tuple tree is completed. Since tuple ids are random 64 bit numbers, the chances of an "ack val" accidentally becoming 0 is extremely small. If you work the math, at 10K acks per second, it will take 50,000,000 years until a mistake is made. And even then, it will only cause data loss if that tuple happens to fail in the topology.
+
+Now that you understand the reliability algorithm, let's go over all the failure cases and see how in each case Storm avoids data loss:
+
+- **A tuple isn't acked because the task died**: In this case the spout tuple ids at the root of the trees for the failed tuple will time out and be replayed.
+- **Acker task dies**: In this case all the spout tuples the acker was tracking will time out and be replayed.
+- **Spout task dies**: In this case the source that the spout talks to is responsible for replaying the messages. For example, queues like Kestrel and RabbitMQ will place all pending messages back on the queue when a client disconnects.
+
+As you have seen, Storm's reliability mechanisms are completely distributed, scalable, and fault-tolerant.
+
+### Tuning reliability
+
+Acker tasks are lightweight, so you don't need very many of them in a topology. You can track their performance through the Storm UI (component id "__acker"). If the throughput doesn't look right, you'll need to add more acker tasks.
+
+If reliability isn't important to you -- that is, you don't care about losing tuples in failure situations -- then you can improve performance by not tracking the tuple tree for spout tuples. Not tracking a tuple tree halves the number of messages transferred since normally there's an ack message for every tuple in the tuple tree. Additionally, it requires fewer ids to be kept in each downstream tuple, reducing bandwidth usage.
+
+There are three ways to remove reliability. The first is to set Config.TOPOLOGY_ACKERS to 0. In this case, Storm will call the `ack` method on the spout immediately after the spout emits a tuple. The tuple tree won't be tracked.
+
+The second way is to remove reliability on a message by message basis. You can turn off tracking for an individual spout tuple by omitting a message id in the `SpoutOutputCollector.emit` method.
+
+Finally, if you don't care if a particular subset of the tuples downstream in the topology fail to be processed, you can emit them as unanchored tuples. Since they're not anchored to any spout tuples, they won't cause any spout tuples to fail if they aren't acked.
diff --git a/releases/2.8.8/Hooks.md b/releases/2.8.8/Hooks.md
new file mode 100644
index 0000000000..2267b95299
--- /dev/null
+++ b/releases/2.8.8/Hooks.md
@@ -0,0 +1,26 @@
+---
+title: Hooks
+layout: documentation
+documentation: true
+---
+## Task hooks
+Storm provides hooks with which you can insert custom code to run on any number of events within Storm. You create a hook by extending the [BaseTaskHook](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/hooks/BaseTaskHook.html) class and overriding the appropriate method for the event you want to catch. There are two ways to register your hook:
+
+1. In the open method of your spout or prepare method of your bolt using the [TopologyContext](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/TopologyContext.html#addTaskHook) method.
+2. Through the Storm configuration using the ["topology.auto.task.hooks"](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#TOPOLOGY_AUTO_TASK_HOOKS) config. These hooks are automatically registered in every spout or bolt, and are useful for doing things like integrating with a custom monitoring system.
+
+## Worker hooks
+Storm also provides worker-level hooks that are called during worker startup, before any bolts or spouts are prepared/opened. You can create such a hook by extending [BaseWorkerHook](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/hooks/BaseWorkerHook) (an implementation of [IWorkerHook](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/hooks/IWorkerHook.html)) and overriding the methods you want to implement. You can register your ho [...]
+The `IWorkerHook#start(Map, WorkerUserContext)` lifecycle method exposes [WorkerUserContext](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/hooks/IWorkerHook.html) which provides a way to set application-level common resources via `setResource(String, Object)` method. This resource can then be retrieved by tasks, both spouts (via `open(Map, TopologyContext, SpoutOutputCollector`) and bolts (via `prepare(Map, TopologyContext, OutputCollector`), by calling `Top [...]
+
+## Shared State amongst components and hooks
+Storm provides ways to share resources across different components via the following ways:
+1. taskData: this pertains to the task level data and can be written and read by task and task hooks in their corresponding lifecycle methods (`open` for spout and `prepare` for bolt and task hook).
+ 1. write access: `TopologyContext#setTaskData(String, Object)`
+ 2. read access: `TopologyContext#getTask(String)`
+2. executorData: this pertains to executor level data and is shared across tasks and task hooks which are managed by the concerned executor. Similar to above it is accessible to spouts via `open` and to bolts and task hooks via `prepare` lifecycle method.
+ 1. write access: `TopologyContext#setExecutorData`
+ 2. read access: `TopologyContext#getExecutorData(String)`
+3. userResources: this pertains to worker level data and is shared across executors, tasks, worker hooks and task hooks which are managed by the concerned worker. Unlike others it can only be written by worker hooks.
+ 1. write access: `WorkerUserContext#setResource(String, Object)`
+ 2. read access: `WorkerTopologyContext#getResouce(String)` or `TopologyContext#getResource(String)`
diff --git a/releases/2.8.8/IConfigLoader.md b/releases/2.8.8/IConfigLoader.md
new file mode 100644
index 0000000000..3c2f4de1ed
--- /dev/null
+++ b/releases/2.8.8/IConfigLoader.md
@@ -0,0 +1,51 @@
+---
+title: IConfigLoader
+layout: documentation
+documentation: true
+---
+
+
+### Introduction
+IConfigLoader is an interface designed to allow dynamic loading of scheduler resource constraints. Currently, the MultiTenant scheduler uses this interface to dynamically load the number of isolated nodes a given user has been guaranteed, and the ResoureAwareScheduler uses the interface to dynamically load per user resource guarantees.
+
+The following interface is provided for users to create an IConfigLoader instance based on the scheme of the `scheduler.config.loader.uri`.
+```
+ConfigLoaderFactoryService.createConfigLoader(Map<String, Object> conf)
+```
+
+------
+
+### Interface
+```
+public interface IConfigLoader {
+ Map<?,?> load();
+};
+```
+#### Description
+ - load is called by the scheduler whenever it wishes to retrieve the most recent configuration map.
+
+#### Loader Configuration
+The loaders are dynamically selected and dynamically configured through configuration items in the scheduler implementations.
+
+##### Example
+```
+scheduler.config.loader.uri: "artifactory+http://artifactory.my.company.com:8000/artifactory/configurations/clusters/my_cluster/ras_pools"
+scheduler.config.loader.timeout.sec: 30
+```
+Or
+```
+scheduler.config.loader.uri: "file:///path/to/my/config.yaml"
+```
+### Implementations
+
+There are currently two implemenations of IConfigLoader
+ - org.apache.storm.scheduler.utils.ArtifactoryConfigLoader: Load configurations from an Artifactory server.
+ It will be used if users add `artifactory+` to the scheme of the real URI and set to `scheduler.config.loader.uri`.
+ - org.apache.storm.scheduler.utils.FileConfigLoader: Load configurations from a local file. It will be used if users use `file` scheme.
+
+#### Configurations
+ - scheduler.config.loader.uri: For `ArtifactoryConfigLoader`, this can either be a reference to an individual file in Artifactory or to a directory. If it is a directory, the file with the largest lexographic name will be returned.
+ For `FileConfigLoader`, this is the URI pointing to a file.
+ - scheduler.config.loader.timeout.secs: Currently only used in `ArtifactoryConfigLoader`. It is the amount of time an http connection to the artifactory server will wait before timing out. The default is 10.
+ - scheduler.config.loader.polltime.secs: Currently only used in `ArtifactoryConfigLoader`. It is the frequency at which the plugin will call out to artifactory instead of returning the most recently cached result. The default is 600 seconds.
+ - scheduler.config.loader.artifactory.base.directory: Only used in `ArtifactoryConfigLoader`. It is the part of the uri, configurable in Artifactory, which represents the top of the directory tree. It defaults to "/artifactory".
\ No newline at end of file
diff --git a/releases/2.8.8/Implementation-docs.md b/releases/2.8.8/Implementation-docs.md
new file mode 100644
index 0000000000..46fe88cd38
--- /dev/null
+++ b/releases/2.8.8/Implementation-docs.md
@@ -0,0 +1,13 @@
+---
+title: Storm Internal Implementation
+layout: documentation
+documentation: true
+---
+This section of the wiki is dedicated to explaining how Storm is implemented. You should have a good grasp of how to use Storm before reading these sections.
+
+- [Structure of the codebase](Structure-of-the-codebase.html)
+- [Lifecycle of a topology](Lifecycle-of-a-topology.html)
+- [Message passing implementation](Message-passing-implementation.html)
+- [Acking framework implementation](Acking-framework-implementation.html)
+- [Metrics](Metrics.html)
+- [Nimbus HA](nimbus-ha-design.html)
diff --git a/releases/2.8.8/Installing-native-dependencies.md b/releases/2.8.8/Installing-native-dependencies.md
new file mode 100644
index 0000000000..1937d4bffc
--- /dev/null
+++ b/releases/2.8.8/Installing-native-dependencies.md
@@ -0,0 +1,38 @@
+---
+layout: documentation
+---
+The native dependencies are only needed on actual Storm clusters. When running Storm in local mode, Storm uses a pure Java messaging system so that you don't need to install native dependencies on your development machine.
+
+Installing ZeroMQ and JZMQ is usually straightforward. Sometimes, however, people run into issues with autoconf and get strange errors. If you run into any issues, please email the [Storm mailing list](http://groups.google.com/group/storm-user) or come get help in the #storm-user room on freenode.
+
+Storm has been tested with ZeroMQ 2.1.7, and this is the recommended ZeroMQ release that you install. You can download a ZeroMQ release [here](http://download.zeromq.org/). Installing ZeroMQ should look something like this:
+
+```
+wget http://download.zeromq.org/zeromq-2.1.7.tar.gz
+tar -xzf zeromq-2.1.7.tar.gz
+cd zeromq-2.1.7
+./configure
+make
+sudo make install
+```
+
+JZMQ is the Java bindings for ZeroMQ. JZMQ doesn't have any releases (we're working with them on that), so there is risk of a regression if you always install from the master branch. To prevent a regression from happening, you should instead install from [this fork](http://github.com/nathanmarz/jzmq) which is tested to work with Storm. Installing JZMQ should look something like this:
+
+```
+#install jzmq
+git clone https://github.com/nathanmarz/jzmq.git
+cd jzmq
+./autogen.sh
+./configure
+make
+sudo make install
+```
+
+To get the JZMQ build to work, you may need to do one or all of the following:
+
+1. Set JAVA_HOME environment variable appropriately
+2. Install Java dev package (more info [here](http://codeslinger.posterous.com/getting-zeromq-and-jzmq-running-on-mac-os-x) for Mac OSX users)
+3. Upgrade autoconf on your machine
+4. Follow the instructions in [this blog post](http://blog.pmorelli.com/getting-zeromq-and-jzmq-running-on-mac-os-x)
+
+If you run into any errors when running `./configure`, [this thread](http://stackoverflow.com/questions/3522248/how-do-i-compile-jzmq-for-zeromq-on-osx) may provide a solution.
diff --git a/releases/2.8.8/Joins.md b/releases/2.8.8/Joins.md
new file mode 100644
index 0000000000..9efb7c68f1
--- /dev/null
+++ b/releases/2.8.8/Joins.md
@@ -0,0 +1,139 @@
+---
+title: Joining Streams in Storm Core
+layout: documentation
+documentation: true
+---
+
+Storm core supports joining multiple data streams into one with the help of `JoinBolt`.
+`JoinBolt` is a Windowed bolt, i.e. it waits for the configured window duration to match up the
+tuples among the streams being joined. This helps align the streams within a Window boundary.
+
+Each of `JoinBolt`'s incoming data streams must be Fields Grouped on a single field. A stream
+should only be joined with the other streams using the field on which it has been FieldsGrouped.
+Knowing this will help understand the join syntax described below.
+
+## Performing Joins
+Consider the following SQL join involving 4 tables:
+
+```sql
+select userId, key4, key2, key3
+from table1
+inner join table2 on table2.userId = table1.key1
+inner join table3 on table3.key3 = table2.userId
+left join table4 on table4.key4 = table3.key3
+```
+
+Similar joins could be expressed on tuples generated by 4 spouts using `JoinBolt`:
+
+```java
+JoinBolt jbolt = new JoinBolt("spout1", "key1") // from spout1
+ .join ("spout2", "userId", "spout1") // inner join spout2 on spout2.userId = spout1.key1
+ .join ("spout3", "key3", "spout2") // inner join spout3 on spout3.key3 = spout2.userId
+ .leftJoin ("spout4", "key4", "spout3") // left join spout4 on spout4.key4 = spout3.key3
+ .select ("userId, key4, key2, spout3:key3") // chose output fields
+ .withTumblingWindow( new Duration(10, TimeUnit.MINUTES) ) ;
+
+topoBuilder.setBolt("joiner", jbolt, 1)
+ .fieldsGrouping("spout1", new Fields("key1") )
+ .fieldsGrouping("spout2", new Fields("userId") )
+ .fieldsGrouping("spout3", new Fields("key3") )
+ .fieldsGrouping("spout4", new Fields("key4") );
+```
+
+The bolt constructor takes two arguments. The 1st argument introduces the data from `spout1`
+to be the first stream and specifies that it will always use field `key1` when joining this with the others streams.
+The name of the component specified must refer to the spout or bolt that is directly connected to the Join bolt.
+Here data received from `spout1` must be fields grouped on `key1`. Similarly, each of the `leftJoin()` and `join()` method
+calls introduce a new stream along with the field to use for the join. As seen in above example, the same FieldsGrouping
+requirement applies to these streams as well. The 3rd argument to the join methods refers to another stream with which
+to join.
+
+The `select()` method is used to specify the output fields. The argument to `select` is a comma separated list of fields.
+Individual field names can be prefixed with a stream name to disambiguate between the same field name occurring in
+multiple streams as follows: `.select("spout3:key3, spout4:key3")`. Nested tuple types are supported if the
+nesting has been done using `Map`s. For example `outer.inner.innermost` refers to a field that is nested three levels
+deep where `outer` and `inner` are of type `Map`.
+
+Stream name prefix is not allowed for the fields in any of the join() arguments, but nested fields are supported.
+
+The call to `withTumblingWindow()` above, configures the join window to be a 10 minute tumbling window. Since `JoinBolt`
+is a Windowed Bolt, we can also use the `withWindow` method to configure it as a sliding window (see tips section below).
+
+## Stream names and Join order
+* Stream names must be introduced (in constructor or as 1st arg to various join methods) before being referred
+to (in the 3rd argument of the join methods). Forward referencing of stream names, as shown below, is not allowed:
+
+```java
+new JoinBolt( "spout1", "key1")
+ .join ( "spout2", "userId", "spout3") //not allowed. spout3 not yet introduced
+ .join ( "spout3", "key3", "spout1")
+```
+* Internally, the joins will be performed in the order expressed by the user.
+
+## Joining based on Stream names
+
+For simplicity, Storm topologies often use the `default` stream. Topologies can also use named streams
+instead of `default` streams. To support such topologies, `JoinBolt` can be configured to use stream
+names, instead of source component (spout/bolt) names, via the constructor's first argument:
+
+```java
+new JoinBolt(JoinBolt.Selector.STREAM, "stream1", "key1")
+ .join("stream2", "key2")
+ ...
+```
+The first argument `JoinBolt.Selector.STREAM` informs the bolt that `stream1/2/3/4` refer to named streams
+(as opposed to names of upstream spouts/bolts).
+
+
+The below example joins two named streams from four spouts:
+
+```java
+new JoinBolt(JoinBolt.Selector.STREAM, "stream1", "key1")
+ .join ("stream2", "userId", "stream1" )
+ .select ("userId, key1, key2")
+ .withTumblingWindow( new Duration(10, TimeUnit.MINUTES) ) ;
+
+topoBuilder.setBolt("joiner", jbolt, 1)
+ .fieldsGrouping("bolt1", "stream1", new Fields("key1") )
+ .fieldsGrouping("bolt2", "stream1", new Fields("key1") )
+ .fieldsGrouping("bolt3", "stream2", new Fields("userId") )
+ .fieldsGrouping("bolt4", "stream1", new Fields("key1") );
+```
+
+In the above example, it is possible that `bolt1`, for example, is emitting other streams also. But the join bolt
+is only subscribing to `stream1` & `stream2` from the different bolts. `stream1` from `bolt1`, `bolt2` and `bolt4`
+is treated as a single stream and joined against `stream2` from `bolt3`.
+
+## Limitations:
+1. Currently only INNER and LEFT joins are supported.
+
+2. Unlike SQL, which allows joining the same table on different keys to different tables, here the same one field must be used
+ on a stream. Fields Grouping ensures the right tuples are routed to the right instances of a Join Bolt. Consequently the
+ FieldsGrouping field must be same as the join field, for correct results. To perform joins on multiple fields, the fields
+ can be combined into one field and then sent to the Join bolt.
+
+
+## Tips:
+
+1. Joins can be CPU and memory intensive. The larger the data accumulated in the current window (proportional to window
+ length), the longer it takes to do the join. Having a short sliding interval (few seconds for example) triggers frequent
+ joins. Consequently performance can suffer if using large window lengths or small sliding intervals or both.
+
+2. Duplication of joined records across windows is possible when using Sliding Windows. This is because the tuples continue to exist
+ across multiple windows when using Sliding Windows.
+
+3. If message timeouts are enabled, ensure the timeout setting (topology.message.timeout.secs) is large enough to comfortably
+ accommodate the window size, plus the additional processing by other spouts and bolts.
+
+4. Joining a window of two streams with M and N elements each, *in the worst case*, can produce MxN elements with every output tuple
+ anchored to one tuple from each input stream. This can mean a lot of output tuples from JoinBolt and even more ACKs for downstream bolts
+ to emit. This can place a substantial pressure on the messaging system and dramatically slowdown the topology if not careful.
+ To manage the load on the messaging subsystem, it is advisable to:
+ * Increase the worker's heap (topology.worker.max.heap.size.mb).
+ * **If** ACKing is not necessary for your topology, disable ACKers (topology.acker.executors=0).
+ * Disable event logger (topology.eventlogger.executors=0).
+ * Turn of topology debugging (topology.debug=false).
+ * Set topology.max.spout.pending to a value large enough to accommodate an estimated full window worth of tuples plus some more for headroom.
+ This helps mitigate the possibility of spouts emitting excessive tuples when messaging subsystem is experiencing excessive load. This situation
+ can occur when its value is set to null.
+ * Lastly, keep the window size to the minimum value necessary for solving the problem at hand.
diff --git a/releases/2.8.8/Kestrel-and-Storm.md b/releases/2.8.8/Kestrel-and-Storm.md
new file mode 100644
index 0000000000..1efe39768d
--- /dev/null
+++ b/releases/2.8.8/Kestrel-and-Storm.md
@@ -0,0 +1,202 @@
+---
+title: Storm and Kestrel
+layout: documentation
+documentation: true
+---
+This page explains how to use Storm to consume items from a Kestrel cluster.
+
+## Preliminaries
+### Storm
+This tutorial uses examples from the [storm-kestrel](https://github.com/nathanmarz/storm-kestrel) project and the [storm-starter](http://github.com/apache/storm/blob/{{page.version}}/examples/storm-starter) project. It's recommended that you clone those projects and follow along with the examples. Read [Setting up development environment](Setting-up-development-environment.html) and [Creating a new Storm project](Creating-a-new-Storm-project.html) to get your machine set up.
+### Kestrel
+It assumes you are able to run locally a Kestrel server as described [here](https://github.com/nathanmarz/storm-kestrel).
+
+## Kestrel Server and Queue
+A single kestrel server has a set of queues. A Kestrel queue is a very simple message queue that runs on the JVM and uses the memcache protocol (with some extensions) to talk to clients. For details, look at the implementation of the [KestrelThriftClient](https://github.com/nathanmarz/storm-kestrel/blob/master/src/jvm/org/apache/storm/spout/KestrelThriftClient.java) class provided in [storm-kestrel](https://github.com/nathanmarz/storm-kestrel) project.
+
+Each queue is strictly ordered following the FIFO (first in, first out) principle. To keep up with performance items are cached in system memory; though, only the first 128MB is kept in memory. When stopping the server, the queue state is stored in a journal file.
+
+Further, details can be found [here](https://github.com/nathanmarz/kestrel/blob/master/docs/guide.md).
+
+Kestrel is:
+* fast
+* small
+* durable
+* reliable
+
+For instance, Twitter uses Kestrel as the backbone of its messaging infrastructure as described [here] (http://bhavin.directi.com/notes-on-kestrel-the-open-source-twitter-queue/).
+
+## Add items to Kestrel
+At first, we need to have a program that can add items to a Kestrel queue. The following method takes benefit of the KestrelClient implementation in [storm-kestrel](https://github.com/nathanmarz/storm-kestrel). It adds sentences into a Kestrel queue randomly chosen out of an array that holds five possible sentences.
+
+```
+ private static void queueSentenceItems(KestrelClient kestrelClient, String queueName)
+ throws ParseError, IOException {
+
+ String[] sentences = new String[] {
+ "the cow jumped over the moon",
+ "an apple a day keeps the doctor away",
+ "four score and seven years ago",
+ "snow white and the seven dwarfs",
+ "i am at two with nature"};
+
+ Random _rand = new Random();
+
+ for(int i=1; i<=10; i++){
+
+ String sentence = sentences[_rand.nextInt(sentences.length)];
+
+ String val = "ID " + i + " " + sentence;
+
+ boolean queueSucess = kestrelClient.queue(queueName, val);
+
+ System.out.println("queueSucess=" +queueSucess+ " [" + val +"]");
+ }
+ }
+```
+
+## Remove items from Kestrel
+
+This method dequeues items from a queue without removing them.
+
+```
+ private static void dequeueItems(KestrelClient kestrelClient, String queueName) throws IOException, ParseError
+ {
+ for(int i=1; i<=12; i++){
+
+ Item item = kestrelClient.dequeue(queueName);
+
+ if(item==null){
+ System.out.println("The queue (" + queueName + ") contains no items.");
+ }
+ else
+ {
+ byte[] data = item._data;
+
+ String receivedVal = new String(data);
+
+ System.out.println("receivedItem=" + receivedVal);
+ }
+ }
+```
+
+This method dequeues items from a queue and then removes them.
+
+```
+ private static void dequeueAndRemoveItems(KestrelClient kestrelClient, String queueName)
+ throws IOException, ParseError
+ {
+ for(int i=1; i<=12; i++){
+
+ Item item = kestrelClient.dequeue(queueName);
+
+
+ if(item==null){
+ System.out.println("The queue (" + queueName + ") contains no items.");
+ }
+ else
+ {
+ int itemID = item._id;
+
+
+ byte[] data = item._data;
+
+ String receivedVal = new String(data);
+
+ kestrelClient.ack(queueName, itemID);
+
+ System.out.println("receivedItem=" + receivedVal);
+ }
+ }
+ }
+```
+
+## Add Items continuously to Kestrel
+
+This is our final program to run in order to add continuously sentence items to a queue called **sentence_queue** of a locally running Kestrel server.
+
+In order to stop it type a closing bracket char ']' in console and hit 'Enter'.
+
+```
+ import java.io.IOException;
+ import java.io.InputStream;
+ import java.util.Random;
+
+ import org.apache.storm.spout.KestrelClient;
+ import org.apache.storm.spout.KestrelClient.Item;
+ import org.apache.storm.spout.KestrelClient.ParseError;
+
+ public class AddSentenceItemsToKestrel {
+
+ /**
+ * @param args
+ */
+ public static void main(String[] args) {
+
+ InputStream is = System.in;
+
+ char closing_bracket = ']';
+
+ int val = closing_bracket;
+
+ boolean aux = true;
+
+ try {
+
+ KestrelClient kestrelClient = null;
+ String queueName = "sentence_queue";
+
+ while(aux){
+
+ kestrelClient = new KestrelClient("localhost",22133);
+
+ queueSentenceItems(kestrelClient, queueName);
+
+ kestrelClient.close();
+
+ Thread.sleep(1000);
+
+ if(is.available()>0){
+ if(val==is.read())
+ aux=false;
+ }
+ }
+ } catch (IOException e) {
+ // TODO Auto-generated catch block
+ e.printStackTrace();
+ }
+ catch (ParseError e) {
+ // TODO Auto-generated catch block
+ e.printStackTrace();
+ } catch (InterruptedException e) {
+ // TODO Auto-generated catch block
+ e.printStackTrace();
+ }
+
+ System.out.println("end");
+
+ }
+ }
+```
+## Using KestrelSpout
+
+This topology reads sentences off of a Kestrel queue using KestrelSpout, splits the sentences into its constituent words (Bolt: SplitSentence), and then emits for each word the number of times it has seen that word before (Bolt: WordCount). How data is processed is described in detail in [Guaranteeing message processing](Guaranteeing-message-processing.html).
+
+```
+ TopologyBuilder builder = new TopologyBuilder();
+ builder.setSpout("sentences", new KestrelSpout("localhost",22133,"sentence_queue",new StringScheme()));
+ builder.setBolt("split", new SplitSentence(), 10)
+ .shuffleGrouping("sentences");
+ builder.setBolt("count", new WordCount(), 20)
+ .fieldsGrouping("split", new Fields("word"));
+```
+
+## Execution
+
+At first, start your local kestrel server in production or development mode.
+
+Than, wait about 5 seconds in order to avoid a ConnectException.
+
+Now execute the program to add items to the queue and launch the Storm topology. The order in which you launch the programs is of no importance.
+
+If you run the topology with TOPOLOGY_DEBUG you should see tuples being emitted in the topology.
diff --git a/releases/2.8.8/Lifecycle-of-a-topology.md b/releases/2.8.8/Lifecycle-of-a-topology.md
new file mode 100644
index 0000000000..fe785f1e4c
--- /dev/null
+++ b/releases/2.8.8/Lifecycle-of-a-topology.md
@@ -0,0 +1,82 @@
+---
+title: Lifecycle of a Storm Topology
+layout: documentation
+documentation: true
+---
+(**NOTE**: this page is based on the 0.7.1 code; many things have changed since then, including a split between tasks and executors, and a reorganization of the code under `storm-client/src` rather than `src/`.)
+
+This page explains in detail the lifecycle of a topology from running the "storm jar" command to uploading the topology to Nimbus to the supervisors starting/stopping workers to workers and tasks setting themselves up. It also explains how Nimbus monitors topologies and how topologies are shutdown when they are killed.
+
+First a couple of important notes about topologies:
+
+1. The actual topology that runs is different than the topology the user specifies. The actual topology has implicit streams and an implicit "acker" bolt added to manage the acking framework (used to guarantee data processing). The implicit topology is created via the [system-topology!](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/common.clj#L188) function.
+2. `system-topology!` is used in two places:
+ - when Nimbus is creating tasks for the topology [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L316)
+ - in the worker so it knows where it needs to route messages to [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L90)
+
+## Starting a topology
+
+- "storm jar" command executes your class with the specified arguments. The only special thing that "storm jar" does is set the "storm.jar" environment variable for use by `StormSubmitter` later. [code](https://github.com/apache/storm/blob/0.7.1/bin/storm#L101)
+- When your code uses `StormSubmitter.submitTopology`, `StormSubmitter` takes the following actions:
+ - First, `StormSubmitter` uploads the jar if it hasn't been uploaded before. [code](https://github.com/apache/storm/blob/0.7.1/src/jvm/org/apache/storm/StormSubmitter.java#L83)
+ - Jar uploading is done via Nimbus's Thrift interface [code](https://github.com/apache/storm/blob/0.7.1/src/storm.thrift#L200)
+ - `beginFileUpload` returns a path in Nimbus's inbox
+ - 15 kilobytes are uploaded at a time through `uploadChunk`
+ - `finishFileUpload` is called when it's finished uploading
+ - Here is Nimbus's implementation of those Thrift methods: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L694)
+ - Second, `StormSubmitter` calls `submitTopology` on the Nimbus thrift interface [code](https://github.com/apache/storm/blob/0.7.1/src/jvm/org/apache/storm/StormSubmitter.java#L60)
+ - The topology config is serialized using JSON (JSON is used so that writing DSL's in any language is as easy as possible)
+ - Notice that the Thrift `submitTopology` call takes in the Nimbus inbox path where the jar was uploaded
+
+- Nimbus receives the topology submission. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L639)
+- Nimbus normalizes the topology configuration. The main purpose of normalization is to ensure that every single task will have the same serialization registrations, which is critical for getting serialization working correctly. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L557)
+- Nimbus sets up the static state for the topology [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L661)
+ - Jars and configs are kept on local filesystem because they're too big for Zookeeper. The jar and configs are copied into the path {nimbus local dir}/stormdist/{topology id}
+ - `setup-storm-static` writes task -> component mapping into ZK
+ - `setup-heartbeats` creates a ZK "directory" in which tasks can heartbeat
+- Nimbus calls `mk-assignment` to assign tasks to machines [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L458)
+ - Assignment record definition is here: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/common.clj#L25)
+ - Assignment contains:
+ - `master-code-dir`: used by supervisors to download the correct jars/configs for the topology from Nimbus
+ - `task->node+port`: Map from a task id to the worker that task should be running on. (A worker is identified by a node/port pair)
+ - `node->host`: A map from node id to hostname. This is used so workers know which machines to connect to to communicate with other workers. Node ids are used to identify supervisors so that multiple supervisors can be run on one machine. One place this is done is with Mesos integration.
+ - `task->start-time-secs`: Contains a map from task id to the timestamp at which Nimbus launched that task. This is used by Nimbus when monitoring topologies, as tasks are given a longer timeout to heartbeat when they're first launched (the launch timeout is configured by "nimbus.task.launch.secs" config)
+- Once topologies are assigned, they're initially in a deactivated mode. `start-storm` writes data into Zookeeper so that the cluster knows the topology is active and can start emitting tuples from spouts. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L504)
+
+- TODO cluster state diagram (show all nodes and what's kept everywhere)
+
+- Supervisor runs two functions in the background:
+ - `synchronize-supervisor`: This is called whenever assignments in Zookeeper change and also every 10 seconds. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/supervisor.clj#L241)
+ - Downloads code from Nimbus for topologies assigned to this machine for which it doesn't have the code yet. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/supervisor.clj#L258)
+ - Writes into local filesystem what this node is supposed to be running. It writes a map from port -> LocalAssignment. LocalAssignment contains a topology id as well as the list of task ids for that worker. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/supervisor.clj#L13)
+ - `sync-processes`: Reads from the LFS what `synchronize-supervisor` wrote and compares that to what's actually running on the machine. It then starts/stops worker processes as necessary to synchronize. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/supervisor.clj#L177)
+
+- Worker processes start up through the `mk-worker` function [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L67)
+ - Worker connects to other workers and starts a thread to monitor for changes. So if a worker gets reassigned, the worker will automatically reconnect to the other worker's new location. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L123)
+ - Monitors whether a topology is active or not and stores that state in the `storm-active-atom` variable. This variable is used by tasks to determine whether or not to call `nextTuple` on the spouts. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L155)
+ - The worker launches the actual tasks as threads within it [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L178)
+- Tasks are set up through the `mk-task` function [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L160)
+ - Tasks set up routing function which takes in a stream and an output tuple and returns a list of task ids to send the tuple to [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L207) (there's also a 3-arity version used for direct streams)
+ - Tasks set up the spout-specific or bolt-specific code with [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L241)
+
+## Topology Monitoring
+
+- Nimbus monitors the topology during its lifetime
+ - Schedules recurring task on the timer thread to check the topologies [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L623)
+ - Nimbus's behavior is represented as a finite state machine [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L98)
+ - The "monitor" event is called on a topology every "nimbus.monitor.freq.secs", which calls `reassign-topology` through `reassign-transition` [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L497)
+ - `reassign-topology` calls `mk-assignments`, the same function used to assign the topology the first time. `mk-assignments` is also capable of incrementally updating a topology
+ - `mk-assignments` checks heartbeats and reassigns workers as necessary
+ - Any reassignments change the state in ZK, which will trigger supervisors to synchronize and start/stop workers
+
+## Killing a topology
+
+- "storm kill" command runs this code which just calls the Nimbus Thrift interface to kill the topology: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/command/kill_topology.clj)
+- Nimbus receives the kill command [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L671)
+- Nimbus applies the "kill" transition to the topology [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L676)
+- The kill transition function changes the status of the topology to "killed" and schedules the "remove" event to run "wait time seconds" in the future. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L63)
+ - The wait time defaults to the topology message timeout but can be overridden with the -w flag in the "storm kill" command
+ - This causes the topology to be deactivated for the wait time before its actually shut down. This gives the topology a chance to finish processing what it's currently processing before shutting down the workers
+ - Changing the status during the kill transition ensures that the kill protocol is fault-tolerant to Nimbus crashing. On startup, if the status of the topology is "killed", Nimbus schedules the remove event to run "wait time seconds" in the future [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L111)
+- Removing a topology cleans out the assignment and static information from ZK [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L116)
+- A separate cleanup thread runs the `do-cleanup` function which will clean up the heartbeat dir and the jars/configs stored locally. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/nimbus.clj#L577)
diff --git a/releases/2.8.8/Local-mode.md b/releases/2.8.8/Local-mode.md
new file mode 100644
index 0000000000..ced00bde8c
--- /dev/null
+++ b/releases/2.8.8/Local-mode.md
@@ -0,0 +1,86 @@
+---
+title: Local Mode
+layout: documentation
+documentation: true
+---
+Local mode simulates a Storm cluster in process and is useful for developing and testing topologies. Running topologies in local mode is similar to running topologies [on a cluster](Running-topologies-on-a-production-cluster.html).
+
+To run a topology in local mode you have two options. The most common option is to run your topology with `storm local` instead of `storm jar`
+
+This will bring up a local simulated cluster and force all interactions with nimbus to go through the simulated cluster instead of going to a separate process. By default this will run the process for 20 seconds before tearing down the entire cluster. You can override this by including a `--local-ttl` command line option which sets the number of seconds it should run for.
+
+### Programmatic
+
+If you want to do some automated testing but without actually launching a storm cluster you can use the same classes internally that `storm local` does.
+
+To do this you first need to pull in the dependencies needed to access these classes. For the java API you should depend on `storm-server` as a `test` dependency.
+
+To create an in-process cluster, simply use the `LocalCluster` class. For example:
+
+```java
+import org.apache.storm.LocalCluster;
+
+...
+
+try (LocalCluster cluster = new LocalCluster()) {
+ //Interact with the cluster...
+}
+```
+
+You can then submit topologies using the `submitTopology` method on the `LocalCluster` object. Just like the corresponding method on [StormSubmitter](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/StormSubmitter.html), `submitTopology` takes a name, a topology configuration, and the topology object. You can then kill a topology using the `killTopology` method which takes the topology name as an argument.
+
+The `LocalCluster` is an `AutoCloseable` and will shut down when close is called.
+
+many of the Nimbus APIs are also available through the LocalCluster.
+
+### DRPC
+
+DRPC can be run in local mode as well. Here's how to run the above example in local mode:
+
+```java
+try (LocalDRPC drpc = new LocalDRPC();
+ LocalCluster cluster = new LocalCluster();
+ LocalTopology topo = cluster.submitTopology("drpc-demo", conf, builder.createLocalTopology(drpc))) {
+
+ System.out.println("Results for 'hello':" + drpc.execute("exclamation", "hello"));
+}
+```
+
+First you create a `LocalDRPC` object. This object simulates a DRPC server in process, just like how `LocalCluster` simulates a Storm cluster in process. Then you create the `LocalCluster` to run the topology in local mode. `LinearDRPCTopologyBuilder` has separate methods for creating local topologies and remote topologies. In local mode the `LocalDRPC` object does not bind to any ports so the topology needs to know about the object to communicate with it. This is why `createLocalTopolog [...]
+
+After launching the topology, you can do DRPC invocations using the `execute` method on `LocalDRPC`.
+
+Because all of the objects used are instances of AutoCloseable when the try blocks scope ends the topology is killed, the cluster is shut down and the drpc server also shuts down.
+
+### Clojure API
+
+Storm also offers a clojure API for testing.
+
+[This blog post](http://www.pixelmachine.org/2011/12/21/Testing-Storm-Topologies-Part-2.html) talk about this, but is a little out of date. To get this functionality you need to include the `storm-clojure-test` dependency. This will pull in a lot of storm itself that should not be packaged with your topology, sp please make sure it is a test dependency only,.
+
+### Debugging your topology with an IDE
+
+One of the great use cases for local mode is to be able to walk through the code execution of your bolts and spouts using an IDE. You can do this on the command line by adding the `--java-debug` option followed by the parameter you would pass to jdwp. This makes it simple to launch the local cluster with `-agentlib:jdwp=` turned on.
+
+When running from within an IDE itself you can modify your code run run withing a call to `LocalCluster.withLocalModeOverride`
+
+```java
+public static void main(final String args[]) {
+ LocalCluster.withLocalModeOverride(() -> originalMain(args), 10);
+}
+```
+
+Or you could also modify the IDE to run "org.apache.storm.LocalCluster" instead of your main class when launching, and pass in the name of the class as an argument to it. This will also trigger local mode, and is what `storm local` does behind the scenes.
+
+### Common configurations for local mode
+
+You can see a full list of configurations [here](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html).
+
+1. **Config.TOPOLOGY_MAX_TASK_PARALLELISM**: This config puts a ceiling on the number of threads spawned for a single component. Oftentimes production topologies have a lot of parallelism (hundreds of threads) which places unreasonable load when trying to test the topology in local mode. This config lets you easy control that parallelism.
+2. **Config.TOPOLOGY_DEBUG**: When this is set to true, Storm will log a message every time a tuple is emitted from any spout or bolt. This is extremely useful for debugging.A
+
+These, like all other configs, can be set on the command line when launching your topology with the `-c` flag. The flag is of the form `-c <conf_name>=<JSON_VALUE>` so to enable debugging when launching your topology in local mode you could run
+
+```
+storm local topology.jar <MY_MAIN_CLASS> -c topology.debug=true
+```
diff --git a/releases/2.8.8/LocalityAwareness.md b/releases/2.8.8/LocalityAwareness.md
new file mode 100644
index 0000000000..517f1c7d96
--- /dev/null
+++ b/releases/2.8.8/LocalityAwareness.md
@@ -0,0 +1,84 @@
+---
+title: Locality Awareness In LoadAwareShuffleGrouping
+layout: documentation
+documentation: true
+---
+
+# Locality Awareness In LoadAwareShuffleGrouping
+
+### Motivation
+
+Apache Storm has introduced locality awareness to LoadAwareShuffleGrouping based on Bang-Bang control theory.
+It aims to keep traffic to closer downstream executors to avoid network latency when those executors are not under heavy load.
+It can also avoid serialization/deserialization overhead if the traffic happens in the same worker.
+
+### How it works
+
+An executor (say `E`) which has LoadAwareShuffleGrouping to downstream executors views them in four `scopes` based on their locations relative to the executor `E` itself.
+The four scopes are:
+
+* `WORKER_LOCAL`: every downstream executor located on the same worker as this executor `E`
+* `HOST_LOCAL`: every downstream executor located on the same host as this executor `E`
+* `RACK_LOCAL`: every downstream executor located on the same rack as this executor `E`
+* `EVERYTHING`: every downstream executor of the executor `E`
+
+It starts with sending tuples to the downstream executors in the scope of `WORKER_LOCAL`.
+The downstream executors in the scope are chosen based on their load. Executors with lower load are more likely to be chosen.
+Once the average load of these `WORKER_LOCAL` executors reaches `topology.localityaware.higher.bound`,
+it switches to the higher scope which is `HOST_LOCAL` and starts sending tuples in that scope.
+And if the average load is still higher than the `higher bound`, it switches to a higher scope.
+
+On the other hand, it switches to a lower scope if the average load of the lower scope is less than `topology.localityaware.lower.bound`.
+
+
+### How is Load calculated
+
+The load of an downstream executor is the maximum of the following two:
+
+* The population percentage of the receive queue
+* Math.min(pendingMessages, 1024) / 1024.
+
+`pendingMessages`: The upstream executor `E` sends messages to the downstream executor through Netty and the `pendingMessages` is the number of messages that haven't got through to the server.
+
+If the downstream executor located on the same worker as the executor `E`, the load of that downstream executor is:
+* The population percentage of the receive queue
+
+### Relationship between Load and Capacity
+
+The capacity of a bolt executor on Storm UI is calculated as:
+ * (number executed * average execute latency) / measurement time
+
+It basically means how busy this executor is. If this is around 1.0, the corresponding Bolt is running as fast as it can. A `__capacity` metric exists to track this value for each executor.
+
+The `Capacity` is not related to the `Load`:
+
+* If the `Load` of the executor `E1` is high,
+ * the `Capacity` of `E1` could be high: population of the receive queue of `E1` could be high and it means the executor `E` has more work to do.
+ * the `Capacity` could also be low: `pendingMessage` could be high because other executors share the netty connection between the two workers and they are sending too many messages.
+ But the actual population of the receive queue of `E1` might be low.
+* If the `Load` is low,
+ * the `Capacity` could be low: lower `Load` means less work to do.
+ * the `Capacity` could also be high: because the executor could be receiving tuples and executing tuples at the similar average rate.
+* If the `Capacity` is high,
+ * the `Load` could be high: high `Capacity` means the executor is busy. It could be because it's receiving too many tuples.
+ * the `Load` could also be low: because the executor could be receiving tuples and executing tuples at the similar average rate.
+* If the `Capacity` is low,
+ * the `Load` could be low: if the `pendingMessage` is low
+ * the `Load` could also be high: because the `pendingMessage` might be very high.
+
+
+### Troubleshooting
+
+#### I am seeing high capacity (close to 1.0) on some executors and low capacity (close to 0) on other executors
+
+1. It could mean that you could reduce parallelism. Your executors are able to keep up and the load never gets to a very high point.
+
+2. You can try to adjust `topology.localityaware.higher.bound` and `topology.localityaware.lower.bound`
+
+3. You can try to enable `topology.ras.order.executors.by.proximity.needs`. With this config, unassigned executors will be sorted by topological order
+with network proximity needs before being scheduled. This is a best-effort to split the topology to slices and allocate executors in each slice to as closest physical location as possible.
+
+
+#### I just want the capacity on every downstream executor to be even
+
+You can turn off LoadAwareShuffleGrouping by setting `topology.disable.loadaware.messaging` to `true`.
diff --git a/releases/2.8.8/Logs.md b/releases/2.8.8/Logs.md
new file mode 100644
index 0000000000..46cc922c9e
--- /dev/null
+++ b/releases/2.8.8/Logs.md
@@ -0,0 +1,30 @@
+---
+title: Storm Logs
+layout: documentation
+documentation: true
+---
+Logs in Storm are essential for tracking the status, operations, error messages and debug information for all the
+daemons (e.g., nimbus, supervisor, logviewer, drpc, ui, pacemaker) and topologies' workers.
+
+### Location of the Logs
+All the daemon logs are placed under ${storm.log.dir} directory, which an administrator can set in the System properties or
+in the cluster configuration. By default, ${storm.log.dir} points to ${storm.home}/logs.
+
+All the worker logs are placed under the workers-artifacts directory in a hierarchical manner, e.g.,
+${workers-artifacts}/${topologyId}/${port}/worker.log. Users can set the workers-artifacts directory
+by configuring the variable "storm.workers.artifacts.dir". By default, workers-artifacts directory
+locates at ${storm.log.dir}/logs/workers-artifacts.
+
+### Using the Storm UI for Log View/Download and Log Search
+Daemon and worker logs are allowed to view and download through Storm UI by authorized users.
+
+To improve the debugging of Storm, we provide the Log Search feature.
+Log Search supports searching in a certain log file or in all of a topology's log files:
+
+String search in a log file: In the log page for a worker, a user can search a certain string, e.g., "Exception", in a certain worker log. This search can happen for both normal text log or rolled zip log files. In the results, the offset and matched lines will be displayed.
+
+
+
+Search in a topology: a user can also search a string for a certain topology by clicking the icon of the magnifying lens at the top right corner of the UI page. This means the UI will try to search on all the supervisor nodes in a distributed way to find the matched string in all logs for this topology. The search can happen for either normal text log files or rolled zip log files by checking/unchecking the "Search archived logs:" box. Then the matched results can be shown on the UI with [...]
+
+
diff --git a/releases/2.8.8/Maven.md b/releases/2.8.8/Maven.md
new file mode 100644
index 0000000000..47fd1b1acc
--- /dev/null
+++ b/releases/2.8.8/Maven.md
@@ -0,0 +1,22 @@
+---
+title: Maven
+layout: documentation
+documentation: true
+---
+To develop topologies, you'll need the Storm jars on your classpath. You should either include the unpacked jars in the classpath for your project or use Maven to include Storm as a development dependency. Storm is hosted on Maven Central. To include Storm in your project as a development dependency, add the following to your pom.xml:
+
+
+```xml
+<dependency>
+ <groupId>org.apache.storm</groupId>
+ <artifactId>storm-client</artifactId>
+ <version>{{page.version}}</version>
+ <scope>provided</scope>
+</dependency>
+```
+
+[Here's an example]({{page.git-blob-base}}/examples/storm-starter/pom.xml) of a pom.xml for a Storm project.
+
+### Developing Storm
+
+Please refer to [DEVELOPER.md]({{page.git-blob-base}}/DEVELOPER.md) for more details.
diff --git a/releases/2.8.8/Message-passing-implementation.md b/releases/2.8.8/Message-passing-implementation.md
new file mode 100644
index 0000000000..fd4cf2cbee
--- /dev/null
+++ b/releases/2.8.8/Message-passing-implementation.md
@@ -0,0 +1,30 @@
+---
+title: Message Passing Implementation
+layout: documentation
+documentation: true
+---
+(Note: this walkthrough is out of date as of 0.8.0. 0.8.0 revamped the message passing infrastructure to be based on the Disruptor)
+
+This page walks through how emitting and transferring tuples works in Storm.
+
+- Worker is responsible for message transfer
+ - `refresh-connections` is called every "task.refresh.poll.secs" or whenever assignment in ZK changes. It manages connections to other workers and maintains a mapping from task -> worker [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L123)
+ - Provides a "transfer function" that is used by tasks to send tuples to other tasks. The transfer function takes in a task id and a tuple, and it serializes the tuple and puts it onto a "transfer queue". There is a single transfer queue for each worker. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L56)
+ - The serializer is thread-safe [code](https://github.com/apache/storm/blob/0.7.1/src/jvm/org/apache/storm/serialization/KryoTupleSerializer.java#L26)
+ - The worker has a single thread which drains the transfer queue and sends the messages to other workers [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L185)
+ - Message sending happens through this protocol: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/messaging/protocol.clj)
+ - The implementation for distributed mode uses ZeroMQ [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/messaging/zmq.clj)
+ - The implementation for local mode uses in memory Java queues (so that it's easy to use Storm locally without needing to get ZeroMQ installed) [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/messaging/local.clj)
+- Receiving messages in tasks works differently in local mode and distributed mode
+ - In local mode, the tuple is sent directly to an in-memory queue for the receiving task [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/messaging/local.clj#L21)
+ - In distributed mode, each worker listens on a single TCP port for incoming messages and then routes those messages in-memory to tasks. The TCP port is called a "virtual port", because it receives [task id, message] and then routes it to the actual task. [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/worker.clj#L204)
+ - The virtual port implementation is here: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/zilch/virtual_port.clj)
+ - Tasks listen on an in-memory ZeroMQ port for messages from the virtual port [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L201)
+ - Bolts listen here: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L489)
+ - Spouts listen here: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L382)
+- Tasks are responsible for message routing. A tuple is emitted either to a direct stream (where the task id is specified) or a regular stream. In direct streams, the message is only sent if that bolt subscribes to that direct stream. In regular streams, the stream grouping functions are used to determine the task ids to send the tuple to.
+ - Tasks have a routing map from {stream id} -> {component id} -> {stream grouping function} [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L198)
+ - The "tasks-fn" returns the task ids to send the tuples to for either regular stream emit or direct stream emit [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L207)
+ - After getting the output task ids, bolts and spouts use the transfer-fn provided by the worker to actually transfer the tuples
+ - Bolt transfer code here: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L429)
+ - Spout transfer code here: [code](https://github.com/apache/storm/blob/0.7.1/src/clj/org/apache/storm/daemon/task.clj#L329)
diff --git a/releases/2.8.8/Metrics.md b/releases/2.8.8/Metrics.md
new file mode 100644
index 0000000000..0d32e263b9
--- /dev/null
+++ b/releases/2.8.8/Metrics.md
@@ -0,0 +1,313 @@
+---
+title: Storm Metrics
+layout: documentation
+documentation: true
+---
+Storm exposes a metrics interface to report summary statistics across the full topology.
+The numbers you see on the UI come from some of these built in metrics, but are reported through the worker heartbeats instead of through the IMetricsConsumer described below.
+
+If you are looking for cluster wide monitoring please see [Cluster Metrics](ClusterMetrics.html).
+
+### Metric Types
+
+Metrics have to implement [`IMetric`]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/IMetric.java) which contains just one method, `getValueAndReset` -- do any remaining work to find the summary value, and reset back to an initial state. For example, the MeanReducer divides the running total by its running count to find the mean, then initializes both values back to zero.
+
+Storm gives you these metric types:
+
+* [AssignableMetric]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/AssignableMetric.java) -- set the metric to the explicit value you supply. Useful if it's an external value or in the case that you are already calculating the summary statistic yourself.
+* [CombinedMetric]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/CombinedMetric.java) -- generic interface for metrics that can be updated associatively.
+* [CountMetric]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/CountMetric.java) -- a running total of the supplied values. Call `incr()` to increment by one, `incrBy(n)` to add/subtract the given number.
+ - [MultiCountMetric]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/MultiCountMetric.java) -- a hashmap of count metrics.
+* [ReducedMetric]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/ReducedMetric.java)
+ - [MeanReducer]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/MeanReducer.java) -- track a running average of values given to its `reduce()` method. (It accepts `Double`, `Integer` or `Long` values, and maintains the internal average as a `Double`.) Despite his reputation, the MeanReducer is actually a pretty nice guy in person.
+ - [MultiReducedMetric]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/MultiReducedMetric.java) -- a hashmap of reduced metrics.
+
+Be aware that even though `getValueAndReset` can return an object returning any object makes it very difficult for an `IMetricsConsumer` to know how to translate it into something usable. Also note that because it is sent to the `IMetricsConsumer` as a part of a tuple the values returned need to be able to be [serialized](Serialization.html) by your topology.
+
+### Metrics Consumer
+
+You can listen and handle the topology metrics via registering Metrics Consumer to your topology.
+
+To register metrics consumer to your topology, add to your topology's configuration like:
+
+```java
+conf.registerMetricsConsumer(org.apache.storm.metric.LoggingMetricsConsumer.class, 1);
+```
+
+You can refer [Config#registerMetricsConsumer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html#registerMetricsConsumer-java.lang.Class-) and overloaded methods from javadoc.
+
+Otherwise edit the storm.yaml config file:
+
+```yaml
+topology.metrics.consumer.register:
+ - class: "org.apache.storm.metric.LoggingMetricsConsumer"
+ parallelism.hint: 1
+ - class: "org.apache.storm.metric.HttpForwardingMetricsConsumer"
+ parallelism.hint: 1
+ argument: "http://example.com:8080/metrics/my-topology/"
+```
+
+Storm adds a MetricsConsumerBolt to your topology for each class in the `topology.metrics.consumer.register` list. Each MetricsConsumerBolt subscribes to receive metrics from all tasks in the topology. The parallelism for each Bolt is set to `parallelism.hint` and `component id` for that Bolt is set to `__metrics_<metrics consumer class name>`. If you register the same class name more than once, postfix `#<sequence number>` is appended to component id.
+
+Storm provides some built-in metrics consumers for you to try out to see which metrics are provided in your topology.
+
+* [`LoggingMetricsConsumer`]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/LoggingMetricsConsumer.java) -- listens for all metrics and dumps them to log file with TSV (Tab Separated Values).
+* [`HttpForwardingMetricsConsumer`]({{page.git-blob-base}}/examples/storm-loadgen/src/main/java/org/apache/storm/loadgen/HttpForwardingMetricsConsumer.java) -- listens for all metrics and POSTs them serialized to a configured URL via HTTP. Storm also provides [`HttpForwardingMetricsServer`]({{page.git-blob-base}}/storm-core/src/jvm/org/apache/storm/metric/HttpForwardingMetricsServer.java) as abstract class so you can extend this class and run as a HTTP server, and handle metrics sent by [...]
+
+Also, Storm exposes the interface [`IMetricsConsumer`]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/metric/api/IMetricsConsumer.java) for implementing Metrics Consumer so you can create custom metrics consumers and attach to their topologies, or use other great implementation of Metrics Consumers provided by Storm community. Some of examples are [versign/storm-graphite](https://github.com/verisign/storm-graphite), and [storm-metrics-statsd](https://github.com/endgameinc/s [...]
+
+When you implement your own metrics consumer, `argument` is passed to Object when [IMetricsConsumer#prepare](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/metric/api/IMetricsConsumer.html#prepare-java.util.Map-java.lang.Object-org.apache.storm.task.TopologyContext-org.apache.storm.task.IErrorReporter-) is called, so you need to infer the Java type of configured value on yaml, and do explicit type casting.
+
+Please keep in mind that MetricsConsumerBolt is just a kind of Bolt, so whole throughput of the topology will go down when registered metrics consumers cannot keep up handling incoming metrics, so you may want to take care of those Bolts like normal Bolts. One of idea to avoid this is making your implementation of Metrics Consumer as `non-blocking` fashion.
+
+
+### Build your own metric (task level)
+
+You can measure your own metric by registering `IMetric` to Metric Registry.
+
+Suppose we would like to measure execution count of Bolt#execute. Let's start with defining metric instance. CountMetric seems to fit our use case.
+
+```java
+private transient CountMetric countMetric;
+```
+
+Notice we define it as transient. IMertic is not Serializable so we defined as transient to avoid any serialization issues.
+
+Next, let's initialize and register the metric instance.
+
+```java
+@Override
+public void prepare(Map conf, TopologyContext context, OutputCollector collector) {
+ // other initialization here.
+ countMetric = new CountMetric();
+ context.registerMetric("execute_count", countMetric, 60);
+}
+```
+
+The meaning of first and second parameters are straightforward, metric name and instance of IMetric. Third parameter of [TopologyContext#registerMetric](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/task/TopologyContext.html#registerMetric-java.lang.String-T-int-) is the period (seconds) to publish and reset the metric.
+
+Last, let's increment the value when Bolt.execute() is executed.
+
+```java
+public void execute(Tuple input) {
+ countMetric.incr();
+ // handle tuple here.
+}
+```
+
+Note that sample rate for topology metrics is not applied to custom metrics since we're calling incr() ourselves.
+
+Done! `countMetric.getValueAndReset()` is called every 60 seconds as we registered as period, and pair of ("execute_count", value) will be pushed to MetricsConsumer.
+
+### Build your own metrics (worker level)
+
+You can register your own worker level metrics by adding them to `Config.WORKER_METRICS` for all workers in cluster, or `Config.TOPOLOGY_WORKER_METRICS` for all workers in specific topology.
+
+For example, we can add `worker.metrics` to storm.yaml in cluster,
+
+```yaml
+worker.metrics:
+ metricA: "aaa.bbb.ccc.ddd.MetricA"
+ metricB: "aaa.bbb.ccc.ddd.MetricB"
+ ...
+```
+
+or put `Map<String, String>` (metric name, metric class name) with key `Config.TOPOLOGY_WORKER_METRICS` to config map.
+
+There are some restrictions for worker level metric instances:
+
+A) Metrics for worker level should be kind of gauge since it is initialized and registered from SystemBolt and not exposed to user tasks.
+
+B) Metrics will be initialized with default constructor, and no injection for configuration or object will be performed.
+
+C) Bucket size (seconds) for metrics is fixed to `Config.TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS`.
+
+### Builtin Metrics
+
+The [builtin metrics]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/daemon/metrics/BuiltinMetricsUtil.java) instrument Storm itself.
+
+[BuiltinMetricsUtil.java]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/daemon/metrics/BuiltinMetricsUtil.java) sets up data structures for the built-in metrics, and facade methods that the other framework components can use to update them. The metrics themselves are calculated in the calling code -- see for example [`ackSpoutMsg`]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/executor/Executor.java).
+
+#### Reporting Rate
+
+The rate at which built in metrics are reported is configurable through the `topology.builtin.metrics.bucket.size.secs` config. If you set this too low it can overload the consumers,
+so please use caution when modifying it.
+
+#### Tuple Counting Metrics
+
+There are several different metrics related to counting what a bolt or spout does to a tuple. These include things like emitting, transferring, acking, and failing of tuples.
+
+In general all of these tuple count metrics are randomly sub-sampled unless otherwise stated. This means that the counts you see both on the UI and from the built in metrics are not necessarily exact. In fact by default we sample only 5% of the events and estimate the total number of events from that. The sampling percentage is configurable per topology through the `topology.stats.sample.rate` config. Setting it to 1.0 will make the counts exact, but be aware that the more events we [...]
+
+The tuple counting metric names contain `"${stream_name}"` or `"${upstream_component}:${stream_name}"`. The former is used for all spout metrics and for outgoing bolt metrics (`__emit-count` and `__transfer-count`). The latter is used for bolt metrics that deal with incoming tuples.
+
+So for a word count topology the count bolt might show something like the following for the `__ack-count` metric
+
+```
+ "__ack-count-split:default": 80080
+```
+
+But the spout instead would show something like the following for the `__ack-count` metric.
+
+```
+ "__ack-count-default": 12500
+```
+
+
+##### `__ack-count`
+
+For bolts it is the number of incoming tuples that had the `ack` method called on them. For spouts it is the number of tuples trees that were fully acked. See Guaranteeing Message Processing[](Guaranteeing-message-processing.html) for more information about what a tuple tree is. If acking is disabled this metric is still reported, but it is not really meaningful.
+
+##### `__fail-count`
+
+For bolts this is the number of incoming tuples that had the `fail` method called on them. For spouts this is the number of tuple trees that failed. Tuple trees may fail from timing out or because a bolt called fail on it. The two are not separated out by this metric.
+
+##### `__emit-count`
+
+This is the total number of times the `emit` method was called to send a tuple. This is the same for both bolts and spouts.
+
+##### `__transfer-count`
+
+This is the total number of tuples transferred to a downstream bolt/spout for processing. This number will not always match `__emit_count`. If nothing is registered to receive a tuple down stream the number will be 0 even if tuples were emitted. Similarly if there are multiple down stream consumers it may be a multiple of the number emitted. The grouping also can play a role if it sends the tuple to multiple instances of a single bolt down stream.
+
+##### `__execute-count`
+
+This count metric is bolt specific. It counts the number of times that a bolt's `execute` method was called.
+
+#### Tuple Latency Metrics
+
+Similar to the tuple counting metrics storm also collects average latency metrics for bolts and spouts. These follow the same structure as the bolt/spout maps and are sub-sampled in the same way as well. In all cases the latency is measured in milliseconds.
+
+##### `__complete-latency`
+
+The complete latency is just for spouts. It is the average amount of time it took for `ack` or `fail` to be called for a tuple after it was emitted. If acking is disabled this metric is likely to be blank or 0 for all values, and should be ignored.
+
+##### `__execute-latency`
+
+This is just for bolts. It is the average amount of time that the bolt spent in the call to the `execute` method. The higher this gets, the lower the throughput of tuples per bolt instance.
+
+##### `__process-latency`
+
+This is also just for bolts. It is the average amount of time between when `execute` was called to start processing a tuple, to when it was acked or failed by the bolt. If your bolt is a very simple bolt and the processing is synchronous then `__process-latency` and `__execute-latency` should be very close to one another, with process latency being slightly smaller. If you are doing a join or have asynchronous processing then it may take a while for a tuple to be acked so the process [...]
+
+##### `__skipped-max-spout-ms`
+
+This metric records how much time a spout was idle because more tuples than `topology.max.spout.pending` were still outstanding. This is the total time in milliseconds, not the average amount of time and is not sub-sampled.
+
+
+##### `__skipped-backpressure-ms`
+
+This metric records how much time a spout was idle because back-pressure indicated that downstream queues in the topology were too full. This is the total time in milliseconds, not the average amount of time and is not sub-sampled. This is similar to skipped-throttle-ms in Storm 1.x.
+
+##### `__backpressure-last-overflow-count`
+
+This metric indicates the overflow count last time BP status was sent, with a minimum value of 1 if a task has backpressure on.
+
+##### `skipped-inactive-ms`
+
+This metric records how much time a spout was idle because the topology was deactivated. This is the total time in milliseconds, not the average amount of time and is not sub-sampled.
+
+#### Error Reporting Metrics
+
+Storm also collects error reporting metrics for bolts and spouts.
+
+##### `__reported-error-count`
+
+This metric records how many errors were reported by a spout/bolt. It is the total number of times the `reportError` method was called.
+
+#### Queue Metrics
+
+Each bolt or spout instance in a topology has a receive queue. Each worker also has a worker transfer queue for sending messages to other workers. All of these have metrics that are reported.
+
+The receive queue metrics are reported under the `receive_queue` name. The metrics for the queue that sends messages to other workers is under the `worker-transfer-queue` metric name for the system bolt (`__system`).
+
+These queues report the following metrics:
+
+```
+{
+ "arrival_rate_secs": 1229.1195171893523,
+ "overflow": 0,
+ "sojourn_time_ms": 2.440771591407277,
+ "capacity": 1024,
+ "population": 19,
+ "pct_full": "0.018".
+ "insert_failures": "0",
+ "dropped_messages": "0"
+}
+```
+
+`arrival_rate_secs` is an estimation of the number of tuples that are inserted into the queue in one second, although it is actually the dequeue rate.
+The `sojourn_time_ms` is calculated from the arrival rate and is an estimate of how many milliseconds each tuple sits in the queue before it is processed.
+
+The queue has a set maximum number of entries. If the regular queue fills up an overflow queue takes over. The number of tuples stored in this overflow section are represented by the `overflow` metric. Note that an overflow queue is only used for executors to receive tuples from remote workers. It doesn't apply to intra-worker tuple transfer.
+
+`capacity` is the maximum number of entries in the queue. `population` is the number of entries currently filled in the queue. 'pct_full' tracks the percentage of capacity in use.
+
+'insert_failures' tracks the number of failures inserting into the queue. 'dropped_messages' tracks messages dropped due to the overflow queue being full.
+
+#### System Bolt (Worker) Metrics
+
+The System Bolt `__system` provides lots of metrics for different worker wide things. The one metric not described here is the `__transfer` queue metric, because it fits with the other disruptor metrics described above.
+
+Be aware that the `__system` bolt is an actual bolt so regular bolt metrics described above also will be reported for it.
+
+##### Receive (NettyServer)
+`__recv-iconnection` reports stats for the netty server on the worker. This is what gets messages from other workers. It is of the form
+
+```
+{
+ "dequeuedMessages": 0,
+ "enqueued": {
+ "/127.0.0.1:49952": 389951
+ }
+}
+```
+
+`dequeuedMessages` is a throwback to older code where there was an internal queue between the server and the bolts/spouts. That is no longer the case and the value can be ignored.
+`enqueued` is a map between the address of the remote worker and the number of tuples that were sent from it to this worker.
+
+##### Send (Netty Client)
+
+The `__send-iconnection` metrics report information about all of the clients for this worker. They are named __send-iconnection-METRIC_TYPE-HOST:PORT for a given Client that is
+connected to a worker with the given host/port. These metrics can be disabled by setting topology.enable.send.iconnection.metrics to false.
+
+The metric types reported for each client are:
+
+ * `reconnects` the number of reconnections that have happened.
+ * `pending` the number of messages that have not been sent. (This corresponds to messages, not tuples)
+ * `sent` the number of messages that have been sent. (This is messages not tuples)
+ * `lostOnSend`. This is the number of messages that were lost because of connection issues. (This is messages not tuples).
+
+##### JVM Memory
+
+JVM memory usage is reported through `memory.non-heap` for off heap memory, `memory.heap` for on heap memory and `memory.total` for combined values. These values come from the [MemoryUsage](https://docs.oracle.com/javase/8/docs/api/index.html?java/lang/management/MemoryUsage.html) mxbean. Each of the metrics are reported as a map with the following keys, and values returned by the corresponding java code.
+
+| Key | Corresponding Code |
+|--------|--------------------|
+| `max` | `memUsage.getMax()` |
+| `committed` | `memUsage.getCommitted()` |
+| `init` | `memUsage.getInit()` |
+| `used` | `memUsage.getUsed()` |
+| `usage` | `Ratio.of(memUsage.getUsed(), memUsage.getMax())` |
+
+##### JVM Garbage Collection
+
+The exact GC metric name depends on the garbage collector that your worker uses. The data is all collected from `ManagementFactory.getGarbageCollectorMXBeans()` and the name of the metrics is `"GC"` followed by the name of the returned bean with white space removed. The reported metrics are just
+
+* `count` the number of gc events that happened and
+* `time` the total number of milliseconds that were spent doing gc.
+
+Please refer to the [JVM documentation](https://docs.oracle.com/javase/8/docs/api/java/lang/management/ManagementFactory.html#getGarbageCollectorMXBeans--) for more details.
+
+##### JVM Misc
+
+* There are metrics prefixed with `threads` providing the number of threads, daemon threads, blocked and deadlocked threads.
+
+##### Other worker metrics
+
+* `doHeartbeat-calls` is a meter that indicates the rate the worker is performing heartbeats.
+* `newWorkerEvent` is 1 when a worker is first started and 0 all other times. This can be used to tell when a worker has crashed and is restarted.
+* `startTimeSecs` is when the worker started in seconds since the epoch
+* `uptimeSecs` reports the number of seconds the worker has been up for
+* `workerCpuUsage` reports the cpu usage of the worker as a percentage of cores. 1.0 indicates 1 cpu core.
+
diff --git a/releases/2.8.8/Multilang-protocol.md b/releases/2.8.8/Multilang-protocol.md
new file mode 100644
index 0000000000..77cb9314cb
--- /dev/null
+++ b/releases/2.8.8/Multilang-protocol.md
@@ -0,0 +1,337 @@
+---
+title: Multi-Lang Protocol
+layout: documentation
+documentation: true
+---
+This page explains the multilang protocol as of Storm 0.7.1. Versions prior to 0.7.1 used a somewhat different protocol, documented [here](Storm-multi-language-protocol-(versions-0.7.0-and-below\).html).
+
+# Storm Multi-Language Protocol
+
+## Supported Languages
+
+Storm Multi-Language has implementation in the following languages:
+
+- [JavaScript](https://github.com/apache/storm/tree/master/storm-multilang/javascript)
+- [Python](https://github.com/apache/storm/tree/master/storm-multilang/python)
+- [Ruby](https://github.com/apache/storm/tree/master/storm-multilang/ruby)
+
+Third party libraries are available for the following languages:
+
+- [c# (on .net core 2.0)](https://github.com/Azure/net-storm-multilang-adapter)
+
+## Shell Components
+
+Support for multiple languages is implemented via the ShellBolt,
+ShellSpout, and ShellProcess classes. These classes implement the
+IBolt and ISpout interfaces and the protocol for executing a script or
+program via the shell using Java's ProcessBuilder class.
+
+### Packaging of shell scripts
+
+By default the ShellProcess assumes that your code is packaged inside of your topology jar under the resources subdirectory of your jar and by default will change the current working directory of
+the executable process to be that resources directory extracted from the jar.
+A jar file does not store permissions of the files in it. This includes the execute bit that would allow a shell script to be loaded and run by the operating systme.
+As such in most examples the scripts are of the form `python3 mybolt.py` because the Python executable is already on the supervisor and mybolt is packaged in the resources directory of the jar.
+
+If you want to package something more complicated, like a new version of Python itself, you need to instead use the blob store for this and a `.tgz` archive that does support permissions.
+
+See the docs on the [Blob Store](distcache-blobstore.html) for more details on how to ship a jar.
+
+To make a ShellBolt/ShellSpout work with executables + scripts shipped in the blob store dist cache add
+
+```
+changeChildCWD(false);
+```
+
+in the constructor of your ShellBolt/ShellSpout. The shell command will then be relative to the cwd of the worker. Where the sym-links to the resources are.
+
+So if I shipped python with a symlink named `newPython` and a python ShellSpout I shipped into `shell_spout.py` I would have a something like
+
+```
+public MyShellSpout() {
+ super("./newPython/bin/python3", "./shell_spout.py");
+ changeChildCWD(false);
+}
+```
+
+## Output fields
+
+Output fields are part of the Thrift definition of the topology. This means that when you multilang in Java, you need to create a bolt that extends ShellBolt, implements IRichBolt, and declare the fields in `declareOutputFields` (similarly for ShellSpout).
+
+You can learn more about this on [Concepts](Concepts.html)
+
+## Protocol Preamble
+
+A simple protocol is implemented via the STDIN and STDOUT of the
+executed script or program. All data exchanged with the process is
+encoded in JSON, making support possible for pretty much any language.
+
+# Packaging Your Stuff
+
+To run a shell component on a cluster, the scripts that are shelled
+out to must be in the `resources/` directory within the jar submitted
+to the master.
+
+However, during development or testing on a local machine, the resources
+directory just needs to be on the classpath.
+
+## The Protocol
+
+Notes:
+
+* Both ends of this protocol use a line-reading mechanism, so be sure to
+trim off newlines from the input and to append them to your output.
+* All JSON inputs and outputs are terminated by a single line containing "end". Note that this delimiter is not itself JSON encoded.
+* The bullet points below are written from the perspective of the script writer's
+STDIN and STDOUT.
+
+### Initial Handshake
+
+The initial handshake is the same for both types of shell components:
+
+* STDIN: Setup info. This is a JSON object with the Storm configuration, a PID directory, and a topology context, like this:
+
+```
+{
+ "conf": {
+ "topology.message.timeout.secs": 3,
+ // etc
+ },
+ "pidDir": "...",
+ "context": {
+ "task->component": {
+ "1": "example-spout",
+ "2": "__acker",
+ "3": "example-bolt1",
+ "4": "example-bolt2"
+ },
+ "taskid": 3,
+ // Everything below this line is only available in Storm 0.10.0+
+ "componentid": "example-bolt"
+ "stream->target->grouping": {
+ "default": {
+ "example-bolt2": {
+ "type": "SHUFFLE"}}},
+ "streams": ["default"],
+ "stream->outputfields": {"default": ["word"]},
+ "source->stream->grouping": {
+ "example-spout": {
+ "default": {
+ "type": "FIELDS",
+ "fields": ["word"]
+ }
+ }
+ }
+ "source->stream->fields": {
+ "example-spout": {
+ "default": ["word"]
+ }
+ }
+ }
+}
+```
+
+Your script should create an empty file named with its PID in this directory. e.g.
+the PID is 1234, so an empty file named 1234 is created in the directory. This
+file lets the supervisor know the PID so it can shutdown the process later on.
+
+As of Storm 0.10.0, the context sent by Storm to shell components has been
+enhanced substantially to include all aspects of the topology context available
+to JVM components. One key addition is the ability to determine a shell
+component's source and targets (i.e., inputs and outputs) in the topology via
+the `stream->target->grouping` and `source->stream->grouping` dictionaries. At
+the innermost level of these nested dictionaries, groupings are represented as
+a dictionary that minimally has a `type` key, but can also have a `fields` key
+to specify which fields are involved in a `FIELDS` grouping.
+
+* STDOUT: Your PID, in a JSON object, like `{"pid": 1234}`. The shell component will log the PID to its log.
+
+What happens next depends on the type of component:
+
+### Spouts
+
+Shell spouts are synchronous. The rest happens in a while(true) loop:
+
+* STDIN: Either a next, ack, activate, deactivate or fail command.
+
+"next" is the equivalent of ISpout's `nextTuple`. It looks like:
+
+```
+{"command": "next"}
+```
+
+"ack" looks like:
+
+```
+{"command": "ack", "id": "1231231"}
+```
+
+"activate" is the equivalent of ISpout's `activate`:
+```
+{"command": "activate"}
+```
+
+"deactivate" is the equivalent of ISpout's `deactivate`:
+```
+{"command": "deactivate"}
+```
+
+"fail" looks like:
+
+```
+{"command": "fail", "id": "1231231"}
+```
+
+* STDOUT: The results of your spout for the previous command. This can
+ be a sequence of emits and logs.
+
+An emit looks like:
+
+```
+{
+ "command": "emit",
+ // The id for the tuple. Leave this out for an unreliable emit. The id can
+ // be a string or a number.
+ "id": "1231231",
+ // The id of the stream this tuple was emitted to. Leave this empty to emit to default stream.
+ "stream": "1",
+ // If doing an emit direct, indicate the task to send the tuple to
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["field1", 2, 3]
+}
+```
+
+If not doing an emit direct, you will immediately receive the task ids to which the tuple was emitted on STDIN as a JSON array.
+
+A "log" will log a message in the worker log. It looks like:
+
+```
+{
+ "command": "log",
+ // the message to log
+ "msg": "hello world!"
+}
+```
+
+* STDOUT: a "sync" command ends the sequence of emits and logs. It looks like:
+
+```
+{"command": "sync"}
+```
+
+After you sync, ShellSpout will not read your output until it sends another next, ack, or fail command.
+
+Note that, similarly to ISpout, all of the spouts in the worker will be locked up after a next, ack, or fail, until you sync. Also like ISpout, if you have no tuples to emit for a next, you should sleep for a small amount of time before syncing. ShellSpout will not automatically sleep for you.
+
+
+### Bolts
+
+The shell bolt protocol is asynchronous. You will receive tuples on STDIN as soon as they are available, and you may emit, ack, and fail, and log at any time by writing to STDOUT, as follows:
+
+* STDIN: A tuple! This is a JSON encoded structure like this:
+
+```
+{
+ // The tuple's id - this is a string to support languages lacking 64-bit precision
+ "id": "-6955786537413359385",
+ // The id of the component that created this tuple
+ "comp": "1",
+ // The id of the stream this tuple was emitted to
+ "stream": "1",
+ // The id of the task that created this tuple
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["snow white and the seven dwarfs", "field2", 3]
+}
+```
+
+* STDOUT: An ack, fail, emit, or log. Emits look like:
+
+```
+{
+ "command": "emit",
+ // The ids of the tuples this output tuples should be anchored to
+ "anchors": ["1231231", "-234234234"],
+ // The id of the stream this tuple was emitted to. Leave this empty to emit to default stream.
+ "stream": "1",
+ // If doing an emit direct, indicate the task to send the tuple to
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["field1", 2, 3]
+}
+```
+
+If not doing an emit direct, you will receive the task ids to which
+the tuple was emitted on STDIN as a JSON array. Note that, due to the
+asynchronous nature of the shell bolt protocol, when you read after
+emitting, you may not receive the task ids. You may instead read the
+task ids for a previous emit or a new tuple to process. You will
+receive the task id lists in the same order as their corresponding
+emits, however.
+
+An ack looks like:
+
+```
+{
+ "command": "ack",
+ // the id of the tuple to ack
+ "id": "123123"
+}
+```
+
+A fail looks like:
+
+```
+{
+ "command": "fail",
+ // the id of the tuple to fail
+ "id": "123123"
+}
+```
+
+A "log" will log a message in the worker log. It looks like:
+
+```
+{
+ "command": "log",
+ // the message to log
+ "msg": "hello world!"
+}
+```
+
+* Note that, as of version 0.7.1, there is no longer any need for a
+ shell bolt to 'sync'.
+
+### Handling Heartbeats (0.9.3 and later)
+
+As of Storm 0.9.3, heartbeats have been between ShellSpout/ShellBolt and their
+multi-lang subprocesses to detect hanging/zombie subprocesses. Any libraries
+for interfacing with Storm via multi-lang must take the following actions
+regarding hearbeats:
+
+#### Spout
+
+Shell spouts are synchronous, so subprocesses always send `sync` commands at the
+end of `next()`, so you should not have to do much to support heartbeats for
+spouts. That said, you must not let subprocesses sleep more than the worker
+timeout during `next()`.
+
+#### Bolt
+
+Shell bolts are asynchronous, so a ShellBolt will send heartbeat tuples to its
+subprocess periodically. Heartbeat tuple looks like:
+
+```
+{
+ "id": "-6955786537413359385",
+ "comp": "1",
+ "stream": "__heartbeat",
+ // this shell bolt's system task id
+ "task": -1,
+ "tuple": []
+}
+```
+
+When subprocess receives heartbeat tuple, it must send a `sync` command back to
+ShellBolt.
diff --git a/releases/2.8.8/NUMA.md b/releases/2.8.8/NUMA.md
new file mode 100644
index 0000000000..9ba3225fd4
--- /dev/null
+++ b/releases/2.8.8/NUMA.md
@@ -0,0 +1,97 @@
+---
+title: NUMA Support
+layout: documentation
+documentation: true
+---
+
+# Table of Contents
+1. [Introduction](#Introduction)
+2. [Architecture](#Architecture)
+3. [Resource Isolation Interface interaction](#RII-interaction)
+ 1. [CgroupManager](#Setting-Memory-Requirement)
+ 2. [DockerManager](#Setting-Shared-Memory)
+ 3. [RuncLibManager](#Setting-CPU-Requirement)
+4. [Configuring NUMA](#Configuring-NUMA)
+
+<div id='Introduction'/>
+
+## Introduction
+
+Non Uniform Memory Access ([NUMA](https://www.cc.gatech.edu/~echow/ipcc/hpc-course/HPC-numa.pdf)) is a new system architecture
+where the hosts resources are grouped by cores and memory in NUMA zones. Storm supports isolating/pinning worker processes to specific
+NUMA zones via the supervisor to take advantage of this resource isolation and avoid the penalty of using cross zone bus transfers
+<div id='Architecture'/>
+
+## Architecture
+
+Once Storm supervisors are configured for NUMA (see the section below) they now heartbeat multiple heartbeats - one for each NUMA zone.
+Each of these NUMA supervisors has a supervisor id with the same prefixed supervisor id with the NUMA id differentiating them.
+Nimbus, and by extension the scheduler, see these heartbeats and views the supervisor as multiple supervisors - one per configured NUMA zone.
+Nimbus schedules topologies and assignments accordingly. The supervisor reads all assignments with the prefixed assignments and then
+pins each worker to the NUMA zone according to the numa id in the assignment. The pinning depends on the Resource Isolation Interface used
+and is elaborated on in the following section.
+
+<div id='RII-Interaction'/>
+
+### Resource Isolation Interface interaction
+
+Each implementation of the Resource Isolation Interface (RII) should now implement NUMA pinning. The following are the current/soon to be available
+implementations
+
+#### CgroupManager
+
+The CgroupManager prefixes the worker launch command with the numactl command for Linux hosts-
+
+```
+numactl --cpunodebind=<numaId>> --membind=<numaId> <worker launch command>
+```
+
+The worker is then bound to the NUMA zone's CPU cores and memory zone.
+#### DockerManager
+
+Will be updated upon adding Docker support
+
+#### RuncLibManager
+
+Will be updated upon adding Runc support
+
+<div id='Configuring NUMA'/>
+
+### Configuring NUMA
+
+In the Supervisor Config the following settings need to be set
+
+```
+supervisor.numa.meta:
+ "0": # Numa zone id
+ numa.cores: # Cores in NUMA zone (can be determined by using the numastat command)
+ - 0
+ - 1
+ - 2
+ - 3
+ - 4
+ - 5
+ - 12
+ - 13
+ - 14
+ - 15
+ - 16
+
+ numa.generic.resources.map: # Generic Resources in the zone to be used for generic resource scheduling (optional)
+ network.resource.units: 50.0
+
+ numa.memory.mb: 42461 # Size of memory zone
+ numa.ports: # Ports to be assigned to workers pinned to the NUMA zone (this may include ports not specified in SUPERVISOR_SLOTS_PORTS
+ - 6700
+ - 6701
+ - 6702
+ - 6703
+ - 6704
+ - 6705
+ - 6706
+ - 6707
+ - 6708
+ - 6709
+ - 6710
+ - 6711
+```
diff --git a/releases/2.8.8/OCI-support.md b/releases/2.8.8/OCI-support.md
new file mode 100644
index 0000000000..a44959ea1c
--- /dev/null
+++ b/releases/2.8.8/OCI-support.md
@@ -0,0 +1,1710 @@
+---
+title: OCI/Squashfs Runtime
+layout: documentation
+documentation: true
+---
+
+# OCI/Squashfs Runtime for Workers Running in Containers
+
+OCI/Squashfs is a container runtime that allows topologies to run inside docker containers. However, unlike the existing
+Docker runtime, the images are fetched from HDFS rather than from the Docker registry or requiring images to be pre-loaded
+into Docker on each node. Docker does not need to be installed on the nodes in order for this runtime to work.
+
+Note: This has only been tested on RHEL7.
+
+## Motivation
+
+#### Docker runtime drawbacks
+Using the current Docker runtime (see [Docker-support.md](Docker-support.md#Docker-Support) ) has some drawbacks:
+
+##### Docker Daemons Dependency
+
+The Docker daemons `dockerd` and `containerd` must be running on the system in order for the Docker runtime to function.
+And these daemons can get out of sync which could cause nontrivial issues to the containers.
+
+##### Docker Registry Issues at Scale
+
+Using the Docker runtime on a large scale Storm cluster can overwhelm the Docker registry. In practice this requires
+admins to pre-load a Docker image on all the cluster nodes in a controlled fashion before a large job requesting
+the image can run.
+
+##### Image Costs in Time and Space
+
+Docker stores each image layer as a tar.gz archive. In order to use the layer, the compressed archive must be unpacked
+into the node's filesystem. This can consume significant disk space, especially when the reliable image store location
+capacity is relatively small. In addition, unpacking an image layer takes time, especially when the layer is large or
+contains thousands of files. This additional time for unpacking delays container launch beyond the time needed to transfer
+the layer data over the network.
+
+#### OCI/Squashfs Runtime advantages
+
+The OCI/Squashfs runtime avoids the drawback listed above in the following ways.
+
+##### No Docker dependencies on The Node
+
+Docker does not need to be installed on each node, nor is there a dependency on a daemon or service that needs to be started
+by an admin before containers can be launched. All that is required to be present on each node is an OCI-compatible runtime like
+`runc`.
+
+##### Leverages Distributed File Systems For Scale
+
+Image can be fetched via HDFS or other distributed file systems instead of the Docker registry. This prevents a large cluster from
+overwhelming a Docker registry when a big topology causes all of the nodes to request an image at once. This also allows large clusters
+to run topologies more dynamically, as images would not need to be pre-loaded by admins on each node to prevent a large Docker registry
+image request storm.
+
+##### Smaller, Faster images on The Node
+
+The new runtime handles layer localization directly, so layer formats other than tar archive can be supported. For example, each image layer
+can be converted to squashfs images as part of copying the layers to HDFS. squashfs is a file system optimized for running directly on a
+compressed image. With squashfs layers the layer data can remain compressed on the node saving disk space. Container launch after layer
+localization is also faster, as the layers no longer need to be unpacked into a directory to become usable.
+
+
+## Prerequisite
+
+First you need to use the`docker-to-squash.py` script to download docker images and configs, convert layers to squashfs files and put them to a directory in HDFS, for example
+
+```bash
+python3 docker-to-squash.py pull-build-push-update --hdfs-root hdfs://hostname:port/containers \
+ docker.xxx.com:4443/hadoop-user-images/storm/rhel7:20201202-232133,storm/rhel7:dev_current --log DEBUG --bootstrap
+```
+
+With this command, all the layers belonging to this image will be converted to squashfs files and be placed under `./layers` directory
+under the directory specified by `--hdfs-root`;
+the manifest of this image will be placed under `./manifests` directory with the name as the sha256 value of the manifest content;
+the config of this image will be placed under `./config` directory with the name as the sha256 value of the config content;
+the mapping from the image tag to the sha256 value of the manifest will be written to the "./image-tag-to-manifest-file".
+
+Note that `--hdfs-root` can be any directory on HDFS, as long as it matches with the `storm.oci.image.hdfs.toplevel.dir` config.
+
+##### Example
+
+For example, the directory structure is like this:
+
+```bash
+-bash-4.2$ hdfs dfs -ls /containers/*
+Found 1 items
+-r--r--r-- 3 hdfsqa hadoop 7877 2020-12-04 14:29 /containers/config/ef1ff2c7167a1a6cd01e106f51b84a4d400611ba971c53cbc28de7919515ca4e
+-r--r--r-- 3 hdfsqa hadoop 160 2020-12-04 14:30 /containers/image-tag-to-hash
+Found 7 items
+-r--r--r-- 3 hdfsqa hadoop 84697088 2020-12-04 14:28 /containers/layers/152ee1d2cccea9dfe6393d2bdf9d077b67616b2b417b25eb74fc5ffaadcb96f5.sqsh
+-r--r--r-- 3 hdfsqa hadoop 545267712 2020-12-04 14:28 /containers/layers/18ee671016a1bf3ecab07395d93c2cbecd352d59c497a1551e2074d64e1098d9.sqsh
+-r--r--r-- 3 hdfsqa hadoop 12906496 2020-10-06 15:24 /containers/layers/1b73e9433ecca0a6bb152bd7525f2b7c233484d51c24f8a6ba483d5cfd3035dc.sqsh
+-r--r--r-- 3 hdfsqa hadoop 4096 2020-12-04 14:29 /containers/layers/344224962010c03c9ca1f11a9bff0dfcc296ac46d0a55e4ff30a0ad13b9817af.sqsh
+-r--r--r-- 3 hdfsqa hadoop 26091520 2020-10-06 15:22 /containers/layers/3692c3483ef6516fba685b316448e8aaf0fc10bb66818116edc8e5e6800076c7.sqsh
+-r--r--r-- 3 hdfsqa hadoop 4096 2020-12-04 14:29 /containers/layers/8710a3d72f75b45c48ab6b9b67eb6d77caea3dac91a0c30e0831f591cba4887e.sqsh
+-r--r--r-- 3 hdfsqa hadoop 121122816 2020-10-06 15:23 /containers/layers/ea067172a7138f035d89a5c378db6d66c1581d98b0497b21f256e04c3d2b5303.sqsh
+Found 1 items
+-r--r--r-- 3 hdfsqa hadoop 1793 2020-12-04 14:29 /containers/manifests/26fd443859325d5911f3be5c5e231dddca88ee0d526456c0c92dd794148d8585
+```
+
+The `image-tag-to-manifest-file`:
+```bash
+-bash-4.2$ hdfs dfs -cat /containers/image-tag-to-hash
+storm/rhel7:dev_current:26fd443859325d5911f3be5c5e231dddca88ee0d526456c0c92dd794148d8585#docker.xxx.com:4443/hadoop-user-images/storm/rhel7:20201202-232133
+```
+
+The manifest file `26fd443859325d5911f3be5c5e231dddca88ee0d526456c0c92dd794148d8585`:
+```json
+{
+ "schemaVersion": 2,
+ "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
+ "config": {
+ "mediaType": "application/vnd.docker.container.image.v1+json",
+ "size": 7877,
+ "digest": "sha256:ef1ff2c7167a1a6cd01e106f51b84a4d400611ba971c53cbc28de7919515ca4e"
+ },
+ "layers": [
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 26858854,
+ "digest": "sha256:3692c3483ef6516fba685b316448e8aaf0fc10bb66818116edc8e5e6800076c7"
+ },
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 123300113,
+ "digest": "sha256:ea067172a7138f035d89a5c378db6d66c1581d98b0497b21f256e04c3d2b5303"
+ },
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 12927624,
+ "digest": "sha256:1b73e9433ecca0a6bb152bd7525f2b7c233484d51c24f8a6ba483d5cfd3035dc"
+ },
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 567401434,
+ "digest": "sha256:18ee671016a1bf3ecab07395d93c2cbecd352d59c497a1551e2074d64e1098d9"
+ },
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 85748864,
+ "digest": "sha256:152ee1d2cccea9dfe6393d2bdf9d077b67616b2b417b25eb74fc5ffaadcb96f5"
+ },
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 186,
+ "digest": "sha256:344224962010c03c9ca1f11a9bff0dfcc296ac46d0a55e4ff30a0ad13b9817af"
+ },
+ {
+ "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
+ "size": 156,
+ "digest": "sha256:8710a3d72f75b45c48ab6b9b67eb6d77caea3dac91a0c30e0831f591cba4887e"
+ }
+ ]
+}
+```
+
+And the config file `ef1ff2c7167a1a6cd01e106f51b84a4d400611ba971c53cbc28de7919515ca4e` (some of the content is omitted):
+```json
+{
+ "architecture": "amd64",
+ "config": {
+ "Hostname": "",
+ "Domainname": "",
+ "User": "root",
+ "AttachStdin": false,
+ "AttachStdout": false,
+ "AttachStderr": false,
+ "Tty": false,
+ "OpenStdin": false,
+ "StdinOnce": false,
+ "Env": [
+ "X_SCLS=rh-git218",
+ "LD_LIBRARY_PATH=/opt/rh/httpd24/root/usr/lib64",
+ "PATH=/opt/rh/rh-git218/root/usr/bin:/home/y/bin64:/home/y/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/y/share/yjava_jdk/java/bin",
+ "PERL5LIB=/opt/rh/rh-git218/root/usr/share/perl5/vendor_perl",
+ "LANG=en_US.UTF-8",
+ "LANGUAGE=en_US:en",
+ "LC_ALL=en_US.UTF-8",
+ "JAVA_HOME=/home/y/share/yjava_jdk/java"
+ ],
+ "Cmd": [
+ "/bin/bash"
+ ],
+ "Image": "sha256:6977cd0735c96d14248e834f775373e40230c134b70f10163c05ce6c6c8873ca",
+ "Volumes": null,
+ "WorkingDir": "",
+ "Entrypoint": null,
+ "OnBuild": null,
+ "Labels": {
+ "name": "xxxxx"
+ }
+ },
+ "container": "344ff1084dea3e0501a0d426e52c43cd589d6b29f33ab0915b7be8906b9aec41",
+ "container_config": {
+ "Hostname": "344ff1084dea",
+ "Domainname": "",
+ "User": "root",
+ "AttachStdin": false,
+ "AttachStdout": false,
+ "AttachStderr": false,
+ "Tty": false,
+ "OpenStdin": false,
+ "StdinOnce": false,
+ "Env": [
+ "X_SCLS=rh-git218",
+ "LD_LIBRARY_PATH=/opt/rh/httpd24/root/usr/lib64",
+ "PATH=/opt/rh/rh-git218/root/usr/bin:/home/y/bin64:/home/y/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/y/share/yjava_jdk/java/bin",
+ "PERL5LIB=/opt/rh/rh-git218/root/usr/share/perl5/vendor_perl",
+ "LANG=en_US.UTF-8",
+ "LANGUAGE=en_US:en",
+ "LC_ALL=en_US.UTF-8",
+ "JAVA_HOME=/home/y/share/yjava_jdk/java"
+ ],
+ "Cmd": [
+ "/bin/sh",
+ "-c"
+ ],
+ "Image": "sha256:6977cd0735c96d14248e834f775373e40230c134b70f10163c05ce6c6c8873ca",
+ "Volumes": null,
+ "WorkingDir": "",
+ "Entrypoint": null,
+ "OnBuild": null,
+ "Labels": {
+ "name": "xxxxx"
+ }
+ },
+ "created": "2020-12-02T23:25:47.354704574Z",
+ "docker_version": "19.03.8",
+ "history": [
+ {
+ "created": "2020-02-18T21:43:36.934503462Z",
+ "created_by": "/bin/sh"
+ },
+ {
+ "created": "2020-02-18T21:45:05.729764427Z",
+ "created_by": "/bin/sh"
+ },
+ {
+ "created": "2020-02-18T21:46:36.638896031Z",
+ "created_by": "/bin/sh"
+ },
+ {
+ "created": "2020-12-02T23:21:54.595662813Z",
+ "created_by": "/bin/sh -c #(nop) USER root",
+ "empty_layer": true
+ },
+ {
+ "created": "2020-12-02T23:25:45.822235539Z",
+ "created_by": "/bin/sh -c /opt/python/bin/pip3.6 install --no-cache-dir numpy scipy pandas requests setuptools scikit-learn matplotlib"
+ },
+ {
+ "created": "2020-12-02T23:25:46.708884538Z",
+ "created_by": "/bin/sh -c #(nop) ENV JAVA_HOME=/home/y/share/yjava_jdk/java",
+ "empty_layer": true
+ },
+ {
+ "created": "2020-12-02T23:25:46.770226108Z",
+ "created_by": "/bin/sh -c #(nop) ENV PATH=/opt/rh/rh-git218/root/usr/bin:/home/y/bin64:/home/y/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/y/share/yjava_jdk/java/bin",
+ "empty_layer": true
+ },
+ {
+ "created": "2020-12-02T23:25:46.837263533Z",
+ "created_by": "/bin/sh -c #(nop) COPY file:33283617fbd796b25e53eaf4d26012eea1f610ff9acc0706f11281e86be440dc in /etc/krb5.conf "
+ },
+ {
+ "created": "2020-12-02T23:25:47.237515768Z",
+ "created_by": "/bin/sh -c echo '7.7.4' \u003e /etc/hadoop-dockerfile-version"
+ }
+ ],
+ "os": "linux",
+ "rootfs": {
+ "type": "layers",
+ "diff_ids": [
+ "sha256:9f627fdb0292afbe5e2eb96edc1b3a5d3a8f468e3acf1d29f1509509285c7341",
+ "sha256:83d2667f9458eaf719588a96bb63f2520bd377d29d52f6dbd4ff13c819c08037",
+ "sha256:fcba5f49eef4f3d77d3e73e499a1a4e1914b3f20d903625d27c0aa3ab82f41a3",
+ "sha256:3bd4567d0726f5d6560b548bc0c0400e868f6a27067887a36edd7e8ceafff96c",
+ "sha256:ad56900a1f10e6ef96f17c7e8019384540ab1b34ccce6bda06675473b08d787e",
+ "sha256:ac0a645609f957ab9c4a8a62f8646e99f09a74ada54ed2eaca204c6e183c9ae8",
+ "sha256:9bf10102fc145156f4081c2cacdbadab5816dce4f88eb02881ab739239d316e6"
+ ]
+ }
+}
+```
+
+Note: To use the `docker-to-squash.py`, you need to install [skopeo](https://github.com/containers/skopeo), [jq](https://stedolan.github.io/jq/) and squashfs-tools.
+
+
+## Configurations
+
+Then you need to set up storm with the following configs:
+
+| Setting | Description [...]
+|-------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- [...]
+| `storm.resource.isolation.plugin.enable` | set to `true` to enable isolation plugin. `storm.resource.isolation.plugin` determines which plugin to use. If this is set to `false`, `org.apache.storm.container.DefaultResourceIsolationManager` will be used. |
+| `storm.resource.isolation.plugin` | set to `"org.apache.storm.container.oci.RuncLibContainerManager"` to enable OCI/Squash runtime support [...]
+| `storm.oci.allowed.images` | An allowlist of docker images that can be used. Users can only choose a docker image from the list.
+| `storm.oci.image` | The default docker image to be used if user doesn't specify which image to use. And it must belong to the `storm.oci.allowed.images`
+| `topology.oci.image` | Topologies can specify which image to use. It must belong to the `storm.oci.allowed.images` |
+| `storm.oci.cgroup.root` | The root path of cgroup for docker to use. On RHEL7, it should be "/sys/fs/cgroup".
+| `storm.oci.cgroup.parent` | --cgroup-parent config for docker command. It must follow the constraints of docker commands. The path will be made as absolute path if it's a relative path because we saw some weird bugs ((the cgroup memory directory disappears after a while) when a relative path is used.
+| `storm.oci.readonly.bindmounts` | A list of read only bind mounted directories.
+| `storm.oci.readwrite.bindmounts` | A list of read-write bind mounted directories.
+| `storm.oci.nscd.dir` | The directory of nscd (name service cache daemon), e.g. "/var/run/nscd/". nscd must be running so that profiling can work properly.
+| `storm.oci.seccomp.profile` | Specify the seccomp Json file to be used as a seccomp filter
+| `supervisor.worker.launcher` | Full path to the worker-launcher executable.
+| `storm.oci.image.hdfs.toplevel.dir` | The HDFS location under which the oci image manifests, layers and configs directories exist.
+| `storm.oci.image.tag.to.manifest.plugin` | The plugin to be used to get the image-tag to manifest mappings.
+| `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.hdfs.hash.file` | The hdfs location of the image-tag to manifest mapping file. If `org.apache.storm.container.oci.LocalOrHdfsImageTagToManifestPlugin` is used as `storm.oci.image.tag.to.manifest.plugin`, either `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.hdfs.hash.file` or `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.local.hash.file` needs to be configured.
+| `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.local.hash.file` | The local file system location where the image-tag to manifest mapping file exists. If `org.apache.storm.container.oci.LocalOrHdfsImageTagToManifestPlugin` is used as `storm.oci.image.tag.to.manifest.plugin`, either `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.hdfs.hash.file` or `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.local.hash.file` needs to be configured.
+| `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.cache.refresh.interval.secs` | The interval in seconds between refreshing the image-tag to manifest mapping cache, used by `org.apache.storm.container.oci.LocalOrHdfsImageTagToManifestPlugin`.|
+| `storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.num.manifests.to.cache` | The number of manifests to cache, used by `org.apache.storm.container.oci.LocalOrHdfsImageTagToManifestPlugin`.|
+| `storm.oci.manifest.to.resources.plugin` | The plugin to be used to get oci resource according to the manifest.
+| `storm.oci.resources.localizer` | The plugin to use for oci resources localization. |
+| `storm.oci.resources.local.dir` | The local directory for localized oci resources. |
+
+For example,
+```bash
+storm.resource.isolation.plugin: "org.apache.storm.container.oci.RuncLibContainerManager"
+
+storm.oci.allowed.images:
+ - "storm/rhel7:dev_current"
+ - "storm/rhel7:dev_previous"
+ - "storm/rhel7:dev_test"
+storm.oci.image: "storm/rhel7:dev_current"
+
+storm.oci.cgroup.parent: "/storm"
+storm.oci.cgroup.root: "/sys/fs/cgroup"
+storm.oci.image.hdfs.toplevel.dir: "hdfs://host:port/containers/"
+storm.oci.image.tag.to.manifest.plugin: "org.apache.storm.container.oci.LocalOrHdfsImageTagToManifestPlugin"
+storm.oci.local.or.hdfs.image.tag.to.manifest.plugin.hdfs.hash.file: "hdfs://host:port/containers/image-tag-to-hash"
+storm.oci.manifest.to.resources.plugin: "org.apache.storm.container.oci.HdfsManifestToResourcesPlugin"
+storm.oci.readonly.bindmounts:
+ - "/home/y/lib64/storm"
+ - "/etc/krb5.conf"
+
+storm.oci.resources.localizer: "org.apache.storm.container.oci.HdfsOciResourcesLocalizer"
+storm.oci.seccomp.profile: "/home/y/conf/storm/seccomp.json"
+```
+
+To use built-in plugins from `external/storm-hdfs-oci`, you need to build `external/storm-hdfs-oci` and copy `storm-hdfs-oci.jar` and its dependencies to the `extlib-daemon` directory.
+
+Additionally, if you want to access to secure hdfs, you also need to set the following configs.
+```
+storm.hdfs.login.keytab
+storm.hdfs.login.principal
+```
+
+For example,
+```
+storm.hdfs.login.keytab: /etc/keytab
+storm.hdfs.login.principal: primary/instance@REALM
+```
+
+## Implementation
+
+##### Launch a container
+
+The supervisor calls RuncLibContainerManager to launch the container and the worker inside the container. It will first call the `storm.oci.image.tag.to.manifest.plugin`
+to fetch the mapping of image tag to manifest. Then it calls `storm.oci.manifest.to.resources.plugin` to get the list of resources to be downloaded and invokes
+`storm.oci.resources.localizer` to download the config of the image and the layers of the image to a local directory. It then composes a `oci-config.json` (see example in Appendix) and
+invokes worker-launcher to launch the container.
+
+The worker-launcher parses the `oci-config.json` file and do some necessary initialization and set up. It then creates /run/worker-launcher/layers/xxx/mnt directories
+and associate them with loopback devices, for example:
+
+```bash
+-bash-4.2$ cat /proc/mounts
+...
+/dev/loop3 /run/worker-launcher/layers/f7452c2657900c53da1a4f7e430485a267b89c7717466ee61ffefba85f690226/mnt squashfs ro,relatime 0 0
+/dev/loop4 /run/worker-launcher/layers/8156da43228752c7364b71dabba6aef6bd1cc081e9ea59cf92ea0f79fd8a50b6/mnt squashfs ro,relatime 0 0
+/dev/loop5 /run/worker-launcher/layers/c7c9b1d6df043edf307c49d75c7d2bc3df72f8dcaf7d17b733c97022387902e6/mnt squashfs ro,relatime 0 0
+/dev/loop6 /run/worker-launcher/layers/f0d08d5707855b02def8ac622a6c60203b380e31c6c237e5b691f5856594a3e7/mnt squashfs ro,relatime 0 0
+/dev/loop11 /run/worker-launcher/layers/34b0bc9c446a9be565fb50b04db1e9d1c1c4d14a22a885a7aba6981748b6635e/mnt squashfs ro,relatime 0 0
+/dev/loop12 /run/worker-launcher/layers/0ba001c025aa172a7d630914c75c1772228606f622e2c9d46a8fedf10774623e/mnt squashfs ro,relatime 0 0
+/dev/loop13 /run/worker-launcher/layers/a5e4e615565081e04eaf4c5ab5b20d37de271db704fc781c7b1e07c5dcdf96e5/mnt squashfs ro,relatime 0 0
+...
+
+```
+
+Then it mounts the layers, for example:
+```bash
+-bash-4.2$ mount
+...
+/home/y/var/storm/supervisor/oci-resources/layers/3692c3483ef6516fba685b316448e8aaf0fc10bb66818116edc8e5e6800076c7.sqsh on /run/worker-launcher/layers/f7452c2657900c53da1a4f7e430485a267b89c7717466ee61ffefba85f690226/mnt type squashfs (ro,relatime)
+/home/y/var/storm/supervisor/oci-resources/layers/ea067172a7138f035d89a5c378db6d66c1581d98b0497b21f256e04c3d2b5303.sqsh on /run/worker-launcher/layers/8156da43228752c7364b71dabba6aef6bd1cc081e9ea59cf92ea0f79fd8a50b6/mnt type squashfs (ro,relatime)
+/home/y/var/storm/supervisor/oci-resources/layers/1b73e9433ecca0a6bb152bd7525f2b7c233484d51c24f8a6ba483d5cfd3035dc.sqsh on /run/worker-launcher/layers/c7c9b1d6df043edf307c49d75c7d2bc3df72f8dcaf7d17b733c97022387902e6/mnt type squashfs (ro,relatime)
+/home/y/var/storm/supervisor/oci-resources/layers/18ee671016a1bf3ecab07395d93c2cbecd352d59c497a1551e2074d64e1098d9.sqsh on /run/worker-launcher/layers/f0d08d5707855b02def8ac622a6c60203b380e31c6c237e5b691f5856594a3e7/mnt type squashfs (ro,relatime)
+/home/y/var/storm/supervisor/oci-resources/layers/152ee1d2cccea9dfe6393d2bdf9d077b67616b2b417b25eb74fc5ffaadcb96f5.sqsh on /run/worker-launcher/layers/34b0bc9c446a9be565fb50b04db1e9d1c1c4d14a22a885a7aba6981748b6635e/mnt type squashfs (ro,relatime)
+/home/y/var/storm/supervisor/oci-resources/layers/344224962010c03c9ca1f11a9bff0dfcc296ac46d0a55e4ff30a0ad13b9817af.sqsh on /run/worker-launcher/layers/0ba001c025aa172a7d630914c75c1772228606f622e2c9d46a8fedf10774623e/mnt type squashfs (ro,relatime)
+/home/y/var/storm/supervisor/oci-resources/layers/8710a3d72f75b45c48ab6b9b67eb6d77caea3dac91a0c30e0831f591cba4887e.sqsh on /run/worker-launcher/layers/a5e4e615565081e04eaf4c5ab5b20d37de271db704fc781c7b1e07c5dcdf96e5/mnt type squashfs (ro,relatime)
+...
+```
+
+It creates the rootfs and mount the overlay filesystem (with lowerdir,upperdir,workdir) for the worker with the command
+```bash
+mount -t overlay overlay -o lowerdir=/lower1:/lower2:/lower3,upperdir=/upper,workdir=/work /merged
+```
+
+```bash
+-bash-4.2$ mount
+...
+overlay on /run/worker-launcher/6703-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/rootfs type overlay (rw,relatime,lowerdir=/run/worker-launcher/layers/a5e4e615565081e04eaf4c5ab5b20d37de271db704fc781c7b1e07c5dcdf96e5/mnt:/run/worker-launcher/layers/0ba001c025aa172a7d630914c75c1772228606f622e2c9d46a8fedf10774623e/mnt:/run/worker-launcher/layers/34b0bc9c446a9be565fb50b04db1e9d1c1c4d14a22a885a7aba6981748b6635e/mnt:/run/worker-launcher/layers/f0d08d5707855b02def8ac622a6c60203b380e31c6c237e5b691f5856 [...]
+...
+```
+
+It then produce a `config.json` (see example at Appendix) under `/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb` directory and launch the container with
+the command
+```bash
+/usr/bin/runc run -d \
+ --pid-file /home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/artifacts/container-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb.pid \
+ -b /home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb \
+ 6703-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb
+```
+
+##### Kill a container
+
+To kill a container, `RuncLibContainerManager` sends the `SIGTERM` or `SIGKILL` signal to the container process. It then invokes worker-launcher to to umount the mounts and clean up the directories.
+The worker-launcher will invoke `runc delete container-id` to delete the container at the end.
+
+
+## Profile the processes inside the container
+If you have sudo permission, you can also run `sudo nsenter --target <container-pid> --pid --mount --setuid <uid> --setgid <gid>` to enter the container.
+Then you can run `jstack`, `jmap` etc inside the container. `<container-pid>` is the pid of the container process on the host.
+`<container-pid>` can be obtained by running `runc list` command.
+
+## Seccomp security profiles
+
+You can set `storm.oci.seccomp.profile` to restrict the actions available within the container. If it's not set, the container runs without
+restrictions. You can use `conf/seccomp.json.example` provided or you can specify our own `seccomp.json` file.
+
+
+## Appendix
+
+##### Example oci-config.json file
+```json
+{
+ "version": "0.1",
+ "username": "username1",
+ "containerId": "6703-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "pidFile": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/artifacts/container-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb.pid",
+ "containerScriptPath": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/storm-worker-script.sh",
+ "layers": [
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/3692c3483ef6516fba685b316448e8aaf0fc10bb66818116edc8e5e6800076c7.sqsh"
+ },
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/ea067172a7138f035d89a5c378db6d66c1581d98b0497b21f256e04c3d2b5303.sqsh"
+ },
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/1b73e9433ecca0a6bb152bd7525f2b7c233484d51c24f8a6ba483d5cfd3035dc.sqsh"
+ },
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/18ee671016a1bf3ecab07395d93c2cbecd352d59c497a1551e2074d64e1098d9.sqsh"
+ },
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/152ee1d2cccea9dfe6393d2bdf9d077b67616b2b417b25eb74fc5ffaadcb96f5.sqsh"
+ },
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/344224962010c03c9ca1f11a9bff0dfcc296ac46d0a55e4ff30a0ad13b9817af.sqsh"
+ },
+ {
+ "mediaType": "application/vnd.squashfs",
+ "path": "/home/y/var/storm/supervisor/oci-resources/layers/8710a3d72f75b45c48ab6b9b67eb6d77caea3dac91a0c30e0831f591cba4887e.sqsh"
+ }
+ ],
+ "reapLayerKeepCount": 100,
+ "ociRuntimeConfig": {
+ "mounts": [
+ {
+ "destination": "/home/y/lib64/storm",
+ "type": "bind",
+ "source": "/home/y/lib64/storm",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/krb5.conf",
+ "type": "bind",
+ "source": "/etc/krb5.conf",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/resolv.conf",
+ "type": "bind",
+ "source": "/etc/resolv.conf",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/hostname",
+ "type": "bind",
+ "source": "/etc/hostname",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/hosts",
+ "type": "bind",
+ "source": "/etc/hosts",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/var/run/nscd",
+ "type": "bind",
+ "source": "/var/run/nscd",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/sys/fs/cgroup",
+ "type": "bind",
+ "source": "/sys/fs/cgroup",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/supervisor",
+ "type": "bind",
+ "source": "/home/y/var/storm/supervisor",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/workers-artifacts/wc1-2-1608581491/6703",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers-artifacts/wc1-2-1608581491/6703",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/workers-users/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers-users/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/shared_by_topology",
+ "type": "bind",
+ "source": "/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/shared_by_topology",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/tmp",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/tmp",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ }
+ ],
+ "process": {
+ "cwd": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "env": [
+ "X_SCLS=rh-git218",
+ "LD_LIBRARY_PATH=/opt/rh/httpd24/root/usr/lib64",
+ "PATH=/opt/rh/rh-git218/root/usr/bin:/home/y/bin64:/home/y/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/y/share/yjava_jdk/java/bin",
+ "PERL5LIB=/opt/rh/rh-git218/root/usr/share/perl5/vendor_perl",
+ "LANG=en_US.UTF-8",
+ "LANGUAGE=en_US:en",
+ "LC_ALL=en_US.UTF-8",
+ "JAVA_HOME=/home/y/share/yjava_jdk/java",
+ "LD_LIBRARY_PATH=/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/resources/Linux-amd64:/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/resources:/home/y/lib64:/usr/local/lib64:/usr/lib64:/lib64:"
+ ],
+ "args": [
+ "bash",
+ "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/storm-worker-script.sh"
+ ]
+ },
+ "linux": {
+ "cgroupsPath": "/storm/6703-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "resources": {
+ "cpu": {
+ "quota": 140000,
+ "period": 100000
+ }
+ },
+ "seccomp": {
+ "defaultAction": "SCMP_ACT_ERRNO",
+ "syscalls": [
+ {
+ "names": [
+ "accept",
+ "accept4",
+ "access",
+ "alarm",
+ "alarm",
+ "bind",
+ "brk",
+ "capget",
+ "capset",
+ "chdir",
+ "chmod",
+ "chown",
+ "chown32",
+ "clock_getres",
+ "clock_gettime",
+ "clock_nanosleep",
+ "close",
+ "connect",
+ "copy_file_range",
+ "creat",
+ "dup",
+ "dup2",
+ "dup3",
+ "epoll_create",
+ "epoll_create1",
+ "epoll_ctl",
+ "epoll_ctl_old",
+ "epoll_pwait",
+ "epoll_wait",
+ "epoll_wait_old",
+ "eventfd",
+ "eventfd2",
+ "execve",
+ "execveat",
+ "exit",
+ "exit_group",
+ "faccessat",
+ "fadvise64",
+ "fadvise64_64",
+ "fallocate",
+ "fanotify_mark",
+ "fchdir",
+ "fchmod",
+ "fchmodat",
+ "fchown",
+ "fchown32",
+ "fchownat",
+ "fcntl",
+ "fcntl64",
+ "fdatasync",
+ "fgetxattr",
+ "flistxattr",
+ "flock",
+ "fork",
+ "fremovexattr",
+ "fsetxattr",
+ "fstat",
+ "fstat64",
+ "fstatat64",
+ "fstatfs",
+ "fstatfs64",
+ "fsync",
+ "ftruncate",
+ "ftruncate64",
+ "futex",
+ "futimesat",
+ "getcpu",
+ "getcwd",
+ "getdents",
+ "getdents64",
+ "getegid",
+ "getegid32",
+ "geteuid",
+ "geteuid32",
+ "getgid",
+ "getgid32",
+ "getgroups",
+ "getgroups32",
+ "getitimer",
+ "getpeername",
+ "getpgid",
+ "getpgrp",
+ "getpid",
+ "getppid",
+ "getpriority",
+ "getrandom",
+ "getresgid",
+ "getresgid32",
+ "getresuid",
+ "getresuid32",
+ "getrlimit",
+ "get_robust_list",
+ "getrusage",
+ "getsid",
+ "getsockname",
+ "getsockopt",
+ "get_thread_area",
+ "gettid",
+ "gettimeofday",
+ "getuid",
+ "getuid32",
+ "getxattr",
+ "inotify_add_watch",
+ "inotify_init",
+ "inotify_init1",
+ "inotify_rm_watch",
+ "io_cancel",
+ "ioctl",
+ "io_destroy",
+ "io_getevents",
+ "ioprio_get",
+ "ioprio_set",
+ "io_setup",
+ "io_submit",
+ "ipc",
+ "kill",
+ "lchown",
+ "lchown32",
+ "lgetxattr",
+ "link",
+ "linkat",
+ "listen",
+ "listxattr",
+ "llistxattr",
+ "_llseek",
+ "lremovexattr",
+ "lseek",
+ "lsetxattr",
+ "lstat",
+ "lstat64",
+ "madvise",
+ "mbind",
+ "memfd_create",
+ "mincore",
+ "mkdir",
+ "mkdirat",
+ "mknod",
+ "mknodat",
+ "mlock",
+ "mlock2",
+ "mlockall",
+ "mmap",
+ "mmap2",
+ "mprotect",
+ "mq_getsetattr",
+ "mq_notify",
+ "mq_open",
+ "mq_timedreceive",
+ "mq_timedsend",
+ "mq_unlink",
+ "mremap",
+ "msgctl",
+ "msgget",
+ "msgrcv",
+ "msgsnd",
+ "msync",
+ "munlock",
+ "munlockall",
+ "munmap",
+ "nanosleep",
+ "newfstatat",
+ "_newselect",
+ "open",
+ "openat",
+ "pause",
+ "pipe",
+ "pipe2",
+ "poll",
+ "ppoll",
+ "prctl",
+ "pread64",
+ "preadv",
+ "prlimit64",
+ "pselect6",
+ "pwrite64",
+ "pwritev",
+ "read",
+ "readahead",
+ "readlink",
+ "readlinkat",
+ "readv",
+ "recv",
+ "recvfrom",
+ "recvmmsg",
+ "recvmsg",
+ "remap_file_pages",
+ "removexattr",
+ "rename",
+ "renameat",
+ "renameat2",
+ "restart_syscall",
+ "rmdir",
+ "rt_sigaction",
+ "rt_sigpending",
+ "rt_sigprocmask",
+ "rt_sigqueueinfo",
+ "rt_sigreturn",
+ "rt_sigsuspend",
+ "rt_sigtimedwait",
+ "rt_tgsigqueueinfo",
+ "sched_getaffinity",
+ "sched_getattr",
+ "sched_getparam",
+ "sched_get_priority_max",
+ "sched_get_priority_min",
+ "sched_getscheduler",
+ "sched_rr_get_interval",
+ "sched_setaffinity",
+ "sched_setattr",
+ "sched_setparam",
+ "sched_setscheduler",
+ "sched_yield",
+ "seccomp",
+ "select",
+ "semctl",
+ "semget",
+ "semop",
+ "semtimedop",
+ "send",
+ "sendfile",
+ "sendfile64",
+ "sendmmsg",
+ "sendmsg",
+ "sendto",
+ "setfsgid",
+ "setfsgid32",
+ "setfsuid",
+ "setfsuid32",
+ "setgid",
+ "setgid32",
+ "setgroups",
+ "setgroups32",
+ "setitimer",
+ "setpgid",
+ "setpriority",
+ "setregid",
+ "setregid32",
+ "setresgid",
+ "setresgid32",
+ "setresuid",
+ "setresuid32",
+ "setreuid",
+ "setreuid32",
+ "setrlimit",
+ "set_robust_list",
+ "setsid",
+ "setsockopt",
+ "set_thread_area",
+ "set_tid_address",
+ "setuid",
+ "setuid32",
+ "setxattr",
+ "shmat",
+ "shmctl",
+ "shmdt",
+ "shmget",
+ "shutdown",
+ "sigaltstack",
+ "signalfd",
+ "signalfd4",
+ "sigreturn",
+ "socket",
+ "socketcall",
+ "socketpair",
+ "splice",
+ "stat",
+ "stat64",
+ "statfs",
+ "statfs64",
+ "symlink",
+ "symlinkat",
+ "sync",
+ "sync_file_range",
+ "syncfs",
+ "sysinfo",
+ "syslog",
+ "tee",
+ "tgkill",
+ "time",
+ "timer_create",
+ "timer_delete",
+ "timerfd_create",
+ "timerfd_gettime",
+ "timerfd_settime",
+ "timer_getoverrun",
+ "timer_gettime",
+ "timer_settime",
+ "times",
+ "tkill",
+ "truncate",
+ "truncate64",
+ "ugetrlimit",
+ "umask",
+ "uname",
+ "unlink",
+ "unlinkat",
+ "utime",
+ "utimensat",
+ "utimes",
+ "vfork",
+ "vmsplice",
+ "wait4",
+ "waitid",
+ "waitpid",
+ "write",
+ "writev",
+ "mount",
+ "umount2",
+ "reboot",
+ "name_to_handle_at",
+ "unshare"
+ ],
+ "action": "SCMP_ACT_ALLOW"
+ },
+ {
+ "names": [
+ "personality"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 0,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_EQ"
+ }
+ ]
+ },
+ {
+ "names": [
+ "personality"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 8,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_EQ"
+ }
+ ]
+ },
+ {
+ "names": [
+ "personality"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 4294967295,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_EQ"
+ }
+ ]
+ },
+ {
+ "names": [
+ "arch_prctl"
+ ],
+ "action": "SCMP_ACT_ALLOW"
+ },
+ {
+ "names": [
+ "modify_ldt"
+ ],
+ "action": "SCMP_ACT_ALLOW"
+ },
+ {
+ "names": [
+ "clone"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 2080505856,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_MASKED_EQ"
+ }
+ ]
+ }
+ ]
+ }
+ }
+ }
+}
+```
+
+##### Example config.json file
+```json
+{
+ "ociVersion": "1.0.0",
+ "hostname": "hostname1",
+ "root": {
+ "path": "/run/worker-launcher/6703-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/rootfs",
+ "readonly": true
+ },
+ "process": {
+ "args": [
+ "bash",
+ "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/storm-worker-script.sh"
+ ],
+ "cwd": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "env": [
+ "X_SCLS=rh-git218",
+ "LD_LIBRARY_PATH=/opt/rh/httpd24/root/usr/lib64",
+ "PATH=/opt/rh/rh-git218/root/usr/bin:/home/y/bin64:/home/y/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/y/share/yjava_jdk/java/bin",
+ "PERL5LIB=/opt/rh/rh-git218/root/usr/share/perl5/vendor_perl",
+ "LANG=en_US.UTF-8",
+ "LANGUAGE=en_US:en",
+ "LC_ALL=en_US.UTF-8",
+ "JAVA_HOME=/home/y/share/yjava_jdk/java",
+ "LD_LIBRARY_PATH=/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/resources/Linux-amd64:/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/resources:/home/y/lib64:/usr/local/lib64:/usr/lib64:/lib64:"
+ ],
+ "noNewPrivileges": true,
+ "user": {
+ "uid": 31315,
+ "gid": 100,
+ "additionalGids": [
+ 5548
+ ]
+ }
+ },
+ "mounts": [
+ {
+ "source": "proc",
+ "destination": "/proc",
+ "type": "proc"
+ },
+ {
+ "source": "tmpfs",
+ "destination": "/dev",
+ "type": "tmpfs",
+ "options": [
+ "nosuid",
+ "strictatime",
+ "mode=755",
+ "size=65536k"
+ ]
+ },
+ {
+ "source": "devpts",
+ "destination": "/dev/pts",
+ "type": "devpts",
+ "options": [
+ "nosuid",
+ "noexec",
+ "newinstance",
+ "ptmxmode=0666",
+ "mode=0620",
+ "gid=5"
+ ]
+ },
+ {
+ "source": "shm",
+ "destination": "/dev/shm",
+ "type": "tmpfs",
+ "options": [
+ "nosuid",
+ "noexec",
+ "nodev",
+ "mode=1777",
+ "size=65536k"
+ ]
+ },
+ {
+ "source": "mqueue",
+ "destination": "/dev/mqueue",
+ "type": "mqueue",
+ "options": [
+ "nosuid",
+ "noexec",
+ "nodev"
+ ]
+ },
+ {
+ "source": "sysfs",
+ "destination": "/sys",
+ "type": "sysfs",
+ "options": [
+ "nosuid",
+ "noexec",
+ "nodev",
+ "ro"
+ ]
+ },
+ {
+ "source": "cgroup",
+ "destination": "/sys/fs/cgroup",
+ "type": "cgroup",
+ "options": [
+ "nosuid",
+ "noexec",
+ "nodev",
+ "relatime",
+ "ro"
+ ]
+ },
+ {
+ "destination": "/home/y/lib64/storm",
+ "type": "bind",
+ "source": "/home/y/lib64/storm",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/krb5.conf",
+ "type": "bind",
+ "source": "/etc/krb5.conf",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/resolv.conf",
+ "type": "bind",
+ "source": "/etc/resolv.conf",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/hostname",
+ "type": "bind",
+ "source": "/etc/hostname",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/etc/hosts",
+ "type": "bind",
+ "source": "/etc/hosts",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/var/run/nscd",
+ "type": "bind",
+ "source": "/var/run/nscd",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+
+ {
+ "destination": "/sys/fs/cgroup",
+ "type": "bind",
+ "source": "/sys/fs/cgroup",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/supervisor",
+ "type": "bind",
+ "source": "/home/y/var/storm/supervisor",
+ "options": [
+ "ro",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/workers-artifacts/wc1-2-1608581491/6703",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers-artifacts/wc1-2-1608581491/6703",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/workers-users/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers-users/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/shared_by_topology",
+ "type": "bind",
+ "source": "/home/y/var/storm/supervisor/stormdist/wc1-2-1608581491/shared_by_topology",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ },
+ {
+ "destination": "/tmp",
+ "type": "bind",
+ "source": "/home/y/var/storm/workers/1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb/tmp",
+ "options": [
+ "rw",
+ "rbind",
+ "rprivate"
+ ]
+ }
+ ],
+ "linux": {
+ "cgroupsPath": "/storm/6703-1a23ca4b-6062-4d08-8ac3-b09e7d35e7cb",
+ "resources": {
+ "devices": [
+ {
+ "access": "rwm",
+ "allow": false
+ }
+ ],
+ "cpu": {
+ "quota": 140000,
+ "period": 100000
+ }
+ },
+ "namespaces": [
+ {
+ "type": "pid"
+ },
+ {
+ "type": "ipc"
+ },
+ {
+ "type": "uts"
+ },
+ {
+ "type": "mount"
+ }
+ ],
+ "maskedPaths": [
+ "/proc/kcore",
+ "/proc/latency_stats",
+ "/proc/timer_list",
+ "/proc/timer_stats",
+ "/proc/sched_debug",
+ "/proc/scsi",
+ "/sys/firmware"
+ ],
+ "readonlyPaths": [
+ "/proc/asound",
+ "/proc/bus",
+ "/proc/fs",
+ "/proc/irq",
+ "/proc/sys",
+ "/proc/sysrq-trigger"
+ ],
+ "seccomp": {
+ "defaultAction": "SCMP_ACT_ERRNO",
+ "syscalls": [
+ {
+ "names": [
+ "accept",
+ "accept4",
+ "access",
+ "alarm",
+ "alarm",
+ "bind",
+ "brk",
+ "capget",
+ "capset",
+ "chdir",
+ "chmod",
+ "chown",
+ "chown32",
+ "clock_getres",
+ "clock_gettime",
+ "clock_nanosleep",
+ "close",
+ "connect",
+ "copy_file_range",
+ "creat",
+ "dup",
+ "dup2",
+ "dup3",
+ "epoll_create",
+ "epoll_create1",
+ "epoll_ctl",
+ "epoll_ctl_old",
+ "epoll_pwait",
+ "epoll_wait",
+ "epoll_wait_old",
+ "eventfd",
+ "eventfd2",
+ "execve",
+ "execveat",
+ "exit",
+ "exit_group",
+ "faccessat",
+ "fadvise64",
+ "fadvise64_64",
+ "fallocate",
+ "fanotify_mark",
+ "fchdir",
+ "fchmod",
+ "fchmodat",
+ "fchown",
+ "fchown32",
+ "fchownat",
+ "fcntl",
+ "fcntl64",
+ "fdatasync",
+ "fgetxattr",
+ "flistxattr",
+ "flock",
+ "fork",
+ "fremovexattr",
+ "fsetxattr",
+ "fstat",
+ "fstat64",
+ "fstatat64",
+ "fstatfs",
+ "fstatfs64",
+ "fsync",
+ "ftruncate",
+ "ftruncate64",
+ "futex",
+ "futimesat",
+ "getcpu",
+ "getcwd",
+ "getdents",
+ "getdents64",
+ "getegid",
+ "getegid32",
+ "geteuid",
+ "geteuid32",
+ "getgid",
+ "getgid32",
+ "getgroups",
+ "getgroups32",
+ "getitimer",
+ "getpeername",
+ "getpgid",
+ "getpgrp",
+ "getpid",
+ "getppid",
+ "getpriority",
+ "getrandom",
+ "getresgid",
+ "getresgid32",
+ "getresuid",
+ "getresuid32",
+ "getrlimit",
+ "get_robust_list",
+ "getrusage",
+ "getsid",
+ "getsockname",
+ "getsockopt",
+ "get_thread_area",
+ "gettid",
+ "gettimeofday",
+ "getuid",
+ "getuid32",
+ "getxattr",
+ "inotify_add_watch",
+ "inotify_init",
+ "inotify_init1",
+ "inotify_rm_watch",
+ "io_cancel",
+ "ioctl",
+ "io_destroy",
+ "io_getevents",
+ "ioprio_get",
+ "ioprio_set",
+ "io_setup",
+ "io_submit",
+ "ipc",
+ "kill",
+ "lchown",
+ "lchown32",
+ "lgetxattr",
+ "link",
+ "linkat",
+ "listen",
+ "listxattr",
+ "llistxattr",
+ "_llseek",
+ "lremovexattr",
+ "lseek",
+ "lsetxattr",
+ "lstat",
+ "lstat64",
+ "madvise",
+ "mbind",
+ "memfd_create",
+ "mincore",
+ "mkdir",
+ "mkdirat",
+ "mknod",
+ "mknodat",
+ "mlock",
+ "mlock2",
+ "mlockall",
+ "mmap",
+ "mmap2",
+ "mprotect",
+ "mq_getsetattr",
+ "mq_notify",
+ "mq_open",
+ "mq_timedreceive",
+ "mq_timedsend",
+ "mq_unlink",
+ "mremap",
+ "msgctl",
+ "msgget",
+ "msgrcv",
+ "msgsnd",
+ "msync",
+ "munlock",
+ "munlockall",
+ "munmap",
+ "nanosleep",
+ "newfstatat",
+ "_newselect",
+ "open",
+ "openat",
+ "pause",
+ "pipe",
+ "pipe2",
+ "poll",
+ "ppoll",
+ "prctl",
+ "pread64",
+ "preadv",
+ "prlimit64",
+ "pselect6",
+ "pwrite64",
+ "pwritev",
+ "read",
+ "readahead",
+ "readlink",
+ "readlinkat",
+ "readv",
+ "recv",
+ "recvfrom",
+ "recvmmsg",
+ "recvmsg",
+ "remap_file_pages",
+ "removexattr",
+ "rename",
+ "renameat",
+ "renameat2",
+ "restart_syscall",
+ "rmdir",
+ "rt_sigaction",
+ "rt_sigpending",
+ "rt_sigprocmask",
+ "rt_sigqueueinfo",
+ "rt_sigreturn",
+ "rt_sigsuspend",
+ "rt_sigtimedwait",
+ "rt_tgsigqueueinfo",
+ "sched_getaffinity",
+ "sched_getattr",
+ "sched_getparam",
+ "sched_get_priority_max",
+ "sched_get_priority_min",
+ "sched_getscheduler",
+ "sched_rr_get_interval",
+ "sched_setaffinity",
+ "sched_setattr",
+ "sched_setparam",
+ "sched_setscheduler",
+ "sched_yield",
+ "seccomp",
+ "select",
+ "semctl",
+ "semget",
+ "semop",
+ "semtimedop",
+ "send",
+ "sendfile",
+ "sendfile64",
+ "sendmmsg",
+ "sendmsg",
+ "sendto",
+ "setfsgid",
+ "setfsgid32",
+ "setfsuid",
+ "setfsuid32",
+ "setgid",
+ "setgid32",
+ "setgroups",
+ "setgroups32",
+ "setitimer",
+ "setpgid",
+ "setpriority",
+ "setregid",
+ "setregid32",
+ "setresgid",
+ "setresgid32",
+ "setresuid",
+ "setresuid32",
+ "setreuid",
+ "setreuid32",
+ "setrlimit",
+ "set_robust_list",
+ "setsid",
+ "setsockopt",
+ "set_thread_area",
+ "set_tid_address",
+ "setuid",
+ "setuid32",
+ "setxattr",
+ "shmat",
+ "shmctl",
+ "shmdt",
+ "shmget",
+ "shutdown",
+ "sigaltstack",
+ "signalfd",
+ "signalfd4",
+ "sigreturn",
+ "socket",
+ "socketcall",
+ "socketpair",
+ "splice",
+ "stat",
+ "stat64",
+ "statfs",
+ "statfs64",
+ "symlink",
+ "symlinkat",
+ "sync",
+ "sync_file_range",
+ "syncfs",
+ "sysinfo",
+ "syslog",
+ "tee",
+ "tgkill",
+ "time",
+ "timer_create",
+ "timer_delete",
+ "timerfd_create",
+ "timerfd_gettime",
+ "timerfd_settime",
+ "timer_getoverrun",
+ "timer_gettime",
+ "timer_settime",
+ "times",
+ "tkill",
+ "truncate",
+ "truncate64",
+ "ugetrlimit",
+ "umask",
+ "uname",
+ "unlink",
+ "unlinkat",
+ "utime",
+ "utimensat",
+ "utimes",
+ "vfork",
+ "vmsplice",
+ "wait4",
+ "waitid",
+ "waitpid",
+ "write",
+ "writev",
+ "mount",
+ "umount2",
+ "reboot",
+ "name_to_handle_at",
+ "unshare"
+ ],
+ "action": "SCMP_ACT_ALLOW"
+ },
+ {
+ "names": [
+ "personality"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 0,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_EQ"
+ }
+ ]
+ },
+ {
+ "names": [
+ "personality"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 8,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_EQ"
+ }
+ ]
+ },
+ {
+ "names": [
+ "personality"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 4294967295,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_EQ"
+ }
+ ]
+ },
+ {
+ "names": [
+ "arch_prctl"
+ ],
+ "action": "SCMP_ACT_ALLOW"
+ },
+ {
+ "names": [
+ "modify_ldt"
+ ],
+ "action": "SCMP_ACT_ALLOW"
+ },
+ {
+ "names": [
+ "clone"
+ ],
+ "action": "SCMP_ACT_ALLOW",
+ "args": [
+ {
+ "index": 0,
+ "value": 2080505856,
+ "valueTwo": 0,
+ "op": "SCMP_CMP_MASKED_EQ"
+ }
+ ]
+ }
+ ]
+ }
+ }
+}
+```
+
+## CGroup Metrics
+
+Runc internally uses cgroups to control resources for containers. The CGroup Metrics described at [cgroups_in_storm.md](cgroups_in_storm.md#CGroup-Metrics) still apply except CGroupCpuGuarantee. To get CGroup cpu guarantee, use CGroupCpuGuaranteeByCfsQuota instead.
\ No newline at end of file
diff --git a/releases/2.8.8/Pacemaker.md b/releases/2.8.8/Pacemaker.md
new file mode 100644
index 0000000000..75fa748554
--- /dev/null
+++ b/releases/2.8.8/Pacemaker.md
@@ -0,0 +1,112 @@
+---
+title: Pacemaker
+layout: documentation
+documentation: true
+---
+
+
+### Introduction
+Pacemaker is a storm daemon designed to process heartbeats from workers. As Storm is scaled up, ZooKeeper begins to become a bottleneck due to high volumes of writes from workers doing heartbeats. Lots of writes to disk and too much traffic across the network is generated as ZooKeeper tries to maintain consistency.
+
+Because heartbeats are of an ephemeral nature, they do not need to be persisted to disk or synced across nodes; an in-memory store will do. This is the role of Pacemaker. Pacemaker functions as a simple in-memory key/value store with ZooKeeper-like, directory-style keys and byte array values.
+
+The corresponding Pacemaker client is a plugin for the `ClusterState` interface, `org.apache.storm.cluster.PaceMakerStateStorageFactory`. Heartbeat calls are funneled by the `ClusterState` produced by `pacemaker_state_factory` into the Pacemaker daemon, while other set/get operations are forwarded to ZooKeeper.
+
+------
+
+### Configuration
+
+ - `pacemaker.servers` : The hosts that the Pacemaker daemons are running on
+ - `pacemaker.port` : The port that Pacemaker will listen on
+ - `pacemaker.max.threads` : Maximum number of threads Pacemaker daemon will use to handle requests.
+ - `pacemaker.childopts` : Any JVM parameters that need to go to the Pacemaker.
+ - `pacemaker.auth.method` : The authentication method that is used (more info below)
+
+#### Example
+
+To get Pacemaker up and running, set the following option in the cluster config on all nodes:
+```
+storm.cluster.state.store: "org.apache.storm.cluster.PaceMakerStateStorageFactory"
+```
+
+The Pacemaker servers also need to be set on all nodes:
+```
+pacemaker.servers:
+ - somehost.mycompany.com
+ - someotherhost.mycompany.com
+```
+
+And then start all of your daemons
+
+(including Pacemaker):
+```
+$ storm pacemaker
+```
+
+The Storm cluster should now be pushing all worker heartbeats through Pacemaker.
+
+### Security
+
+Currently digest (password-based) and Kerberos security are supported. Security is currently only around reads, not writes. Writes may be performed by anyone, whereas reads may only be performed by authorized and authenticated users. This is an area for future development, as it leaves the cluster open to DoS attacks, but it prevents any sensitive information from reaching unauthorized eyes, which was the main goal.
+
+#### Digest
+To configure digest authentication, set `pacemaker.auth.method: DIGEST` in the cluster config on the nodes hosting Nimbus and Pacemaker.
+The nodes must also have `java.security.auth.login.config` set to point to a JAAS config file containing the following structure:
+```
+PacemakerDigest {
+ username="some username"
+ password="some password";
+};
+```
+
+Any node with these settings configured will be able to read from Pacemaker.
+Worker nodes need not have these configs set, and may keep `pacemaker.auth.method: NONE` set, since they do not need to read from the Pacemaker daemon.
+
+#### Kerberos
+To configure Kerberos authentication, set `pacemaker.auth.method: KERBEROS` in the cluster config on the nodes hosting Nimbus and Pacemaker.
+The nodes must also have `java.security.auth.login.config` set to point to a JAAS config.
+
+The JAAS config on Nimbus must look something like this:
+```
+PacemakerClient {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="/etc/keytabs/nimbus.keytab"
+ storeKey=true
+ useTicketCache=false
+ serviceName="pacemaker"
+ principal="nimbus@MY.COMPANY.COM";
+};
+
+```
+
+The JAAS config on Pacemaker must look something like this:
+```
+PacemakerServer {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="/etc/keytabs/pacemaker.keytab"
+ storeKey=true
+ useTicketCache=false
+ principal="pacemaker@MY.COMPANY.COM";
+};
+```
+
+ - The client's user principal in the `PacemakerClient` section on the Nimbus host must match the `nimbus.daemon.user` storm cluster config value.
+ - The client's `serviceName` value must match the server's user principal in the `PacemakerServer` section on the Pacemaker host.
+
+
+### Fault Tolerance
+
+Pacemaker runs as a single daemon instance, making it a potential Single Point of Failure.
+
+If Pacemaker becomes unreachable by Nimbus, through crash or network partition, the workers will continue to run, and Nimbus will repeatedly attempt to reconnect. Nimbus functionality will be disrupted, but the topologies themselves will continue to run.
+In case of partition of the cluster where Nimbus and Pacemaker are on the same side of the partition, the workers that are on the other side of the partition will not be able to heartbeat, and Nimbus will reschedule the tasks elsewhere. This is probably what we want to happen anyway.
+
+
+### ZooKeeper Comparison
+Compared to ZooKeeper, Pacemaker uses less CPU, less memory, and of course no disk for the same load, thanks to lack of overhead from maintaining consistency between nodes.
+On Gigabit networking, there is a theoretical limit of about 6000 nodes. However, the real limit is likely around 2000-3000 nodes. These limits have not yet been tested.
+On a 270 supervisor cluster, fully scheduled with topologies, Pacemaker resource utilization was 70% of one core and nearly 1GiB of RAM on a machine with 4 `Intel(R) Xeon(R) CPU E5530 @ 2.40GHz` and 24GiB of RAM.
+
+Pacemaker now supports HA. Multiple Pacemaker instances can be used at once in a storm cluster to allow massive scalability. Just include the names of the Pacemaker hosts in the pacemaker.servers config and workers and Nimbus will start communicating with them. They're fault tolerant as well. The system keeps on working as long as there is at least one pacemaker left running - provided it can handle the load.
diff --git a/releases/2.8.8/Performance.md b/releases/2.8.8/Performance.md
new file mode 100644
index 0000000000..df8f3e09df
--- /dev/null
+++ b/releases/2.8.8/Performance.md
@@ -0,0 +1,179 @@
+---
+title: Performance Tuning
+layout: documentation
+documentation: true
+---
+
+Latency, throughput and resource consumption are the three key dimensions involved in performance tuning.
+In the following sections, we discuss the settings that can be used to tune along these dimensions and understand the trade-offs.
+
+It is important to understand that these settings can vary depending on the topology, the type of hardware, and the number of hosts used by the topology.
+
+## 1. Buffer Size
+Spouts and Bolts operate asynchronously using message passing. Message queues used for this purpose are of fixed but configurable size. Buffer size
+refers to the size of these queues. Every consumer has its own receive queue. The messages wait in the queue until the consumer is ready to process them.
+The queue will typically be almost empty or almost full depending on whether the consumer is operating faster or slower than the rate at which producers
+are generating messages for it. Storm queues always have a single consumer and potentially multiple producers. There are two buffer size settings
+of interest:
+
+- `topology.executor.receive.buffer.size` : This is the size of the message queue employed for each spout and bolt executor.
+- `topology.transfer.buffer.size` : This is the size of the outbound message queue used for inter-worker messaging. This queue is referred to as
+the *Worker Transfer Queue*.
+
+**Note:** If the specified buffer size is not a power of 2, it is internally rounded up to the next power of 2.
+
+#### Guidance
+Very small message queues (size < 1024) are likely to hamper throughput by not providing enough isolation between the consumer and producer. This
+can affect the asynchronous nature of the processing as the producers are likely to find the downstream queue to be full.
+
+Very large message queues are also not desirable to deal with slow consumers. Better to employ more consumers (i.e. bolts) on additional CPU cores instead. If queues
+are large and often full, the messages will end up waiting longer in these queues at each step of the processing, leading to poor latency being
+reported on the Storm UI. Large queues also imply higher memory consumption especially if the queues are typically full.
+
+
+## 2. Batch Size
+Producers can either write a batch of messages to the consumer's queue or write each message individually. This batch size can be configured.
+Inserting messages in batches to downstream queues helps reduce the number of synchronization operations required for the inserts. Consequently, this helps achieve higher throughput. However,
+sometimes it may take a little time for the buffer to fill up, before it is flushed into the downstream queue. This implies that the buffered messages
+will take longer to become visible to the downstream consumer who is waiting to process them. This can increase the average end-to-end latency for
+these messages. The latency can get very bad if the batch sizes are large and the topology is not experiencing high traffic.
+
+- `topology.producer.batch.size` : The batch size for writes into the receive queue of any spout/bolt is controlled via this setting. This setting
+impacts the communication within a worker process. Each upstream producer maintains a separate batch to a component's receive queue. So if two spout
+instances are writing to the same downstream bolt instance, each of the spout instances will have to maintain a separate batch.
+
+- `topology.transfer.batch.size` : Messages that are destined to a spout/bolt running on a different worker process, are sent to a queue called
+the **Worker Transfer Queue**. The Worker Transfer Thread is responsible for draining the messages in this queue and send them to the appropriate
+worker process over the network. This setting controls the batch size for writes into the Worker Transfer Queue. This impacts the communication
+between worker processes.
+
+#### Guidance
+
+**For Low latency:** Set batch size to 1. This basically disables batching. This is likely to reduce peak sustainable throughput under heavy traffic, but
+not likely to impact throughput much under low/medium traffic situations.
+
+**For High throughput:** Set batch size > 1. Try values like 10, 100, 1000 or even higher and see what yields the best throughput for the topology.
+Beyond a certain point the throughput is likely to get worse.
+
+**Varying throughput:** Topologies often experience fluctuating amounts of incoming traffic over the day. Other topos may experience higher traffic in some
+paths and lower throughput in other paths simultaneously. If latency is not a concern, a small bach size (e.g. 10) and in conjunction with the right flush
+frequency may provide a reasonable compromise for such scenarios. For meeting stricter latency SLAs, consider setting it to 1.
+
+
+## 3. Flush Tuple Frequency
+In low/medium traffic situations or when batch size is too large, the batches may take too long to fill up and consequently the messages could take unacceptably
+long time to become visible to downstream components. In such case, periodic flushing of batches is necessary to keep the messages moving and avoid compromising
+latencies when batching is enabled.
+
+When batching has been enabled, special messages called *flush tuples* are inserted periodically into the receive queues of all spout and bolt instances.
+This causes each spout/bolt instance to flush all its outstanding batches to their respective downstream components.
+
+`topology.flush.tuple.freq.millis` : This setting controls how often the flush tuples are generated. Flush tuples are not generated if this configuration is
+set to 0 or if (`topology.producer.batch.size`=1 and `topology.transfer.batch.size`=1).
+
+
+#### Guidance
+Flushing interval can be used as a tool to retain the higher throughput benefits of batching and avoid batched messages getting stuck for too long waiting for theirs.
+batch to fill. Preferably this value should be larger than the average execute latencies of the bolts in the topology. Trying to flush the queues more frequently than
+the amount of time it takes to produce the messages may hurt performance. Understanding the average execute latencies of each bolt will help determine the average
+number of messages in the queues between two flushes.
+
+**For Low latency:** A smaller value helps achieve tighter latency SLAs.
+
+**For High throughput:** When trying to maximize throughput under high traffic situations, the batches are likely to get filled and flushed automatically.
+To optimize for such cases, this value can be set to a higher number.
+
+**Varying throughput:** If latency is not a concern, a larger value will optimize for high traffic situations. For meeting tighter SLAs set this to lower
+values.
+
+
+## 4. Wait Strategy
+Wait strategies are used to conserve CPU usage by trading off some latency and throughput. They are applied for the following situations:
+
+4.1 **Spout Wait:** In low/no traffic situations, Spout's nextTuple() may not produce any new emits. To prevent invoking the Spout's nextTuple too often,
+this wait strategy is used between nextTuple() calls, allowing the spout's executor thread to idle and conserve CPU. Spout wait strategy is also used
+when the `topology.max.spout.pending` limit has been reached when ACKers are enabled. Select a strategy using `topology.spout.wait.strategy`. Configure the
+chosen wait strategy using one of the `topology.spout.wait.*` settings.
+
+4.2 **Bolt Wait:** : When a bolt polls it's receive a queue for new messages to process, it is possible that the queue is empty. This typically happens
+in case of low/no traffic situations or when the upstream spout/bolt is inherently slower. This wait strategy is used in such cases. It avoids high CPU usage
+due to the bolt continuously checking on a typically empty queue. Select a strategy using `topology.bolt.wait.strategy`. The chosen strategy can be further configured
+using the `topology.bolt.wait.*` settings.
+
+4.3 **Backpressure Wait** : Select a strategy using `topology.backpressure.wait.strategy`. When a spout/bolt tries to write to a downstream component's receive queue,
+there is a possibility that the queue is full. In such cases, the write needs to be retried. This wait strategy is used to induce some idling in-between re-attempts for
+conserving CPU. The chosen strategy can be further configured using the `topology.backpressure.wait.*` settings.
+
+
+#### Built-in wait strategies:
+These wait strategies are available for use with all of the above mentioned wait situations.
+
+- **ProgressiveWaitStrategy** : This strategy can be used for Bolt Wait or Backpressure Wait situations. Set the strategy to 'org.apache.storm.policy.WaitStrategyProgressive' to
+select this wait strategy. This is a dynamic wait strategy that enters into progressively deeper states of CPU conservation if the Backpressure Wait or Bolt Wait situations persist.
+It has 3 levels of idling and allows configuring how long to stay at each level :
+
+ 1. Level1 / No Waiting - The first few times it will return immediately. This does not conserve any CPU. The number of times it remains in this state is configured using
+ `topology.spout.wait.progressive.level1.count` or `topology.bolt.wait.progressive.level1.count` or `topology.backpressure.wait.progressive.level1.count` depending which
+ situation it is being used.
+
+ 2. Level 2 / Park Nanos - In this state it disables the current thread for thread scheduling purposes, for 1 nano second using LockSupport.parkNanos(). This puts the CPU in a minimal
+ conservation state. It remains in this state for `topology.spout.wait.progressive.level2.count` or `topology.bolt.wait.progressive.level2.count` or
+ `topology.backpressure.wait.progressive.level2.count` iterations.
+
+ 3. Level 3 / Thread.sleep() - In this level it calls Thread.sleep() with the value specified in `topology.spout.wait.progressive.level3.sleep.millis` or
+ `topology.bolt.wait.progressive.level3.sleep.millis` or `topology.backpressure.wait.progressive.level3.sleep.millis`. This is the most CPU conserving level and it remains in
+ this level for the remaining iterations.
+
+
+- **ParkWaitStrategy** : This strategy can be used for Bolt Wait or Backpressure Wait situations. Set the strategy to `org.apache.storm.policy.WaitStrategyPark` to use this.
+This strategy disables the current thread for thread scheduling purposes by calling LockSupport.parkNanos(). The amount of park time is configured using either
+`topology.bolt.wait.park.microsec` or `topology.backpressure.wait.park.microsec` based on the wait situation it is used. Setting the park time to 0, effectively disables
+invocation of LockSupport.parkNanos and this mode can be used to achieve busy polling (which at the cost of high CPU utilization even when idle, may improve latency and/or throughput).
+
+
+## 5. Max.spout.pending
+The setting `topology.max.spout.pending` limits the number of un-ACKed tuples at the spout level. Once a spout reaches this limit, the spout's nextTuple()
+method will not be called until some ACKs are received for the outstanding emits. This setting does not have any affect if ACKing is disabled. It
+is a spout throttling mechanism which can impact throughput and latency. Setting it to null disables it for storm-core topologies. Impact on throughput
+is dependent on the topology and its concurrency (workers/executors), so experimentation is necessary to determine optimal setting. Latency and memory consumption
+is expected to typically increase with higher and higher values for this.
+
+
+## 6. Load Aware messaging
+When load aware messaging is enabled (default), shuffle grouping takes additional factors into consideration for message routing.
+Impact of this on performance is dependent on the topology and its deployment footprint (i.e. distribution over process and machines).
+Consequently it is useful to assess the impact of setting `topology.disable.loadaware.messaging` to `true` or `false` for your
+specific case.
+
+
+## 7. Sampling Rate
+Sampling rate is used to control how often certain metrics are computed on the Spout and Bolt executors. This is configured using `topology.stats.sample.rate`
+Setting it to 1 means, the stats are computed for every emitted message. As an example, to sample once every 1000 messages it can be set to 0.001. It may be
+possible to improve throughput and latency by reducing the sampling rate.
+
+
+## 8. Budgeting CPU cores for Executors
+There are three main types of executors (i.e threads) to take into account when budgeting CPU cores for them. Spout Executors, Bolt Executors, Worker Transfer (handles outbound
+messages) and NettyWorker (handles inbound messages).
+The first two are used to run spout, bolt and acker instances. The Worker Transfer thread is used to serialize and send messages to other workers (in multi-worker mode).
+
+Executors that are expected to remain busy, either because they are handling a lot of messages, or because their processing is inherently CPU intensive, should be allocated
+1 physical core each. Allocating logical cores (instead of physical) or less than 1 physical core for CPU intensive executors increases CPU contention and performance can suffer.
+Executors that are not expected to be busy can be allocated a smaller fraction of the physical core (or even logical cores). It maybe not be economical to allocate a full physical
+core for executors that are not likely to saturate the CPU.
+
+The *system bolt* generally processes very few messages per second, and so requires very little cpu (typically less than 10% of a physical core).
+
+
+## 9. Garbage Collection
+Choice of GC is an important concern for topologies that are latency or throughput sensitive. It is recommended to try both the CMS and G1 collectors. Performance characteristics
+of the collectors can change between single and multiworker modes and are dependent on hardware characteristics such as the number of CPUs and memory localities. Number of GC threads can
+also affect performance. Sometimes fewer GC threads can yield better performance. It is advisable to select a collector and tune it by mimicking anticipated peak data rates on hardware
+similar to what is used in production.
+
+
+## 10. Scaling out with Single Worker mode
+Communication between executors within a worker process is very fast as there is neither a need to serialize and deserialize messages nor does it involve communicating over the network
+stack. In multiworker mode, messages often cross worker process boundaries. For performance sensitive cases, if it is possible to configure a topology to run as many single-worker
+instances (for ex. one worker per input partition) rather than one multiworker instance, it may yield significantly better throughput and latency on the same hardware.
+The downside to this approach is that it adds the overhead of monitoring and managing many instances rather than one multiworker instance.
diff --git a/releases/2.8.8/Project-ideas.md b/releases/2.8.8/Project-ideas.md
new file mode 100644
index 0000000000..1af4805811
--- /dev/null
+++ b/releases/2.8.8/Project-ideas.md
@@ -0,0 +1,6 @@
+---
+layout: documentation
+---
+ * **DSLs for non-JVM languages:** These DSL's should be all-inclusive and not require any Java for the creation of topologies, spouts, or bolts. Since topologies are [Thrift](https://thrift.apache.org/) structs, Nimbus is a Thrift service, and bolts can be written in any language, this is possible.
+ * **Online machine learning algorithms:** Something like [Mahout](http://mahout.apache.org/) but for online algorithms
+ * **Suite of performance benchmarks:** These benchmarks should test Storm's performance on CPU and IO intensive workloads. There should be benchmarks for different classes of applications, such as stream processing (where throughput is the priority) and distributed RPC (where latency is the priority).
diff --git a/releases/2.8.8/Rationale.md b/releases/2.8.8/Rationale.md
new file mode 100644
index 0000000000..45ff396b64
--- /dev/null
+++ b/releases/2.8.8/Rationale.md
@@ -0,0 +1,33 @@
+---
+title: Rationale
+layout: documentation
+documentation: true
+---
+The past decade has seen a revolution in data processing. MapReduce, Hadoop, and related technologies have made it possible to store and process data at scales previously unthinkable. Unfortunately, these data processing technologies are not realtime systems, nor are they meant to be. There's no hack that will turn Hadoop into a realtime system; realtime data processing has a fundamentally different set of requirements than batch processing.
+
+However, realtime data processing at massive scale is becoming more and more of a requirement for businesses. The lack of a "Hadoop of realtime" has become the biggest hole in the data processing ecosystem.
+
+Storm fills that hole.
+
+Before Storm, you would typically have to manually build a network of queues and workers to do realtime processing. Workers would process messages off a queue, update databases, and send new messages to other queues for further processing. Unfortunately, this approach has serious limitations:
+
+1. **Tedious**: You spend most of your development time configuring where to send messages, deploying workers, and deploying intermediate queues. The realtime processing logic that you care about corresponds to a relatively small percentage of your codebase.
+2. **Brittle**: There's little fault-tolerance. You're responsible for keeping each worker and queue up.
+3. **Painful to scale**: When the message throughput get too high for a single worker or queue, you need to partition how the data is spread around. You need to reconfigure the other workers to know the new locations to send messages. This introduces moving parts and new pieces that can fail.
+
+Although the queues and workers paradigm breaks down for large numbers of messages, message processing is clearly the fundamental paradigm for realtime computation. The question is: how do you do it in a way that doesn't lose data, scales to huge volumes of messages, and is dead-simple to use and operate?
+
+Storm satisfies these goals.
+
+## Why Storm is important
+
+Storm exposes a set of primitives for doing realtime computation. Like how MapReduce greatly eases the writing of parallel batch processing, Storm's primitives greatly ease the writing of parallel realtime computation.
+
+The key properties of Storm are:
+
+1. **Extremely broad set of use cases**: Storm can be used for processing messages and updating databases (stream processing), doing a continuous query on data streams and streaming the results into clients (continuous computation), parallelizing an intense query like a search query on the fly (distributed RPC), and more. Storm's small set of primitives satisfy a stunning number of use cases.
+2. **Scalable**: Storm scales to massive numbers of messages per second. To scale a topology, all you have to do is add machines and increase the parallelism settings of the topology. As an example of Storm's scale, one of Storm's initial applications processed 1,000,000 messages per second on a 10 node cluster, including hundreds of database calls per second as part of the topology. Storm's usage of Zookeeper for cluster coordination makes it scale to much larger cluster sizes.
+3. **Guarantees no data loss**: A realtime system must have strong guarantees about data being successfully processed. A system that drops data has a very limited set of use cases. Storm guarantees that every message will be processed, and this is in direct contrast with other systems like S4.
+4. **Extremely robust**: Unlike systems like Hadoop, which are notorious for being difficult to manage, Storm clusters just work. It is an explicit goal of the Storm project to make the user experience of managing Storm clusters as painless as possible.
+5. **Fault-tolerant**: If there are faults during execution of your computation, Storm will reassign tasks as necessary. Storm makes sure that a computation can run forever (or until you kill the computation).
+6. **Programming language agnostic**: Robust and scalable realtime processing shouldn't be limited to a single platform. Storm topologies and processing components can be defined in any language, making Storm accessible to nearly anyone.
diff --git a/releases/2.8.8/Resource_Aware_Scheduler_overview.md b/releases/2.8.8/Resource_Aware_Scheduler_overview.md
new file mode 100644
index 0000000000..7fb0e31e26
--- /dev/null
+++ b/releases/2.8.8/Resource_Aware_Scheduler_overview.md
@@ -0,0 +1,513 @@
+---
+title: Resource Aware Scheduler
+layout: documentation
+documentation: true
+---
+
+# Introduction
+
+The purpose of this document is to provide a description of the Resource Aware Scheduler for the Storm distributed real-time computation system. This document will provide you with both a high level description of the resource aware scheduler in Storm. Some of the benefits are using a resource aware scheduler on top of Storm is outlined in the following presentation at Hadoop Summit 2016:
+
+http://www.slideshare.net/HadoopSummit/resource-aware-scheduling-in-apache-storm
+
+# Table of Contents
+1. [Using Resource Aware Scheduler](#Using-Resource-Aware-Scheduler)
+2. [API Overview](#API-Overview)
+ 1. [Setting Memory Requirement](#Setting-Memory-Requirement)
+ 2. [Shared Memory Requirement](#Setting-Shared-Memory)
+ 3. [Setting CPU Requirement](#Setting-CPU-Requirement)
+ 4. [Limiting the Heap Size per Worker (JVM) Process](#Limiting-the-Heap-Size-per-Worker-(JVM)Process)
+ 5. [Setting Available Resources on Node](#Setting-Available-Resources-on-Node)
+ 6. [Other Configurations](#Other-Configurations)
+3. [Topology Priorities and Per User Resource](#Topology-Priorities-and-Per-User-Resource)
+ 1. [Setup](#Setup)
+ 2. [Specifying Topology Priority](#Specifying-Topology-Priority)
+ 3. [Specifying Scheduling Strategy](#Specifying-Scheduling-Strategy)
+ 4. [Specifying Topology Prioritization Strategy](#Specifying-Topology-Prioritization-Strategy)
+4. [Profiling Resource Usage](#Profiling-Resource-Usage)
+5. [Enhancements on original DefaultResourceAwareStrategy](#Enhancements-on-original-DefaultResourceAwareStrategy)
+
+<div id='Using-Resource-Aware-Scheduler'/>
+
+## Using Resource Aware Scheduler
+
+The user can switch to using the Resource Aware Scheduler by setting the following in *conf/storm.yaml*
+```
+ storm.scheduler: “org.apache.storm.scheduler.resource.ResourceAwareScheduler”
+```
+<div id='API-Overview'/>
+
+## API Overview
+
+For use with Trident, please see the [Trident RAS API](Trident-RAS-API.html)
+
+For a Storm Topology, the user can now specify the amount of resources a topology component (i.e. Spout or Bolt) is required to run a single instance of the component. The user can specify the resource requirement for a topology component by using the following API calls.
+
+<div id='Setting-Memory-Requirement'/>
+
+### Setting Memory Requirement
+
+API to set component memory requirement:
+```
+ public T setMemoryLoad(Number onHeap, Number offHeap)
+```
+Parameters:
+* Number onHeap – The amount of on heap memory an instance of this component will consume in megabytes
+* Number offHeap – The amount of off heap memory an instance of this component will consume in megabytes
+
+The user also has to option to just specify the on heap memory requirement if the component does not have an off heap memory need.
+```
+ public T setMemoryLoad(Number onHeap)
+```
+Parameters:
+* Number onHeap – The amount of on heap memory an instance of this component will consume
+
+If no value is provided for offHeap, 0.0 will be used. If no value is provided for onHeap, or if the API is never called for a component, the default value will be used.
+
+Example of Usage:
+```
+ SpoutDeclarer s1 = builder.setSpout("word", new TestWordSpout(), 10);
+ s1.setMemoryLoad(1024.0, 512.0);
+ builder.setBolt("exclaim1", new ExclamationBolt(), 3)
+ .shuffleGrouping("word").setMemoryLoad(512.0);
+```
+The entire memory requested for this topology is 16.5 GB. That is from 10 spouts with 1GB on heap memory and 0.5 GB off heap memory each and 3 bolts with 0.5 GB on heap memory each.
+
+<div id='Setting-Shared-Memory'/>
+
+### Shared Memory
+
+In some cases you may have memory that is shared between components. It may be a as simple as a large static data structure, or as complex as static data that is memory mapped into a bolt and is shared across workers. In any case you can specify your shared memory request by
+creating one of `SharedOffHeapWithinNode`, `SharedOffHeapWithinWorker`, or `SharedOnHeap` and adding it to bolts and spouts that use that shared memory.
+
+Example of Usage:
+
+```
+ builder.setBolt("exclaim1", new ExclamationBolt(), 3).shuffleGrouping("word")
+ .addSharedMemory(new SharedOnHeap(100, "exclaim-cache"));
+```
+
+In the above example all of the "exclaim1" bolts in a worker will share 100MB of memory.
+
+```
+ builder.setBolt("lookup", new LookupBolt(), 3).shuffleGrouping("spout")
+ .addSharedMemory(new SharedOffHeapWithinNode(500, "static-lookup"));
+```
+
+In this example all "lookup" bolts on a given node will share 500 MB or memory off heap.
+
+
+<div id='Setting-CPU-Requirement'/>
+
+### Setting CPU Requirement
+
+API to set component CPU requirement:
+```
+ public T setCPULoad(Double amount)
+```
+Parameters:
+* Number amount – The amount of on CPU an instance of this component will consume.
+
+Currently, the amount of CPU resources a component requires or is available on a node is represented by a point system. CPU usage is a difficult concept to define. Different CPU architectures perform differently depending on the task at hand. They are so complex that expressing all of that in a single precise portable number is impossible. Instead we take a convention over configuration approach and are primarily concerned with rough level of CPU usage while still providing the possibili [...]
+
+By convention a CPU core typically will get 100 points. If you feel that your processors are more or less powerful you can adjust this accordingly. Heavy tasks that are CPU bound will get 100 points, as they can consume an entire core. Medium tasks should get 50, light tasks 25, and tiny tasks 10. In some cases you have a task that spawns other threads to help with processing. These tasks may need to go above 100 points to express the amount of CPU they are using. If these conventions ar [...]
+
+Example of Usage:
+```
+ SpoutDeclarer s1 = builder.setSpout("word", new TestWordSpout(), 10);
+ s1.setCPULoad(15.0);
+ builder.setBolt("exclaim1", new ExclamationBolt(), 3)
+ .shuffleGrouping("word").setCPULoad(10.0);
+ builder.setBolt("exclaim2", new HeavyBolt(), 1)
+ .shuffleGrouping("exclaim1").setCPULoad(450.0);
+```
+
+<div id='Limiting-the-Heap-Size-per-Worker-(JVM)Process'/>
+
+### Limiting the Heap Size per Worker (JVM) Process
+
+```
+ public void setTopologyWorkerMaxHeapSize(Number size)
+```
+
+Parameters:
+* Number size – The memory limit a worker process will be allocated in megabytes
+
+The user can limit the amount of memory resources the resource aware scheduler allocates to a single worker on a per topology basis by using the above API. This API is in place so that the users can spread executors to multiple workers. However, spreading executors to multiple workers may increase the communication latency since executors will not be able to use Disruptor Queue for intra-process communication.
+
+Example of Usage:
+```
+ Config conf = new Config();
+ conf.setTopologyWorkerMaxHeapSize(512.0);
+```
+
+<div id='Setting-Available-Resources-on-Node'/>
+
+### Setting Available Resources on Node
+
+A storm administrator can specify node resource availability by modifying the *conf/storm.yaml* file located in the storm home directory of that node.
+
+A storm administrator can specify how much available memory a node has in megabytes adding the following to *storm.yaml*
+```
+ supervisor.memory.capacity.mb: [amount<Double>]
+```
+A storm administrator can also specify how much available CPU resources a node has available adding the following to *storm.yaml*
+```
+ supervisor.cpu.capacity: [amount<Double>]
+```
+
+Note: that the amount the user can specify for the available CPU is represented using a point system like discussed earlier.
+
+Example of Usage:
+```
+ supervisor.memory.capacity.mb: 20480.0
+ supervisor.cpu.capacity: 100.0
+```
+
+<div id='Other-Configurations'/>
+
+### Other Configurations
+
+The user can set some default configurations for the Resource Aware Scheduler in *conf/storm.yaml*:
+
+```
+ //default value if on heap memory requirement is not specified for a component
+ topology.component.resources.onheap.memory.mb: 128.0
+
+ //default value if off heap memory requirement is not specified for a component
+ topology.component.resources.offheap.memory.mb: 0.0
+
+ //default value if CPU requirement is not specified for a component
+ topology.component.cpu.pcore.percent: 10.0
+
+ //default value for the max heap size for a worker
+ topology.worker.max.heap.size.mb: 768.0
+```
+
+### Warning
+
+If Resource Aware Scheduling is enabled, it will dynamically calculate the number of workers and the `topology.workers` setting is ignored.
+
+<div id='Topology-Priorities-and-Per-User-Resource'/>
+
+## Topology Priorities and Per User Resource
+
+The Resource Aware Scheduler or RAS also has multi-tenant capabilities since many Storm users typically share a Storm cluster. Resource Aware Scheduler can allocate resources on a per user basis. Each user can be guaranteed a certain amount of resources to run his or her topologies and the Resource Aware Scheduler will meet those guarantees when possible. When the Storm cluster has extra free resources, Resource Aware Scheduler will to be able allocate additional resources to user in [...]
+
+<div id='Setup'/>
+
+### Setup
+
+The resource guarantees of a user can be specified *conf/user-resource-pools.yaml*. Specify the resource guarantees of a user in the following format:
+```
+ resource.aware.scheduler.user.pools:
+ [UserId]
+ cpu: [Amount of Guarantee CPU Resources]
+ memory: [Amount of Guarantee Memory Resources]
+```
+An example of what *user-resource-pools.yaml* can look like:
+```
+ resource.aware.scheduler.user.pools:
+ jerry:
+ cpu: 1000
+ memory: 8192.0
+ derek:
+ cpu: 10000.0
+ memory: 32768
+ bobby:
+ cpu: 5000.0
+ memory: 16384.0
+```
+Please note that the specified amount of Guaranteed CPU and Memory can be either a integer or double
+
+<div id='Specifying-Topology-Priority'/>
+
+### Specifying Topology Priority
+
+The range of topology priorities can range form 0-29. The topologies priorities will be partitioned into several priority levels that may contain a range of priorities.
+For example we can create a priority level mapping:
+
+ PRODUCTION => 0 – 9
+ STAGING => 10 – 19
+ DEV => 20 – 29
+
+Thus, each priority level contains 10 sub priorities. Users can set the priority level of a topology by using the following API
+```
+ conf.setTopologyPriority(int priority)
+```
+Parameters:
+* priority – an integer representing the priority of the topology
+
+Please note that the 0-29 range is not a hard limit. Thus, a user can set a priority number that is higher than 29. However, the property of higher the priority number, lower the importance still holds
+
+<div id='Specifying-Scheduling-Strategy'/>
+
+### Specifying Scheduling Strategy
+
+A user can specify on a per topology basis what scheduling strategy to use. Users can implement the IStrategy interface and define new strategies to schedule specific topologies. This pluggable interface was created since we realize different topologies might have different scheduling needs. A user can set the topology strategy within the topology definition by using the API:
+```
+ public void setTopologyStrategy(Class<? extends IStrategy> clazz)
+```
+Parameters:
+* clazz – The strategy class that implements the IStrategy interface
+
+Example Usage:
+```
+ conf.setTopologyStrategy(org.apache.storm.scheduler.resource.strategies.scheduling.DefaultResourceAwareStrategy.class);
+```
+A default scheduling is provided. The DefaultResourceAwareStrategy is implemented based off the scheduling algorithm in the original paper describing resource aware scheduling in Storm:
+
+Peng, Boyang, Mohammad Hosseini, Zhihao Hong, Reza Farivar, and Roy Campbell. "R-storm: Resource-aware scheduling in storm." In Proceedings of the 16th Annual Middleware Conference, pp. 149-161. ACM, 2015.
+
+http://dl.acm.org/citation.cfm?id=2814808
+
+**Please Note: Enhancements have to made on top of the original scheduling strategy as described in the paper. Please see section "Enhancements on original DefaultResourceAwareStrategy"**
+
+<div id='Specifying-Topology-Prioritization-Strategy'/>
+
+### Specifying Topology Prioritization Strategy
+
+The order of scheduling and eviction is determined by a pluggable interface in which the cluster owner can define how topologies should be scheduled. For the owner to define his or her own prioritization strategy, she or he needs to implement the ISchedulingPriorityStrategy interface. A user can set the scheduling priority strategy by setting the `DaemonConfig.RESOURCE_AWARE_SCHEDULER_PRIORITY_STRATEGY` to point to the class that implements the strategy. For instance:
+```
+ resource.aware.scheduler.priority.strategy: "org.apache.storm.scheduler.resource.strategies.priority.DefaultSchedulingPriorityStrategy"
+```
+
+Topologies are scheduled starting at the beginning of the list returned by this plugin. If there are not enough resources to schedule the topology others are evicted starting at the end of the list. Eviction stops when there are no lower priority topologies left to evict.
+
+**DefaultSchedulingPriorityStrategy**
+
+In the past the order of scheduling was based on the distance between a user’s current resource allocation and his or her guaranteed allocation.
+
+We currently use a slightly different approach. We simulate scheduling the highest priority topology for each user and score the topology for each of the resources using the formula
+
+```
+(Requested + Assigned - Guaranteed)/Available
+```
+
+Where
+
+ * `Requested` is the resource requested by this topology (or a approximation of it for complex requests like shared memory)
+ * `Assigned` is the resources already assigned by the simulation.
+ * `Guaranteed` is the resource guarantee for this user
+ * `Available` is the amount of that resource currently available in the cluster.
+
+This gives a score that is negative for guaranteed requests and a score that is positive for requests that are not within the guarantee.
+
+To combine different resources the maximum of all the individual resource scores is used. This guarantees that if a user would go over a guarantee for a single resource it would not be offset by being under guarantee on any other resources.
+
+For Example:
+
+Assume we have to schedule the following topologies.
+
+|ID|User|CPU|Memory|Priority|
+|---|----|---|------|-------|
+|A-1|A|100|1,000|1|
+|A-2|A|100|1,000|10|
+|B-1|B|100|1,000|1|
+|B-2|B|100|1,000|10|
+
+The cluster as a whole has 300 CPU and 4,000 Memory.
+
+User A is guaranteed 100 CPU and 1,000 Memory. User B is guaranteed 200 CPU and 1,500 Memory. The scores for the most important, lowest priority number, topologies for each user would be.
+
+```
+A-1 Score = max(CPU: (100 + 0 - 100)/300, MEM: (1,000 + 0 - 1,000)/4,000) = 0
+B-1 Score = max(CPU: (100 + 0 - 200)/300, MEM: (1,000 + 0 - 1,500)/4,000) = -0.125
+```
+
+`B-1` has the lowest score so it would be the highest priority topology to schedule. In the next round the scores would be.
+
+```
+A-1 Score = max(CPU: (100 + 0 - 100)/200, MEM: (1,000 + 0 - 1,000)/3,000) = 0
+B-2 Score = max(CPU: (100 + 100 - 200)/200, MEM: (1,000 + 1,000 - 1,500)/3,000) = 0.167
+```
+
+`A-1` has the lowest score now so it would be the next highest priority topology to schedule.
+
+This process would be repeated until all of the topologies are ordered, even if there are no resources left on the cluster to schedule a topology.
+
+**FIFOSchedulingPriorityStrategy**
+
+The FIFO strategy is intended more for test or staging clusters where users are running integration tests or doing development work. Topologies in these situations tend to be short lived and at times a user may forget that they are running a topology at all.
+
+To try and be as fair as possible to users running short lived topologies the `FIFOSchedulingPriorityStrategy` extends the `DefaultSchedulingPriorityStrategy` so that any negative score (a.k.a. a topology that fits within a user's guarantees) would remain unchanged, but positive scores are replaced with the up-time of the topology.
+
+This respects the guarantees of a user, but at the same time it gives priority for the rest of the resources to the most recently launched topology. Older topologies, that have probably been forgotten about, are then least likely to get resources.
+
+<div id='Profiling-Resource-Usage'/>
+
+## Profiling Resource Usage
+
+Figuring out resource usage for your topology:
+
+To get an idea of how much memory/CPU your topology is actually using you can add the following to your topology launch code.
+
+```
+ //Log all storm metrics
+ conf.registerMetricsConsumer(backtype.storm.metric.LoggingMetricsConsumer.class);
+
+ //Add in per worker CPU measurement
+ Map<String, String> workerMetrics = new HashMap<String, String>();
+ workerMetrics.put("CPU", "org.apache.storm.metrics.sigar.CPUMetric");
+ conf.put(Config.TOPOLOGY_WORKER_METRICS, workerMetrics);
+```
+
+The CPU metrics will require you to add
+
+```
+ <dependency>
+ <groupId>org.apache.storm</groupId>
+ <artifactId>storm-metrics</artifactId>
+ <version>1.0.0</version>
+ </dependency>
+```
+
+as a topology dependency (1.0.0 or higher).
+
+You can then go to your topology on the UI, turn on the system metrics, and find the log that the LoggingMetricsConsumer is writing to. It will output results in the log like.
+
+```
+ 1454526100 node1.nodes.com:6707 -1:__system CPU {user-ms=74480, sys-ms=10780}
+ 1454526100 node1.nodes.com:6707 -1:__system memory/nonHeap {unusedBytes=2077536, virtualFreeBytes=-64621729, initBytes=2555904, committedBytes=66699264, maxBytes=-1, usedBytes=64621728}
+ 1454526100 node1.nodes.com:6707 -1:__system memory/heap {unusedBytes=573861408, virtualFreeBytes=694644256, initBytes=805306368, committedBytes=657719296, maxBytes=778502144, usedBytes=83857888}
+```
+
+The metrics with -1:__system are generally metrics for the entire worker. In the example above that worker is running on node1.nodes.com:6707. These metrics are collected every 60 seconds. For the CPU you can see that over the 60 seconds this worker used 74480 + 10780 = 85260 ms of CPU time. This is equivalent to 85260/60000 or about 1.5 cores.
+
+The Memory usage is similar but look at the usedBytes. offHeap is 64621728 or about 62MB, and onHeap is 83857888 or about 80MB, but you should know what you set your heap to in each of your workers already. How do you divide this up per bolt/spout? That is a bit harder and may require some trial and error from your end.
+
+<div id='Enhancements-on-original-DefaultResourceAwareStrategy'/>
+
+## Enhancements on original DefaultResourceAwareStrategy
+
+The default resource aware scheduling strategy as described in the paper above has two main scheduling phases:
+
+1. Task Selection - Calculate the order task/executors in a topology should be scheduled
+2. Node Selection - Given a task/executor, find a node to schedule the task/executor on.
+
+Enhancements have been made for both scheduling phases
+
+### Task Selection Enhancements
+
+Instead of using a breadth first traversal of the topology graph to create a ordering of components and its executors, a new heuristic is used that orders components by the number of in and out edges (potential connections) of the component. This is discovered to be a more effective way to co-locate executors that communicate with each other and reduce the network latency.
+
+
+### Node Selection Enhancements
+
+Node selection comes down first selecting which rack (server rack) and then which node on that rack to choose. The gist of strategy in choosing a rack and node is finding the rack that has the "most" resource available and in that rack find the node with the "most" free resources. The assumption we are making for this strategy is that the node or rack with the most free resources will have the highest probability that allows us to schedule co-locate the most number of executors on the n [...]
+
+Racks and nodes will be sorted from best choice to worst choice. When finding an executor, the strategy will iterate through all racks and nodes, starting from best to worst, before giving up. Racks and nodes will be sorted in the following matter:
+
+1. How many executors are already scheduled on the rack or node
+ -- This is done so we move executors to schedule closer to executors that are already scheduled and running. If a topology partially crashed and a subset of the topology's executors need to be rescheduled, we want to reschedule these executors as close (network wise) as possible to the executors that healthy and running.
+
+2. Subordinate resource availability or the amount "effective" resources on the rack or node
+ -- Please refer the section on Subordinate Resource Availability
+
+3. Average of the all the resource availability
+ -- This is simply taking the average of the percent available (available resources on node or rack divided by the available resources on rack or cluster, respectively). This situation will only be used when "effective resources" for two objects (rack or node) are the same. Then we consider the average of all the percentages of resources as a metric for sorting. For example:
+```
+ Avail Resources:
+ node 1: CPU = 50 Memory = 1024 Slots = 20
+ node 2: CPU = 50 Memory = 8192 Slots = 40
+ node 3: CPU = 1000 Memory = 0 Slots = 0
+
+ Effective resources for nodes:
+ node 1 = 50 / (50+50+1000) = 0.045 (CPU bound)
+ node 2 = 50 / (50+50+1000) = 0.045 (CPU bound)
+ node 3 = 0 (memory and slots are 0)
+```
+ode 1 and node 2 have the same effective resources but clearly node 2 has more resources (memory and slots) than node 1 and we would want to pick node 2 first since there is a higher probability we will be able to schedule more executors on it. This is what the phase 2 averaging does
+
+Thus the sorting follows the following progression. Compare based on 1) and if equal then compare based on 2) and if equal compare based on 3) and if equal we break ties by arbitrarily assigning ordering based on comparing the ids of the node or rack.
+
+**Subordinate Resource Availability**
+
+Originally the getBestClustering algorithm for RAS finds the "Best" rack based on which rack has the "most available" resources by finding the rack with the biggest sum of available memory + available across all nodes in the rack. This method is not very accurate since memory and cpu usage agree values on a different scale and the values are not normalized. This method is also not effective since it does not consider the number of slots available and it fails to identifying racks that ar [...]
+
+The new strategy/algorithm to find the "best" rack or node, I dub subordinate resource availability ordering (inspired by Dominant Resource Fairness), sorts racks and nodes by the subordinate (not dominant) resource availability.
+
+For example given 4 racks with the following resource availabilities
+```
+ //generate some that has a lot of memory but little of cpu
+ rack-3 Avail [ CPU 100.0 MEM 200000.0 Slots 40 ] Total [ CPU 100.0 MEM 200000.0 Slots 40 ]
+ //generate some supervisors that are depleted of one resource
+ rack-2 Avail [ CPU 0.0 MEM 80000.0 Slots 40 ] Total [ CPU 0.0 MEM 80000.0 Slots 40 ]
+ //generate some that has a lot of cpu but little of memory
+ rack-4 Avail [ CPU 6100.0 MEM 10000.0 Slots 40 ] Total [ CPU 6100.0 MEM 10000.0 Slots 40 ]
+ //generate another rack of supervisors with less resources than rack-0
+ rack-1 Avail [ CPU 2000.0 MEM 40000.0 Slots 40 ] Total [ CPU 2000.0 MEM 40000.0 Slots 40 ]
+ //best rack to choose
+ rack-0 Avail [ CPU 4000.0 MEM 80000.0 Slots 40( ] Total [ CPU 4000.0 MEM 80000.0 Slots 40 ]
+ Cluster Overall Avail [ CPU 12200.0 MEM 410000.0 Slots 200 ] Total [ CPU 12200.0 MEM 410000.0 Slots 200 ]
+```
+It is clear that rack-0 is the best cluster since its the most balanced and can potentially schedule the most executors, while rack-2 is the worst rack since rack-2 is depleted of cpu resource thus rendering it unschedulable even though there are other resources available.
+
+We first calculate the resource availability percentage of all the racks for each resource by computing:
+
+ (resource available on rack) / (resource available in cluster)
+
+We do this calculation to normalize the values otherwise the resource values would not be comparable.
+
+So for our example:
+```
+ rack-3 Avail [ CPU 0.819672131147541% MEM 48.78048780487805% Slots 20.0% ] effective resources: 0.00819672131147541
+ rack-2 Avail [ 0.0% MEM 19.51219512195122% Slots 20.0% ] effective resources: 0.0
+ rack-4 Avail [ CPU 50.0% MEM 2.4390243902439024% Slots 20.0% ] effective resources: 0.024390243902439025
+ rack-1 Avail [ CPU 16.39344262295082% MEM 9.75609756097561% Slots 20.0% ] effective resources: 0.0975609756097561
+ rack-0 Avail [ CPU 32.78688524590164% MEM 19.51219512195122% Slots 20.0% ] effective resources: 0.1951219512195122
+```
+The effective resource of a rack, which is also the subordinate resource, is computed by:
+
+ MIN(resource availability percentage of {CPU, Memory, # of free Slots}).
+
+Then we order the racks by the effective resource.
+
+Thus for our example:
+
+ Sorted rack: [rack-0, rack-1, rack-4, rack-3, rack-2]
+
+This metric is used in sorting for both nodes and racks. When sorting racks, we consider resources available on the rack and in the whole cluster (containing all racks). When sorting nodes, we consider resources available on the node and the resources available in the rack (sum of all resources available for all nodes in rack)
+
+Original Jira for this enhancement: [STORM-1766](https://issues.apache.org/jira/browse/STORM-1766)
+
+### Improvements in Scheduling
+This section provides some experimental results on the performance benefits with the enhancements on top of the original scheduling strategy. The experiments are based off of running simulations using:
+
+https://github.com/jerrypeng/storm-scheduler-test-framework
+
+Random topologies and clusters are used in the simulation as well as a comprehensive dataset consisting of all real topologies running in all the storm clusters at Yahoo.
+
+The below graphs provides a comparison of how well the various strategies schedule topologies to minimize network latency. A network metric is calculated for each scheduling of a topology by each scheduling strategy. The network metric is calculated based on how many connections each executor in a topology has to make to another executor residing in the same worker (JVM process), in different worker but same host, different host, different rack. The assumption we are making is the following
+
+1. Intra-worker communication is the fastest
+2. Inter-worker communication is fast
+3. Inter-node communication is slower
+4. Inter-rack communication is the slowest
+
+For this network metric, the larger the number is number is the more potential network latency the topology will have for this scheduling. Two types of experiments are performed. One set experiments are performed with randomly generated topologies and randomly generate clusters. The other set of experiments are performed with a dataset containing all of the running topologies at yahoo and semi-randomly generated clusters based on the size of the topology. Both set of experiments are [...]
+
+For the experiments involving randomly generated topologies, an optimal strategy is implemented that exhaustively finds the most optimal solution if a solution exists. The topologies and clusters used in this experiment are relatively small so that the optimal strategy traverse to solution space to find a optimal solution in a reasonable amount of time. This strategy is not run with the Yahoo topologies since the topologies are large and would take unreasonable amount of time to run, s [...]
+
+| Random Topologies | Yahoo topologies |
+|-------------------|------------------|
+|| |
+
+The next graph displays how close the schedulings from the respectively scheduling strategies are to the schedulings of the optimal strategy. As explained earlier, this is only done for the random generated topologies and clusters.
+
+| Random Topologies |
+|-------------------|
+||
+
+The below graph is a CDF of the network metric:
+
+| Random Topologies | Yahoo topologies |
+|-------------------|------------------|
+|| |
+
+Below is a comparison of the how long the strategies take to run:
+
+| Random Topologies | Yahoo topologies |
+|-------------------|------------------|
+|| |
+
diff --git a/releases/2.8.8/Running-topologies-on-a-production-cluster.md b/releases/2.8.8/Running-topologies-on-a-production-cluster.md
new file mode 100644
index 0000000000..f9de3e49b5
--- /dev/null
+++ b/releases/2.8.8/Running-topologies-on-a-production-cluster.md
@@ -0,0 +1,77 @@
+---
+title: Running Topologies on a Production Cluster
+layout: documentation
+documentation: true
+---
+Running topologies on a production cluster is similar to running in [Local mode](Local-mode.html). Here are the steps:
+
+1) Define the topology (Use [TopologyBuilder](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/topology/TopologyBuilder.html) if defining using Java)
+
+2) Use [StormSubmitter](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/StormSubmitter.html) to submit the topology to the cluster. `StormSubmitter` takes as input the name of the topology, a configuration for the topology, and the topology itself. For example:
+
+```java
+Config conf = new Config();
+conf.setNumWorkers(20);
+conf.setMaxSpoutPending(5000);
+StormSubmitter.submitTopology("mytopology", conf, topology);
+```
+
+3) Create a JAR containing your topology code. You have the option to either bundle all of the dependencies of your code into that JAR (except for Storm -- the Storm JARs will be added to the classpath on the worker nodes), or you can leverage the [Classpath handling](Classpath-handling.html) features in Storm for using external libraries without bundling them into your topology JAR.
+
+If you're using Maven, the [Maven Assembly Plugin](http://maven.apache.org/plugins/maven-assembly-plugin/) can do the packaging for you. Just add this to your pom.xml:
+
+```xml
+ <plugin>
+ <artifactId>maven-assembly-plugin</artifactId>
+ <configuration>
+ <descriptorRefs>
+ <descriptorRef>jar-with-dependencies</descriptorRef>
+ </descriptorRefs>
+ <archive>
+ <manifest>
+ <mainClass>com.path.to.main.Class</mainClass>
+ </manifest>
+ </archive>
+ </configuration>
+ </plugin>
+```
+Then run mvn assembly:assembly to get an appropriately packaged jar. Make sure you [exclude](http://maven.apache.org/plugins/maven-assembly-plugin/examples/single/including-and-excluding-artifacts.html) the Storm jars since the cluster already has Storm on the classpath.
+
+4) Submit the topology to the cluster using the `storm` client, specifying the path to your jar, the classname to run, and any arguments it will use:
+
+`storm jar path/to/allmycode.jar org.me.MyTopology arg1 arg2 arg3`
+
+`storm jar` will submit the jar to the cluster and configure the `StormSubmitter` class to talk to the right cluster. In this example, after uploading the jar `storm jar` calls the main function on `org.me.MyTopology` with the arguments "arg1", "arg2", and "arg3".
+
+You can find out how to configure your `storm` client to talk to a Storm cluster on [Setting up development environment](Setting-up-development-environment.html).
+
+### Common configurations
+
+There are a variety of configurations you can set per topology. A list of all the configurations you can set can be found [here](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html). The ones prefixed with "TOPOLOGY" can be overridden on a topology-specific basis (the other ones are cluster configurations and cannot be overridden). Here are some common ones that are set for a topology:
+
+1. **Config.TOPOLOGY_WORKERS**: This sets the number of worker processes to use to execute the topology. For example, if you set this to 25, there will be 25 Java processes across the cluster executing all the tasks. If you had a combined 150 parallelism across all components in the topology, each worker process will have 6 tasks running within it as threads.
+2. **Config.TOPOLOGY_ACKER_EXECUTORS**: This sets the number of executors that will track tuple trees and detect when a spout tuple has been fully processed. Ackers are an integral part of Storm's reliability model and you can read more about them on [Guaranteeing message processing](Guaranteeing-message-processing.html). By not setting this variable or setting it as null, Storm will set the number of acker executors to be equal to the number of workers configured for this topology. If t [...]
+3. **Config.TOPOLOGY_MAX_SPOUT_PENDING**: This sets the maximum number of spout tuples that can be pending on a single spout task at once (pending means the tuple has not been acked or failed yet). It is highly recommended you set this config to prevent queue explosion.
+4. **Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS**: This is the maximum amount of time a spout tuple has to be fully completed before it is considered failed. This value defaults to 30 seconds, which is sufficient for most topologies. See [Guaranteeing message processing](Guaranteeing-message-processing.html) for more information on how Storm's reliability model works.
+5. **Config.TOPOLOGY_SERIALIZATIONS**: You can register more serializers to Storm using this config so that you can use custom types within tuples.
+
+
+### Killing a topology
+
+To kill a topology, simply run:
+
+`storm kill {stormname}`
+
+Give the same name to `storm kill` as you used when submitting the topology.
+
+Storm won't kill the topology immediately. Instead, it deactivates all the spouts so that they don't emit any more tuples, and then Storm waits Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS seconds before destroying all the workers. This gives the topology enough time to complete any tuples it was processing when it got killed.
+
+### Updating a running topology
+
+To update a running topology, the only option currently is to kill the current topology and resubmit a new one. A planned feature is to implement a `storm swap` command that swaps a running topology with a new one, ensuring minimal downtime and no chance of both topologies processing tuples at the same time.
+
+### Monitoring topologies
+
+The best place to monitor a topology is using the Storm UI. The Storm UI provides information about errors happening in tasks and fine-grained stats on the throughput and latency performance of each component of each running topology.
+
+You can also look at the worker logs on the cluster machines.
diff --git a/releases/2.8.8/SECURITY.md b/releases/2.8.8/SECURITY.md
new file mode 100644
index 0000000000..e3384a7fa5
--- /dev/null
+++ b/releases/2.8.8/SECURITY.md
@@ -0,0 +1,752 @@
+---
+title: Running Apache Storm Securely
+layout: documentation
+documentation: true
+---
+
+# Running Apache Storm Securely
+
+Apache Storm offers a range of configuration options when trying to secure
+your cluster. By default all authentication and authorization is disabled but
+can be turned on as needed.
+
+## Firewall/OS level Security
+
+You can still have a secure storm cluster without turning on formal
+Authentication and Authorization. But to do so usually requires
+configuring your Operating System to restrict the operations that can be done.
+This is generally a good idea even if you plan on running your cluster with Auth.
+
+Storm's OS level security relies on running Storm processes using OS accounts that have only the permissions they need.
+Note that workers run under the same OS account as the Supervisor daemon by default.
+
+The exact detail of how to setup these precautions varies a lot and is beyond
+the scope of this document.
+
+It is generally a good idea to enable a firewall and restrict incoming network
+connections to only those originating from the cluster itself and from trusted
+hosts and services, a complete list of ports storm uses are below.
+
+If the data your cluster is processing is sensitive it might be best to setup
+IPsec to encrypt all traffic being sent between the hosts in the cluster.
+
+### Ports
+
+| Default Port | Storm Config | Client Hosts/Processes | Server |
+|--------------|--------------|------------------------|--------|
+| 2181 | `storm.zookeeper.port` | Nimbus, Supervisors, and Worker processes | Zookeeper |
+| 6627 | `nimbus.thrift.port` | Storm clients, Supervisors, and UI | Nimbus |
+| 6628 | `supervisor.thrift.port` | Nimbus | Supervisors |
+| 8080 | `ui.port` | Client Web Browsers | UI |
+| 8000 | `logviewer.port` | Client Web Browsers | Logviewer |
+| 3772 | `drpc.port` | External DRPC Clients | DRPC |
+| 3773 | `drpc.invocations.port` | Worker Processes | DRPC |
+| 3774 | `drpc.http.port` | External HTTP DRPC Clients | DRPC |
+| 670{0,1,2,3} | `supervisor.slots.ports` | Worker Processes | Worker Processes |
+
+
+### UI/Logviewer
+
+The UI and logviewer processes provide a way to not only see what a cluster is
+doing, but also manipulate running topologies. In general these processes should
+not be exposed except to users of the cluster.
+
+Some form of Authentication is typically required, and can be done using a java servlet filter
+
+```yaml
+ui.filter: "filter.class"
+ui.filter.params: "param1":"value1"
+logviewer.filter: "filter.class"
+logviewer.filter.params: "param1":"value1"
+```
+
+The `ui.filter` is an instance of `javax.servlet.Filter` that is intended to
+filter all incoming requests to the UI and authenticate the request mapping
+it to a "user". Typically this is done by modifying or wrapping the
+`HttpServletRequest` to return the user principal through the
+`getUserPrincipal()` method or returning the user name through the
+`getRemoteUser()` method. If your filter authenticates in a different way you
+can look at setting `ui.http.creds.plugin` to point to an instance of `IHttpCredentialsPlugin`
+that can take the `HttpServletRequest` and return a user name and populate the needed fields
+in the current `ReqContext`. These are advanced features and you may want to look at the
+`DefaultHttpCredentialsPlugin` as an example of how to do this.
+
+These same settings apply to the logviewer too. If you want to have separate control
+over how authentication works in the logviewer you may optionally set `logviewer.filter`
+instead and it will override any `ui.filter` settings for the logviewer process.
+
+If the cluster is single tenant you might want to just restrict access to the UI/log
+viewers ports to only accept connections from local hosts, and then front them with
+another web server, like Apache httpd, that can authenticate/authorize incoming connections and
+proxy the connection to the storm process. To make this work the ui process must have
+logviewer.port set to the port of the proxy in its storm.yaml, while the logviewers
+must have it set to the actual port that they are going to bind to.
+
+The servlet filters are preferred because it allows individual topologies to
+specify who is and who is not allowed to access the pages associated with
+them.
+
+Storm UI (or logviewer) can be configured to use AuthenticationFilter from hadoop-auth.
+```yaml
+ui.filter: "org.apache.hadoop.security.authentication.server.AuthenticationFilter"
+ui.filter.params:
+ "type": "kerberos"
+ "kerberos.principal": "HTTP/nimbus.witzend.com"
+ "kerberos.keytab": "/vagrant/keytabs/http.keytab"
+ "kerberos.name.rules": "RULE:[2:$1@$0]([jt]t@.*EXAMPLE.COM)s/.*/$MAPRED_USER/ RULE:[2:$1@$0]([nd]n@.*EXAMPLE.COM)s/.*/$HDFS_USER/DEFAULT"
+```
+make sure to create a principal 'HTTP/{hostname}' (here hostname should be the one where UI daemon runs
+Be aware that the UI user *MUST* be HTTP.
+
+Once configured users needs to do kinit before accessing UI.
+Ex:
+curl -i --negotiate -u:anyUser -b ~/cookiejar.txt -c ~/cookiejar.txt http://storm-ui-hostname:8080/api/v1/cluster/summary
+
+1. Firefox: Goto about:config and search for network.negotiate-auth.trusted-uris double-click to add value "http://storm-ui-hostname:8080"
+2. Google-chrome: start from command line with: google-chrome --auth-server-whitelist="*storm-ui-hostname" --auth-negotiate-delegate-whitelist="*storm-ui-hostname"
+3. IE: Configure trusted websites to include "storm-ui-hostname" and allow negotiation for that website
+
+**Note**: For viewing any logs via `logviewer` in secure mode, all the hosts that runs `logviewer` should also be added to the above white list. For big clusters you could white list the host's domain (for e.g. set `network.negotiate-auth.trusted-uris` to `.yourdomain.com`).
+
+**Caution**: In AD MIT Keberos setup the key size is bigger than the default UI jetty server request header size. Make sure you set ui.header.buffer.bytes to 65536 in storm.yaml. More details are on [STORM-633](https://issues.apache.org/jira/browse/STORM-633)
+
+
+## DRPC HTTP
+
+The DRPC server optionally supports a REST endpoint as well, and you can configure authentication
+on that endpoint similar to the ui/logviewer.
+
+The `drpc.http.filter` and `drpc.http.filter.params` configs can be used to setup a `Filter` for the DRPC server. Unlike the logviewer
+it does not fall back to the UI configs as the DRPC server is intended to be REST only and often will be hit by headless users.
+
+The `drpc.http.creds.plugin` config can be used in cases where the default plugin is not good enough because of how authentication happens.
+
+
+## UI / DRPC / LOGVIEWER SSL
+
+UI,DRPC and LOGVIEWER allows users to configure ssl .
+
+### UI
+
+These configurations enable HTTPS for the Storm UI, allowing secure communication between users and the cluster.
+
+For UI users needs to set following config in storm.yaml. Generating keystores with proper keys and certs should be taken care by the user before this step.
+
+1. ui.https.port (Port for HTTPS UI access)
+2. ui.https.keystore.type (example "jks")
+3. ui.https.keystore.path (example "/etc/ssl/storm_keystore.jks")
+4. ui.https.keystore.password (keystore password)
+5. ui.https.key.password (private key password)
+
+optional config
+6. ui.https.truststore.path (example "/etc/ssl/storm_truststore.jks")
+7. ui.https.truststore.password (truststore password)
+8. ui.https.truststore.type (example "jks")
+
+If users want to setup 2-way auth
+9. ui.https.want.client.auth (If this set to true server requests for client certificate authentication, but keeps the connection if no authentication provided)
+10. ui.https.need.client.auth (If this set to true server requires client to provide authentication)
+
+
+
+
+### DRPC
+similarly to UI , users need to configure following for DRPC
+
+1. drpc.https.port
+2. drpc.https.keystore.type (example "jks")
+3. drpc.https.keystore.path (example "/etc/ssl/storm_keystore.jks")
+4. drpc.https.keystore.password (keystore password)
+5. drpc.https.key.password (private key password)
+
+optional config
+6. drpc.https.truststore.path (example "/etc/ssl/storm_truststore.jks")
+7. drpc.https.truststore.password (truststore password)
+8. drpc.https.truststore.type (example "jks")
+
+If users want to setup 2-way auth
+9. drpc.https.want.client.auth (If this set to true server requests for client certificate authentication, but keeps the connection if no authentication provided)
+10. drpc.https.need.client.auth (If this set to true server requires client to provide authentication)
+
+
+
+
+### LOGVIEWER
+similarly to UI and DRPC , users need to configure following for LOGVIEWER
+
+1. logviewer.https.port
+2. logviewer.https.keystore.type (example "jks")
+3. logviewer.https.keystore.path (example "/etc/ssl/storm_keystore.jks")
+4. logviewer.https.keystore.password (keystore password)
+5. logviewer.https.key.password (private key password)
+
+optional config
+6. logviewer.https.truststore.path (example "/etc/ssl/storm_truststore.jks")
+7. logviewer.https.truststore.password (truststore password)
+8. logviewer.https.truststore.type (example "jks")
+
+If users want to setup 2-way auth
+9. logviewer.https.want.client.auth (If this set to true server requests for client certificate authentication, but keeps the connection if no authentication provided)
+10. logviewer.https.need.client.auth (If this set to true server requires client to provide authentication)
+
+## Mutual TLS (mTLS) Support
+
+**Available since Storm 2.7.0 (STORM-4070)**
+
+Storm now supports mutual TLS (mTLS) for internal Thrift RPC communication among Nimbus, Supervisors, and workers. Unlike one-way TLS, mTLS requires both parties to present and verify each other's certificates. This ensures full two-way certificate authentication and encryption.
+
+### Overview of TLS Configuration
+
+Storm provides TLS support for securing internal communication between Nimbus, Supervisors, and workers. This is achieved using existing configuration properties for Thrift-based RPC communication and Netty-based messaging.
+
+The TLS setup requires configuring keystore and truststore paths along with enabling TLS flags for each component. Nimbus and Supervisors use Thrift TLS settings, while workers use Netty TLS settings.
+
+The following sections provide example configurations for each component.
+
+### Example TLS Configuration
+
+### 1. Nimbus Settings
+
+```yaml
+# Thrift TLS Listener
+nimbus.thrift.tls.port: 6067
+nimbus.thrift.access.log.enabled: true
+nimbus.thrift.tls.server.only: true
+
+# Server-side certificates & truststore
+nimbus.thrift.tls.server.keystore.path: /etc/ssl/server.keystore.jks
+nimbus.thrift.tls.server.keystore.password: password
+nimbus.thrift.tls.server.truststore.path: /etc/ssl//server.truststore.jks
+nimbus.thrift.tls.server.truststore.password: password
+
+# Client-side certificates & transport plugin
+nimbus.thrift.client.use.tls: true
+nimbus.thrift.tls.client.keystore.path: /etc/ssl/client.keystore.jks
+nimbus.thrift.tls.client.keystore.password: password
+nimbus.thrift.tls.client.truststore.path: /etc/ssl/client.truststore.jks
+nimbus.thrift.tls.client.truststore.password: password
+nimbus.thrift.tls.transport: org.apache.storm.security.auth.tls.TlsTransportPlugin
+```
+### 2. Supervisor Settings
+
+```yaml
+# TLS transport plugin & client enable
+supervisor.thrift.transport: org.apache.storm.security.auth.tls.TlsTransportPlugin
+supervisor.thrift.client.use.tls: true
+
+# Supervisor as Thrift TLS server
+supervisor.thrift.tls.server.keystore.path: /etc/ssl/server.keystore.jks
+supervisor.thrift.tls.server.keystore.password: password
+supervisor.thrift.tls.server.truststore.path: /etc/ssl/server.truststore.jks
+supervisor.thrift.tls.server.truststore.password: password
+
+# Supervisor client settins
+supervisor.thrift.tls.client.keystore.path: /etc/ssl/client.keystore.jks
+supervisor.thrift.tls.client.keystore.password: password
+supervisor.thrift.tls.client.truststore.path: /etc/ssl/client.truststore.jks
+supervisor.thrift.tls.client.truststore.password: password
+```
+
+### 3. Worker Settings
+
+```yaml
+# Storm Netty messaging TLS (worker ↔ worker)
+storm.messaging.netty.tls.enable: true
+storm.messaging.netty.tls.require.open.ssl: true
+
+# Inbound (server-side) credentials
+storm.messaging.netty.tls.keystore.path: /etc/ssl/server.keystore.jks
+storm.messaging.netty.tls.keystore.password: password
+storm.messaging.netty.tls.truststore.path: /etc/ssl/server.truststore.jks
+storm.messaging.netty.tls.truststore.password: password
+
+# Outbound (client-side) credentials
+storm.messaging.netty.tls.client.keystore.path: /etc/ssl/client.keystore.jks
+storm.messaging.netty.tls.client.keystore.password: password
+storm.messaging.netty.tls.client.truststore.path: /etc/ssl/client.truststore.jks
+storm.messaging.netty.tls.client.truststore.password: password
+```
+
+### 4. Setting Descriptions
+
+| Setting | Description |
+|---------------------------------------------------|----------------------------------------------------------------------------------------------|
+| `nimbus.thrift.tls.port` | Port on which Nimbus listens for TLS-encrypted Thrift connections (e.g., 6067) |
+| `nimbus.thrift.tls.server.only` | Nimbus accepts only secure TLS connections |
+| `nimbus.thrift.tls.server.keystore.path` | Path to Nimbus server keystore |
+| `nimbus.thrift.tls.server.keystore.password` | Password for the Nimbus server keystore |
+| `nimbus.thrift.tls.server.truststore.path` | Path to Nimbus server truststore |
+| `nimbus.thrift.tls.server.truststore.password` | Password for the Nimbus truststore |
+| `nimbus.thrift.client.use.tls` | Enable TLS on Nimbus outbound Thrift calls |
+| `nimbus.thrift.tls.client.keystore.path` | Path to Nimbus client keystore (for outbound connections) |
+| `nimbus.thrift.tls.client.keystore.password` | Password for the Nimbus client keystore |
+| `nimbus.thrift.tls.client.truststore.path` | Path to Nimbus client truststore |
+| `nimbus.thrift.tls.client.truststore.password` | Password for the Nimbus client truststore |
+| `nimbus.thrift.tls.transport` | TLS transport plugin class for Nimbus |
+| `storm.principal.tolocal` | Principal-to-local mapping class (for X.509 auth) |
+| `supervisor.thrift.transport` | TLS transport plugin class for Supervisor Thrift |
+| `supervisor.thrift.client.use.tls` | Enable TLS for Supervisor outbound Thrift calls |
+| `supervisor.thrift.tls.server.keystore.path` | Path to Supervisor server keystore |
+| `supervisor.thrift.tls.server.keystore.password` | Password for the Supervisor server keystore |
+| `supervisor.thrift.tls.server.truststore.path` | Path to Supervisor server truststore |
+| `supervisor.thrift.tls.server.truststore.password`| Password for the Supervisor truststore |
+| `supervisor.thrift.tls.client.keystore.path` | Path to Supervisor client keystore |
+| `supervisor.thrift.tls.client.keystore.password` | Password for the Supervisor client keystore |
+| `supervisor.thrift.tls.client.truststore.path` | Path to Supervisor client truststore |
+| `supervisor.thrift.tls.client.truststore.password`| Password for the Supervisor client truststore |
+| `storm.messaging.netty.tls.enable` | Enable TLS for Storm Netty messaging (inter-worker) |
+| `storm.messaging.netty.tls.require.open.ssl` | Require OpenSSL provider for Netty TLS |
+| `storm.messaging.netty.tls.keystore.path` | Path to Netty server keystore |
+| `storm.messaging.netty.tls.keystore.password` | Password for the Netty server keystore |
+| `storm.messaging.netty.tls.truststore.path` | Path to Netty server truststore |
+| `storm.messaging.netty.tls.truststore.password` | Password for the Netty server truststore |
+| `storm.messaging.netty.tls.client.keystore.path` | Path to Netty client keystore |
+| `storm.messaging.netty.tls.client.keystore.password`| Password for the Netty client keystore |
+| `storm.messaging.netty.tls.client.truststore.path`| Path to Netty client truststore |
+| `storm.messaging.netty.tls.client.truststore.password`| Password for the Netty client truststore |
+
+
+## UI Security Hardening
+
+### Reverse Proxy Recommendation
+
+For production deployments, the Storm UI, Logviewer, and DRPC HTTP endpoints should be placed behind a reverse proxy such as Apache httpd, nginx, or a similar gateway. While Storm provides built-in SSL/TLS configuration, a reverse proxy offers significant additional security benefits:
+
+- Proper TLS termination with up-to-date cipher suites and certificate management
+- CSRF token validation or same-origin enforcement on state-mutating requests
+- Security response headers (see below)
+- Rate limiting and request filtering
+- Centralized access logging and monitoring
+
+This is especially important because the Storm UI exposes state-mutating operations (topology kill, activate, deactivate, rebalance) via its REST API.
+
+### CSRF Protection
+
+The Storm UI REST API does not currently include built-in Cross-Site Request Forgery (CSRF) protection. State-mutating endpoints can potentially be triggered by a malicious website if a user has an active authenticated session in the same browser. This is particularly relevant when using browser-based authentication mechanisms such as Kerberos (SPNEGO), as the browser automatically attaches credentials to cross-origin requests.
+
+To mitigate this risk, configure your reverse proxy to enforce CSRF token validation or same-origin checks on POST requests.
+
+### Security Response Headers
+
+The Storm UI does not set security-related HTTP response headers by default. Operators deploying a secured cluster should configure the following headers via their reverse proxy or a custom servlet filter configured through `ui.filter`:
+
+- `X-Frame-Options: DENY` or `SAMEORIGIN` to prevent clickjacking attacks
+- `X-Content-Type-Options: nosniff` to prevent MIME type sniffing
+- `Content-Security-Policy` to restrict resource loading
+- `Strict-Transport-Security` when using HTTPS, to enforce secure connections
+
+## Authentication (Kerberos)
+
+Storm offers pluggable authentication support through thrift and SASL. This
+example only goes off of Kerberos as it is a common setup for most big data
+projects.
+
+Setting up a KDC and configuring kerberos on each node is beyond the scope of
+this document and it is assumed that you have done that already.
+
+### Create Headless Principals and keytabs
+
+Each Zookeeper Server, Nimbus, and DRPC server will need a service principal, which, by convention, includes the FQDN of the host it will run on. Be aware that the zookeeper user *MUST* be zookeeper.
+The supervisors and UI also need a principal to run as, but because they are outgoing connections they do not need to be service principals.
+The following is an example of how to setup kerberos principals, but the
+details may vary depending on your KDC and OS.
+
+
+```bash
+# Zookeeper (Will need one of these for each box in the Zk ensemble)
+sudo kadmin.local -q 'addprinc zookeeper/zk1.example.com@STORM.EXAMPLE.COM'
+sudo kadmin.local -q "ktadd -k /tmp/zk.keytab zookeeper/zk1.example.com@STORM.EXAMPLE.COM"
+# Nimbus and DRPC
+sudo kadmin.local -q 'addprinc storm/storm.example.com@STORM.EXAMPLE.COM'
+sudo kadmin.local -q "ktadd -k /tmp/storm.keytab storm/storm.example.com@STORM.EXAMPLE.COM"
+# All UI logviewer and Supervisors
+sudo kadmin.local -q 'addprinc storm@STORM.EXAMPLE.COM'
+sudo kadmin.local -q "ktadd -k /tmp/storm.keytab storm@STORM.EXAMPLE.COM"
+```
+
+be sure to distribute the keytab(s) to the appropriate boxes and set the FS permissions so that only the headless user running ZK, or storm has access to them.
+
+#### Storm Kerberos Configuration
+
+Both storm and Zookeeper use jaas configuration files to log the user in.
+Each jaas file may have multiple sections for different interfaces being used.
+
+To enable Kerberos authentication in storm you need to set the following storm.yaml configs
+```yaml
+storm.thrift.transport: "org.apache.storm.security.auth.kerberos.KerberosSaslTransportPlugin"
+java.security.auth.login.config: "/path/to/jaas.conf"
+```
+
+Nimbus and the supervisor processes will also connect to ZooKeeper(ZK) and we want to configure them to use Kerberos for authentication with ZK. To do this append
+```
+-Djava.security.auth.login.config=/path/to/jaas.conf
+```
+
+to the childopts of nimbus, ui, and supervisor. Here is an example given the default childopts settings at the time of writing:
+
+```yaml
+nimbus.childopts: "-Xmx1024m -Djava.security.auth.login.config=/path/to/jaas.conf"
+ui.childopts: "-Xmx768m -Djava.security.auth.login.config=/path/to/jaas.conf"
+supervisor.childopts: "-Xmx256m -Djava.security.auth.login.config=/path/to/jaas.conf"
+```
+
+The jaas.conf file should look something like the following for the storm nodes.
+The StormServer section is used by nimbus and the DRPC Nodes. It does not need to be included on supervisor nodes.
+The StormClient section is used by all storm clients that want to talk to nimbus, including the ui, logviewer, and supervisor. We will use this section on the gateways as well but the structure of that will be a bit different.
+The Client section is used by processes wanting to talk to zookeeper and really only needs to be included with nimbus and the supervisors.
+The Server section is used by the zookeeper servers.
+Having unused sections in the jaas is not a problem.
+
+```
+StormServer {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="$keytab"
+ storeKey=true
+ useTicketCache=false
+ principal="$principal";
+};
+StormClient {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="$keytab"
+ storeKey=true
+ useTicketCache=false
+ serviceName="$nimbus_user"
+ principal="$principal";
+};
+Client {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="$keytab"
+ storeKey=true
+ useTicketCache=false
+ serviceName="zookeeper"
+ principal="$principal";
+};
+Server {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="$keytab"
+ storeKey=true
+ useTicketCache=false
+ principal="$principal";
+};
+```
+
+The following is an example based off of the keytabs generated
+```
+StormServer {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="/keytabs/storm.keytab"
+ storeKey=true
+ useTicketCache=false
+ principal="storm/storm.example.com@STORM.EXAMPLE.COM";
+};
+StormClient {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="/keytabs/storm.keytab"
+ storeKey=true
+ useTicketCache=false
+ serviceName="storm"
+ principal="storm@STORM.EXAMPLE.COM";
+};
+Client {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="/keytabs/storm.keytab"
+ storeKey=true
+ useTicketCache=false
+ serviceName="zookeeper"
+ principal="storm@STORM.EXAMPLE.COM";
+};
+Server {
+ com.sun.security.auth.module.Krb5LoginModule required
+ useKeyTab=true
+ keyTab="/keytabs/zk.keytab"
+ storeKey=true
+ useTicketCache=false
+ serviceName="zookeeper"
+ principal="zookeeper/zk1.example.com@STORM.EXAMPLE.COM";
+};
+```
+
+Nimbus also will translate the principal into a local user name, so that other services can use this name. To configure this for Kerberos authentication set
+
+```
+storm.principal.tolocal: "org.apache.storm.security.auth.KerberosPrincipalToLocal"
+```
+
+This only needs to be done on nimbus, but it will not hurt on any node.
+We also need to inform the topology who the supervisor daemon and the nimbus daemon are running as from a ZooKeeper perspective.
+
+```
+storm.zookeeper.superACL: "sasl:${nimbus-user}"
+```
+
+Here *nimbus-user* is the Kerberos user that nimbus uses to authenticate with ZooKeeper. If ZooKeeeper is stripping host and realm then this needs to have host and realm stripped too.
+
+#### ZooKeeper Ensemble
+
+Complete details of how to setup a secure ZK are beyond the scope of this document. But in general you want to enable SASL authentication on each server, and optionally strip off host and realm
+
+```
+authProvider.1 = org.apache.zookeeper.server.auth.SASLAuthenticationProvider
+kerberos.removeHostFromPrincipal = true
+kerberos.removeRealmFromPrincipal = true
+```
+
+And you want to include the jaas.conf on the command line when launching the server so it can use it can find the keytab.
+```
+-Djava.security.auth.login.config=/jaas/zk_jaas.conf
+```
+
+#### Gateways
+
+Ideally the end user will only need to run kinit before interacting with storm. To make this happen seamlessly we need the default jaas.conf on the gateways to be something like
+
+```
+StormClient {
+ com.sun.security.auth.module.Krb5LoginModule required
+ doNotPrompt=false
+ useTicketCache=true
+ serviceName="$nimbus_user";
+};
+```
+
+The end user can override this if they have a headless user that has a keytab.
+
+### Authorization Setup
+
+*Authentication* does the job of verifying who the user is, but we also need *authorization* to do the job of enforcing what each user can do.
+
+The preferred authorization plug-in for nimbus is The *SimpleACLAuthorizer*. To use the *SimpleACLAuthorizer*, set the following:
+
+```yaml
+nimbus.authorizer: "org.apache.storm.security.auth.authorizer.SimpleACLAuthorizer"
+```
+
+DRPC has a separate authorizer configuration for it. Do not use SimpleACLAuthorizer for DRPC.
+
+The *SimpleACLAuthorizer* plug-in needs to know who the supervisor users are, and it needs to know about all of the administrator users, including the user running the ui daemon.
+
+These are set through *nimbus.supervisor.users* and *nimbus.admins* respectively. Each can either be a full Kerberos principal name, or the name of the user with host and realm stripped off.
+
+The Log servers have their own authorization configurations. These are set through *logs.users* and *logs.groups*. These should be set to the admin users or groups for all of the nodes in the cluster.
+
+When a topology is submitted, the submitting user can specify users in this list as well. The users and groups specified-in addition to the users in the cluster-wide setting-will be granted access to the submitted topology's worker logs in the logviewers.
+
+### BlobStore ACL Validation
+
+Storm uses a BlobStore to distribute topology code and configuration across the cluster. Access control lists on blobs are not enforced by default (`storm.blobstore.acl.validation.enabled: false`). In a secured multi-tenant cluster this means any authenticated user can read or modify any topology's blobs, regardless of ACL settings. To enforce BlobStore ACLs, set the following:
+
+```yaml
+storm.blobstore.acl.validation.enabled: true
+```
+
+### Supervisors headless User and group Setup
+
+To ensure isolation of users in multi-tenancy, there is need to run supervisors and headless user and group unique to execution on the supervisor nodes. To enable this follow below steps.
+1. Add headlessuser to all supervisor hosts.
+2. Create unique group and make it the primary group for the headless user on the supervisor nodes.
+3. The set following properties on storm for these supervisor nodes.
+
+### Multi-tenant Scheduler
+
+To support multi-tenancy better we have written a new scheduler. To enable this scheduler set.
+```yaml
+storm.scheduler: "org.apache.storm.scheduler.multitenant.MultitenantScheduler"
+```
+Be aware that many of the features of this scheduler rely on storm authentication. Without them the scheduler will not know what the user is and will not isolate topologies properly.
+
+The goal of the multi-tenant scheduler is to provide a way to isolate topologies from one another, but to also limit the resources that an individual user can have in the cluster.
+
+The scheduler currently has one config that can be set either through =storm.yaml= or through a separate config file called =multitenant-scheduler.yaml= that should be placed in the same directory as =storm.yaml=. It is preferable to use =multitenant-scheduler.yaml= because it can be updated without needing to restart nimbus.
+
+There is currently only one config in =multitenant-scheduler.yaml=, =multitenant.scheduler.user.pools= is a map from the user name, to the maximum number of nodes that user is guaranteed to be able to use for their topologies.
+
+For example:
+
+```yaml
+multitenant.scheduler.user.pools:
+ "evans": 10
+ "derek": 10
+```
+
+### Run worker processes as user who submitted the topology
+By default storm runs workers as the user that is running the supervisor. This is not ideal for security. To make storm run the topologies as the user that launched them set.
+
+```yaml
+supervisor.run.worker.as.user: true
+```
+
+There are several files that go along with this that are needed to be configured properly to make storm secure.
+
+The worker-launcher executable is a special program that allows the supervisor to launch workers as different users. For this to work it needs to be owned by root, but with the group set to be a group that only the supervisor headless user is a part of.
+It also needs to have 6550 permissions.
+There is also a worker-launcher.cfg file, usually located under `/etc/storm` that should look something like the following
+
+```
+storm.worker-launcher.group=$(worker_launcher_group)
+min.user.id=$(min_user_id)
+```
+where worker_launcher_group is the same group the supervisor is a part of, and min.user.id is set to the first real user id on the system.
+This config file also needs to be owned by root and not have world or group write permissions.
+
+### Impersonating a user
+A storm client may submit requests on behalf of another user. For example, if a `userX` submits an oozie workflow and as part of workflow execution if user `oozie` wants to submit a topology on behalf of `userX`
+it can do so by leveraging the impersonation feature.In order to submit topology as some other user , you can use `StormSubmitter.submitTopologyAs` API. Alternatively you can use `NimbusClient.getConfiguredClientAs`
+to get a nimbus client as some other user and perform any nimbus action(i.e. kill/rebalance/activate/deactivate) using this client.
+
+Impersonation authorization is disabled by default which means any user can perform impersonation. To ensure only authorized users can perform impersonation you should start nimbus with `nimbus.impersonation.authorizer` set to `org.apache.storm.security.auth.authorizer.ImpersonationAuthorizer`.
+The `ImpersonationAuthorizer` uses `nimbus.impersonation.acl` as the acl to authorize users. Following is a sample nimbus config for supporting impersonation:
+
+**Important:** If you have enabled authentication (e.g. Kerberos) and authorization (e.g. `SimpleACLAuthorizer`) but have *not* configured `nimbus.impersonation.authorizer`, any authenticated user can impersonate any other user by setting the `doAsUser` HTTP header or query parameter. The `DefaultHttpCredentialsPlugin` always processes `doAsUser` requests, and without an impersonation authorizer these requests are silently allowed. When deploying a secured cluster you should always confi [...]
+
+```yaml
+nimbus.impersonation.authorizer: org.apache.storm.security.auth.authorizer.ImpersonationAuthorizer
+nimbus.impersonation.acl:
+ impersonating_user1:
+ hosts:
+ [comma separated list of hosts from which impersonating_user1 is allowed to impersonate other users]
+ groups:
+ [comma separated list of groups whose users impersonating_user1 is allowed to impersonate]
+ impersonating_user2:
+ hosts:
+ [comma separated list of hosts from which impersonating_user2 is allowed to impersonate other users]
+ groups:
+ [comma separated list of groups whose users impersonating_user2 is allowed to impersonate]
+```
+
+To support the oozie use case following config can be supplied:
+```yaml
+nimbus.impersonation.acl:
+ oozie:
+ hosts:
+ [oozie-host1, oozie-host2, 127.0.0.1]
+ groups:
+ [some-group-that-userX-is-part-of]
+```
+
+### Automatic Credentials Push and Renewal
+Individual topologies have the ability to push credentials (tickets and tokens) to workers so that they can access secure services. Exposing this to all of the users can be a pain for them.
+To hide this from them in the common case plugins can be used to populate the credentials, unpack them on the other side into a java Subject, and also allow Nimbus to renew the credentials if needed. These are controlled by the following configs.
+
+`topology.auto-credentials` is a list of java plugins, all of which must implement the `IAutoCredentials` interface, that populate the credentials on gateway
+and unpack them on the worker side. On a kerberos secure cluster they should be set by default to point to `org.apache.storm.security.auth.kerberos.AutoTGT`
+
+`nimbus.credential.renewers.classes` should also be set to `org.apache.storm.security.auth.kerberos.AutoTGT` so that nimbus can periodically renew the TGT on behalf of the user.
+
+All autocredential classes that desire to implement the IMetricsRegistrant interface can register metrics automatically for each topology. The AutoTGT class currently implements this interface and adds a metric named TGT-TimeToExpiryMsecs showing the remaining time until the TGT needs to be renewed.
+
+`nimbus.credential.renewers.freq.secs` controls how often the renewer will poll to see if anything needs to be renewed, but the default should be fine.
+
+In addition Nimbus itself can be used to get credentials on behalf of the user submitting topologies. This can be configured using `nimbus.autocredential.plugins.classes` which is a list
+of fully qualified class names, all of which must implement `INimbusCredentialPlugin`. Nimbus will invoke the populateCredentials method of all the configured implementation as part of topology
+submission. You should use this config with `topology.auto-credentials` and `nimbus.credential.renewers.classes` so the credentials can be populated on worker side and nimbus can automatically renew
+them. Currently there are 2 examples of using this config, AutoHDFS and AutoHBase which auto populates hdfs and hbase delegation tokens for topology submitter so they don't have to distribute keytabs
+on all possible worker hosts.
+
+### Limits
+By default storm allows any sized topology to be submitted. But ZK and others have limitations on how big a topology can actually be. The following configs allow you to limit the maximum size a topology can be.
+
+| YAML Setting | Description |
+|------------|----------------------|
+| nimbus.slots.perTopology | The maximum number of slots/workers a topology can use. |
+| nimbus.executors.perTopology | The maximum number of executors/threads a topology can use. |
+
+### Serialization Security
+
+Storm uses Kryo for serializing tuple data between spouts and bolts. By default, Kryo requires all classes to be explicitly registered (`topology.fall.back.on.java.serialization: false`). When this setting is changed to `true`, any unregistered class will be serialized using Java's native `ObjectInputStream`/`ObjectOutputStream`, which is known to be vulnerable to deserialization attacks if untrusted data reaches the serialization path.
+
+**Do not set `topology.fall.back.on.java.serialization` to `true` in production.** While topology submitters already run arbitrary code via their spouts and bolts, enabling the Java serialization fallback broadens the attack surface and may allow malicious data from external sources (e.g. message queues) to trigger unintended code execution during deserialization.
+
+For tuple encryption, use TLS-based transport encryption (`storm.messaging.netty.tls.enable`) instead of the deprecated `BlowfishTupleSerializer`, which uses a 64-bit block cipher vulnerable to birthday attacks.
+
+### Log Cleanup
+The Logviewer daemon now is also responsible for cleaning up old log files for dead topologies.
+
+| YAML Setting | Description |
+|--------------|-------------------------------------|
+| logviewer.cleanup.age.mins | How old (by last modification time) must a worker's log be before that log is considered for clean-up. (Living workers' logs are never cleaned up by the logviewer: Their logs are rolled via logback.) |
+| logviewer.cleanup.interval.secs | Interval of time in seconds that the logviewer cleans up worker logs. |
+
+
+### Allowing specific users or groups to access storm
+
+ With SimpleACLAuthorizer any user with valid kerberos ticket can deploy a topology or do further operations such as activate, deactivate , access cluster information.
+ One can restrict this access by specifying nimbus.users or nimbus.groups. If nimbus.users configured only the users in the list can deploy a topology or access cluster.
+ Similarly nimbus.groups restrict storm cluster access to users who belong to those groups.
+
+ To configure specify the following config in storm.yaml
+
+```yaml
+nimbus.users:
+ - "testuser"
+```
+
+or
+
+```yaml
+nimbus.groups:
+ - "storm"
+```
+
+
+### DRPC
+
+Storm provides the Access Control List for the DRPC Authorizer.Users can see [org.apache.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/security/auth/authorizer/DRPCSimpleACLAuthorizer.html) for more details.
+
+There are several DRPC ACL related configurations.
+
+| YAML Setting | Description |
+|------------|----------------------|
+| drpc.authorizer.acl | A class that will perform authorization for DRPC operations. Set this to org.apache.storm.security.auth.authorizer.DRPCSimpleACLAuthorizer when using security.|
+| drpc.authorizer.acl.filename | This is the name of a file that the ACLs will be loaded from. It is separate from storm.yaml to allow the file to be updated without bringing down a DRPC server. Defaults to drpc-auth-acl.yaml |
+| drpc.authorizer.acl.strict| It is useful to set this to false for staging where users may want to experiment, but true for production where you want users to be secure. Defaults to false. |
+
+The file pointed to by drpc.authorizer.acl.filename will have only one config in it drpc.authorizer.acl this should be of the form
+
+```yaml
+drpc.authorizer.acl:
+ "functionName1":
+ "client.users":
+ - "alice"
+ - "bob"
+ "invocation.user": "bob"
+```
+
+In this the users bob and alice as client.users are allowed to run DRPC requests against functionName1, but only bob as the invocation.user is allowed to run the topology that actually processes those requests.
+
+
+## Cluster Zookeeper Authentication
+
+Users can implement cluster Zookeeper authentication by setting several configurations are shown below.
+
+| YAML Setting | Description |
+|------------|----------------------|
+| storm.zookeeper.auth.scheme | The cluster Zookeeper authentication scheme to use, e.g. "digest". Defaults to no authentication. |
+| storm.zookeeper.auth.payload | A string representing the payload for cluster Zookeeper authentication. It should only be set in the storm-cluster-auth.yaml. Users can see storm-cluster-auth.yaml.example for more details. |
+
+
+Also, there are several configurations for topology Zookeeper authentication:
+
+| YAML Setting | Description |
+|------------|----------------------|
+| storm.zookeeper.topology.auth.scheme | The topology Zookeeper authentication scheme to use, e.g. "digest". It is the internal config and user shouldn't set it. |
+| storm.zookeeper.topology.auth.payload | A string representing the payload for topology Zookeeper authentication. |
+
+Note: If storm.zookeeper.topology.auth.payload isn't set, Storm will generate a ZooKeeper secret payload for MD5-digest with generateZookeeperDigestSecretPayload() method.
+
+### ZooKeeper SSL Hostname Verification
+
+When enabling ZooKeeper SSL via `storm.zookeeper.ssl.enable`, be aware that hostname verification is disabled by default (`storm.zookeeper.ssl.hostnameVerification: false`). Without hostname verification, a man-in-the-middle attacker with any valid certificate signed by a trusted CA could intercept communication between Storm and ZooKeeper. For production deployments you should enable this:
+
+```yaml
+storm.zookeeper.ssl.enable: true
+storm.zookeeper.ssl.hostnameVerification: true
+```
+
+
diff --git a/releases/2.8.8/STORM-UI-REST-API.md b/releases/2.8.8/STORM-UI-REST-API.md
new file mode 100644
index 0000000000..bf99960f0d
--- /dev/null
+++ b/releases/2.8.8/STORM-UI-REST-API.md
@@ -0,0 +1,1616 @@
+---
+title: Storm UI REST API
+layout: documentation
+documentation: true
+---
+
+
+The Storm UI daemon provides a REST API that allows you to interact with a Storm cluster, which includes retrieving
+metrics data and configuration information as well as management operations such as starting or stopping topologies.
+
+
+# Data format
+
+The REST API returns JSON responses and supports JSONP.
+Clients can pass a callback query parameter to wrap JSON in the callback function.
+
+
+# Using the UI REST API
+
+_Note: It is recommended to ignore undocumented elements in the JSON response because future versions of Storm may not_
+_support those elements anymore._
+
+
+## REST API Base URL
+
+The REST API is part of the UI daemon of Storm (started by `storm ui`) and thus runs on the same host and port as the
+Storm UI (the UI daemon is often run on the same host as the Nimbus daemon). The port is configured by `ui.port`,
+which is set to `8080` by default (see [defaults.yaml](conf/defaults.yaml)).
+
+The API base URL would thus be:
+
+ http://<ui-host>:<ui-port>/api/v1/...
+
+You can use a tool such as `curl` to talk to the REST API:
+
+ # Request the cluster configuration.
+ # Note: We assume ui.port is configured to the default value of 8080.
+ $ curl http://<ui-host>:8080/api/v1/cluster/configuration
+
+##Impersonating a user in secure environment
+In a secure environment an authenticated user can impersonate another user. To impersonate a user the caller must pass
+`doAsUser` param or header with value set to the user that the request needs to be performed as. Please see SECURITY.MD
+to learn more about how to setup impersonation ACLs and authorization. The rest API uses the same configs and acls that
+are used by nimbus.
+
+Examples:
+
+```no-highlight
+ 1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1425844354\?doAsUser=testUSer1
+ 2. curl 'http://localhost:8080/api/v1/topology/wordcount-1-1425844354/activate' -X POST -H 'doAsUser:testUSer1'
+```
+
+## GET Operations
+
+### /api/v1/cluster/configuration (GET)
+
+Returns the cluster configuration.
+
+Sample response (does not include all the data fields):
+
+```json
+ {
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": false,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ }
+```
+
+### /api/v1/cluster/summary (GET)
+
+Returns cluster summary information such as nimbus uptime or number of supervisors.
+
+Response fields:
+
+|Field |Value|Description
+|--- |--- |---
+|stormVersion|String| Storm version|
+|supervisors|Integer| Number of supervisors running|
+|topologies| Integer| Number of topologies running|
+|slotsTotal| Integer|Total number of available worker slots|
+|slotsUsed| Integer| Number of worker slots used|
+|slotsFree| Integer |Number of worker slots available|
+|executorsTotal| Integer |Total number of executors|
+|tasksTotal| Integer |Total tasks|
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+|totalMem| Double | The total amount of memory in the cluster in MB|
+|totalCpu| Double | The total amount of CPU in the cluster|
+|availMem| Double | The amount of available memory in the cluster in MB|
+|availCpu| Double | The amount of available cpu in the cluster|
+|memAssignedPercentUtil| Double | The percent utilization of assigned memory resources in cluster|
+|cpuAssignedPercentUtil| Double | The percent utilization of assigned CPU resources in cluster|
+
+Sample response:
+
+```json
+ {
+ "stormVersion": "0.9.2-incubating-SNAPSHOT",
+ "supervisors": 1,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "slotsFree": 1,
+ "executorsTotal": 28,
+ "tasksTotal": 28,
+ "schedulerDisplayResource": true,
+ "totalMem": 4096.0,
+ "totalCpu": 400.0,
+ "availMem": 1024.0,
+ "availCPU": 250.0,
+ "memAssignedPercentUtil": 75.0,
+ "cpuAssignedPercentUtil": 37.5
+ }
+```
+
+### /api/v1/supervisor/summary (GET)
+
+Returns summary information for all supervisors.
+
+Response fields:
+
+|Field |Value|Description|
+|--- |--- |---
+|id| String | Supervisor's id|
+|host| String| Supervisor's host name|
+|uptime| String| Shows how long the supervisor is running|
+|uptimeSeconds| Integer| Shows how long the supervisor is running in seconds|
+|slotsTotal| Integer| Total number of available worker slots for this supervisor|
+|slotsUsed| Integer| Number of worker slots used on this supervisor|
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+|totalMem| Double| Total memory capacity on this supervisor|
+|totalCpu| Double| Total CPU capacity on this supervisor|
+|usedMem| Double| Used memory capacity on this supervisor|
+|usedCpu| Double| Used CPU capacity on this supervisor|
+
+Sample response:
+
+```json
+{
+ "supervisors": [
+ {
+ "id": "0b879808-2a26-442b-8f7d-23101e0c3696",
+ "host": "10.11.1.7",
+ "uptime": "5m 58s",
+ "uptimeSeconds": 358,
+ "slotsTotal": 4,
+ "slotsUsed": 3,
+ "totalMem": 3000,
+ "totalCpu": 400,
+ "usedMem": 1280,
+ "usedCPU": 160
+ }
+ ],
+ "schedulerDisplayResource": true
+}
+```
+
+### /api/v1/nimbus/summary (GET)
+
+Returns summary information for all nimbus hosts.
+
+Response fields:
+
+|Field |Value|Description|
+|--- |--- |---
+|host| String | Nimbus' host name|
+|port| int| Nimbus' port number|
+|status| String| Possible values are Leader, Not a Leader, Dead|
+|nimbusUpTime| String| Shows since how long the nimbus has been running|
+|nimbusUpTimeSeconds| String| Shows since how long the nimbus has been running in seconds|
+|nimbusLogLink| String| Logviewer url to view the nimbus.log|
+|version| String| Version of storm this nimbus host is running|
+
+Sample response:
+
+```json
+{
+ "nimbuses":[
+ {
+ "host":"192.168.202.1",
+ "port":6627,
+ "nimbusLogLink":"http:\/\/192.168.202.1:8000\/log?file=nimbus.log",
+ "status":"Leader",
+ "version":"0.10.0-SNAPSHOT",
+ "nimbusUpTime":"3m 33s",
+ "nimbusUpTimeSeconds":"213"
+ }
+ ]
+}
+```
+
+### /api/v1/history/summary (GET)
+
+Returns a list of all running topologies' IDs submitted by the current user.
+
+Response fields:
+
+|Field |Value | Description|
+|--- |--- |---
+|topo-history| List| List of Topologies' IDs|
+
+Sample response:
+
+```json
+{
+ "topo-history":[
+ "wc6-1-1446571009",
+ "wc8-2-1446587178"
+ ]
+}
+```
+
+### /api/v1/supervisor (GET)
+
+Returns summary for a supervisor by id, or all supervisors running on a host.
+
+Examples:
+
+```no-highlight
+ 1. By host: http://ui-daemon-host-name:8080/api/v1/supervisor?host=supervisor-daemon-host-name
+ 2. By id: http://ui-daemon-host-name:8080/api/v1/supervisor?id=f5449110-1daa-43e2-89e3-69917b16dec9-192.168.1.1
+```
+
+Request parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String. Supervisor id | If specified, respond with the supervisor and worker stats with id. Note that when id is specified, the host argument is ignored. |
+|host |String. Host name| If specified, respond with all supervisors and worker stats in the host (normally just one)|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+Response fields:
+
+|Field |Value|Description|
+|--- |--- |---
+|supervisors| Array| Array of supervisor summaries|
+|workers| Array| Array of worker summaries |
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+
+Each supervisor is defined by:
+
+|Field |Value|Description|
+|--- |--- |---
+|id| String | Supervisor's id|
+|host| String| Supervisor's host name|
+|uptime| String| Shows how long the supervisor is running|
+|uptimeSeconds| Integer| Shows how long the supervisor is running in seconds|
+|slotsTotal| Integer| Total number of worker slots for this supervisor|
+|slotsUsed| Integer| Number of worker slots used on this supervisor|
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+|totalMem| Double| Total memory capacity on this supervisor|
+|totalCpu| Double| Total CPU capacity on this supervisor|
+|usedMem| Double| Used memory capacity on this supervisor|
+|usedCpu| Double| Used CPU capacity on this supervisor|
+
+Each worker is defined by:
+
+|Field |Value |Description|
+|-------|-------|-----------|
+|supervisorId | String| Supervisor's id|
+|host | String | Worker's host name|
+|port | Integer | Worker's port|
+|topologyId | String | Topology Id|
+|topologyName | String | Topology Name|
+|executorsTotal | Integer | Number of executors used by the topology in this worker|
+|assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB)|
+|assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB)|
+|assignedCpu | Number | Assigned CPU by Scheduler (%)|
+|componentNumTasks | Dictionary | Components -> # of executing tasks|
+|uptime| String| Shows how long the worker is running|
+|uptimeSeconds| Integer| Shows how long the worker is running in seconds|
+|workerLogLink | String | Link to worker log viewer page|
+
+Sample response:
+
+```json
+{
+ "supervisors": [{
+ "totalMem": 4096.0,
+ "host":"192.168.10.237",
+ "id":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e",
+ "uptime":"7m 8s",
+ "totalCpu":400.0,
+ "usedCpu":495.0,
+ "usedMem":3432.0,
+ "slotsUsed":2,
+ "version":"0.10.1",
+ "slotsTotal":4,
+ "uptimeSeconds":428
+ }],
+ "schedulerDisplayResource":true,
+ "workers":[{
+ "topologyName":"ras",
+ "topologyId":"ras-4-1460229987",
+ "host":"192.168.10.237",
+ "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e",
+ "assignedMemOnHeap":704.0,
+ "uptime":"2m 47s",
+ "uptimeSeconds":167,
+ "port":6707,
+ "workerLogLink":"http:\/\/host:8000\/log?file=ras-4-1460229987%2F6707%2Fworker.log",
+ "componentNumTasks": {
+ "word":5
+ },
+ "executorsTotal":8,
+ "assignedCpu":130.0,
+ "assignedMemOffHeap":80.0
+ },
+ {
+ "topologyName":"ras",
+ "topologyId":"ras-4-1460229987",
+ "host":"192.168.10.237",
+ "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e",
+ "assignedMemOnHeap":904.0,
+ "uptime":"2m 53s",
+ "port":6706,
+ "workerLogLink":"http:\/\/host:8000\/log?file=ras-4-1460229987%2F6706%2Fworker.log",
+ "componentNumTasks":{
+ "exclaim2":2,
+ "exclaim1":3,
+ "word":5
+ },
+ "executorsTotal":10,
+ "uptimeSeconds":173,
+ "assignedCpu":165.0,
+ "assignedMemOffHeap":80.0
+ }]
+}
+```
+
+### /api/v1/owner-resources (GET)
+
+Returns summary information aggregated at the topology owner level. Pass an optional id for a specific owner (e.g. /api/v1/owner-resources/theowner)
+
+Response fields:
+
+|Field |Value|Description|
+|--- |--- |---
+|owner |String |Topology owner
+|totalTopologies|Integer|Total number of topologies owner is running
+|totalExecutors |Integer|Total number of executors used by owner
+|totalWorkers |Integer|Total number of workers used by owner
+|totalTasks|Integer|Total number of tasks used by owner
+|totalMemoryUsage|Double|Total Memory Assigned on behalf of owner in MB
+|totalCpuUsage|Double|Total CPU Resource Assigned on behalf of User. Every 100 means 1 core
+|memoryGuarantee|Double|The amount of memory resource (in MB) guaranteed to owner
+|cpuGuarantee|Double|The amount of CPU resource (every 100 means 1 core) guaranteed to owner
+|isolatedNodes|Integer|The amount of nodes that are guaranteed isolated to owner
+|memoryGuaranteeRemaining|Double|The amount of guaranteed memory resources (in MB) remaining
+|cpuGuaranteeRemaining|Double|The amount of guaranteed CPU resource (every 100 means 1 core) remaining
+|totalReqOnHeapMem|Double| Total On-Heap Memory Requested by User in MB
+|totalReqOffHeapMem|Double|Total Off-Heap Memory Requested by User in MB
+|totalReqMem|Double|Total Memory Requested by User in MB
+|totalReqCpu|Double|Total CPU Resource Requested by User. Every 100 means 1 core
+|totalAssignedOnHeapMem|Double|Total On-Heap Memory Assigned on behalf of owner in MB
+|totalAssignedOffHeapMem|Double|Total Off-Heap Memory Assigned on behalf of owner in MB
+
+Sample response:
+
+```json
+{
+ "owners": [
+ {
+ "totalReqOnHeapMem": 896,
+ "owner": "ownerA",
+ "totalExecutors": 7,
+ "cpuGuaranteeRemaining": 30,
+ "totalReqMem": 896,
+ "cpuGuarantee": 100,
+ "isolatedNodes": "N/A",
+ "memoryGuarantee": 4000,
+ "memoryGuaranteeRemaining": 3104,
+ "totalTasks": 7,
+ "totalMemoryUsage": 896,
+ "totalReqOffHeapMem": 0,
+ "totalReqCpu": 70,
+ "totalWorkers": 2,
+ "totalCpuUsage": 70,
+ "totalAssignedOffHeapMem": 0,
+ "totalAssignedOnHeapMem": 896,
+ "totalTopologies": 1
+ }
+ ],
+ "schedulerDisplayResource": true
+}
+```
+
+### /api/v1/topology/summary (GET)
+
+Returns summary information for all topologies.
+
+Response fields:
+
+|Field |Value | Description|
+|--- |--- |---
+|id| String| Topology Id|
+|name| String| Topology Name|
+|status| String| Topology Status|
+|uptime| String| Shows how long the topology is running|
+|uptimeSeconds| Integer| Shows how long the topology is running in seconds|
+|tasksTotal| Integer |Total number of tasks for this topology|
+|workersTotal| Integer |Number of workers used for this topology|
+|executorsTotal| Integer |Number of executors used for this topology|
+|replicationCount| Integer |Number of nimbus hosts on which this topology code is replicated|
+|requestedMemOnHeap| Double|Requested On-Heap Memory by User (MB)
+|requestedMemOffHeap| Double|Requested Off-Heap Memory by User (MB)|
+|requestedTotalMem| Double|Requested Total Memory by User (MB)|
+|requestedCpu| Double|Requested CPU by User (%)|
+|assignedMemOnHeap| Double|Assigned On-Heap Memory by Scheduler (MB)|
+|assignedMemOffHeap| Double|Assigned Off-Heap Memory by Scheduler (MB)|
+|assignedTotalMem| Double|Assigned Total Memory by Scheduler (MB)|
+|assignedCpu| Double|Assigned CPU by Scheduler (%)|
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+
+Sample response:
+
+```json
+{
+ "topologies": [
+ {
+ "id": "WordCount3-1-1402960825",
+ "name": "WordCount3",
+ "status": "ACTIVE",
+ "uptime": "6m 5s",
+ "uptimeSeconds": 365,
+ "tasksTotal": 28,
+ "workersTotal": 3,
+ "executorsTotal": 28,
+ "replicationCount": 1,
+ "requestedMemOnHeap": 640,
+ "requestedMemOffHeap": 128,
+ "requestedTotalMem": 768,
+ "requestedCpu": 80,
+ "assignedMemOnHeap": 640,
+ "assignedMemOffHeap": 128,
+ "assignedTotalMem": 768,
+ "assignedCpu": 80
+ }
+ ],
+ "schedulerDisplayResource": true
+}
+```
+
+### /api/v1/topology-workers/\<id\> (GET)
+
+Returns the worker' information (host and port) for a topology whose id is substituted for \<id\>.
+The topology id is obtained by the above /topology/summary call.
+
+Response fields:
+
+|Field |Value | Description|
+|--- |--- |---
+|hostPortList| List| Workers' information for a topology|
+|name| Integer| Logviewer Port|
+
+Sample response:
+
+```json
+{
+ "hostPortList":[
+ {
+ "host":"192.168.202.2",
+ "port":6701
+ },
+ {
+ "host":"192.168.202.2",
+ "port":6702
+ },
+ {
+ "host":"192.168.202.3",
+ "port":6700
+ }
+ ],
+ "logviewerPort":8000
+}
+```
+
+### /api/v1/topology/\<id\> (GET)
+
+Returns topology information and statistics. Substitute \<id\> with the topology id.
+
+Request parameters:
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| Window duration for metrics in seconds|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+
+Response fields:
+
+|Field |Value |Description|
+|--- |--- |---
+|id| String| Topology Id|
+|name| String |Topology Name|
+|uptime| String |How long the topology has been running|
+|uptimeSeconds| Integer |How long the topology has been running in seconds|
+|status| String |Current status of the topology, e.g. "ACTIVE"|
+|tasksTotal| Integer |Total number of tasks for this topology|
+|workersTotal| Integer |Number of workers used for this topology|
+|executorsTotal| Integer |Number of executors used for this topology|
+|msgTimeout| Integer | Number of seconds a tuple has before the spout considers it failed |
+|windowHint| String | window param value in "hh mm ss" format. Default value is "All Time"|
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+|replicationCount| Integer |Number of nimbus hosts on which this topology code is replicated|
+|debug| Boolean | If debug is enabled for the topology|
+|samplingPct| Double| Controls downsampling of events before they are sent to event log (percentage)|
+|assignedMemOnHeap| Double|Assigned On-Heap Memory by Scheduler (MB)
+|assignedMemOffHeap| Double|Assigned Off-Heap Memory by Scheduler (MB)|
+|assignedTotalMem| Double|Assigned Off-Heap + On-Heap Memory by Scheduler(MB)|
+|assignedCpu| Double|Assigned CPU by Scheduler(%)|
+|requestedMemOnHeap| Double|Requested On-Heap Memory by User (MB)
+|requestedMemOffHeap| Double|Requested Off-Heap Memory by User (MB)|
+|requestedCpu| Double|Requested CPU by User (%)|
+|topologyStats| Array | Array of all the topology related stats per time window|
+|topologyStats.windowPretty| String |Duration passed in HH:MM:SS format|
+|topologyStats.window| String |User requested time window for metrics|
+|topologyStats.emitted| Long |Number of messages emitted in given window|
+|topologyStats.trasferred| Long |Number messages transferred in given window|
+|topologyStats.completeLatency| String (double value returned in String format) |Total latency for processing the message|
+|topologyStats.acked| Long |Number of messages acked in given window|
+|topologyStats.failed| Long |Number of messages failed in given window|
+|workers| Array | Array of workers in topology|
+|workers.supervisorId | String| Supervisor's id|
+|workers.host | String | Worker's host name|
+|workers.port | Integer | Worker's port|
+|workers.topologyId | String | Topology Id|
+|workers.topologyName | String | Topology Name|
+|workers.executorsTotal | Integer | Number of executors used by the topology in this worker|
+|workers.assignedMemOnHeap | Double | Assigned On-Heap Memory by Scheduler (MB)|
+|workers.assignedMemOffHeap | Double | Assigned Off-Heap Memory by Scheduler (MB)|
+|workers.assignedCpu | Number | Assigned CPU by Scheduler (%)|
+|workers.componentNumTasks | Dictionary | Components -> # of executing tasks|
+|workers.uptime| String| Shows how long the worker is running|
+|workers.uptimeSeconds| Integer| Shows how long the worker is running in seconds|
+|workers.workerLogLink | String | Link to worker log viewer page|
+|spouts| Array | Array of all the spout components in the topology|
+|spouts.spoutId| String |Spout id|
+|spouts.executors| Integer |Number of executors for the spout|
+|spouts.emitted| Long |Number of messages emitted in given window |
+|spouts.completeLatency| String (double value returned in String format) |Total latency for processing the message|
+|spouts.transferred| Long |Total number of messages transferred in given window|
+|spouts.tasks| Integer |Total number of tasks for the spout|
+|spouts.lastError| String |Shows the last error happened in a spout|
+|spouts.errorHost| String | Worker hostname the last error was reported on|
+|spouts.errorPort| String | Worker port the last error was reported on|
+|spouts.errorTime| Integer | Unix timestamp the last error was reported (seconds since epoch) |
+|spouts.errorLapsedSecs| Integer | Number of seconds elapsed since that last error happened in a spout|
+|spouts.errorWorkerLogLink| String | Link to the worker log that reported the exception |
+|spouts.acked| Long |Number of messages acked|
+|spouts.failed| Long |Number of messages failed|
+|spouts.requestedMemOnHeap| Double|Requested On-Heap Memory by User (MB)
+|spouts.requestedMemOffHeap| Double|Requested Off-Heap Memory by User (MB)|
+|spouts.requestedCpu| Double|Requested CPU by User (%)|
+|bolts| Array | Array of bolt components in the topology|
+|bolts.boltId| String |Bolt id|
+|bolts.capacity| String (double value returned in String format) |This value indicates number of messages executed * average execute latency / time window|
+|bolts.processLatency| String (double value returned in String format) |Average time of the bolt to ack a message after it was received|
+|bolts.executeLatency| String (double value returned in String format) |Average time to run the execute method of the bolt|
+|bolts.executors| Integer |Number of executor tasks in the bolt component|
+|bolts.tasks| Integer |Number of instances of bolt|
+|bolts.acked| Long |Number of tuples acked by the bolt|
+|bolts.failed| Long |Number of tuples failed by the bolt|
+|bolts.lastError| String |Shows the last error occurred in the bolt|
+|bolts.errorHost| String | Worker hostname the last error was reported on|
+|bolts.errorPort| String | Worker port the last error was reported on|
+|bolts.errorTime| Integer | Unix timestamp the last error was reported (seconds since epoch) |
+|bolts.errorLapsedSecs| Integer |Number of seconds elapsed since that last error happened in a bolt|
+|bolts.errorWorkerLogLink| String | Link to the worker log that reported the exception |
+|bolts.emitted| Long |Number of tuples emitted|
+|bolts.requestedMemOnHeap| Double|Requested On-Heap Memory by User (MB)
+|bolts.requestedMemOffHeap| Double|Requested Off-Heap Memory by User (MB)|
+|bolts.requestedCpu| Double|Requested CPU by User (%)|
+
+Examples:
+
+```no-highlight
+ 1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825
+ 2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?sys=1
+ 3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825?window=600
+```
+
+Sample response:
+
+```json
+ {
+ "name": "WordCount3",
+ "id": "WordCount3-1-1402960825",
+ "workersTotal": 3,
+ "window": "600",
+ "status": "ACTIVE",
+ "tasksTotal": 28,
+ "executorsTotal": 28,
+ "uptime": "29m 19s",
+ "uptimeSeconds": 1759,
+ "msgTimeout": 30,
+ "windowHint": "10m 0s",
+ "schedulerDisplayResource": true,
+ "workers": [{
+ "topologyName": "WordCount3",
+ "topologyId": "WordCount3-1-1402960825",
+ "host": "my-host",
+ "supervisorId": "9124ca9a-42e8-481e-9bf3-a041d9595430",
+ "assignedMemOnHeap": 1452.0,
+ "uptime": "27m 26s",
+ "port": 6702,
+ "workerLogLink": "logs",
+ "componentNumTasks": {
+ "spout": 2,
+ "count": 3,
+ "split": 10
+ },
+ "executorsTotal": 15,
+ "uptimeSeconds": 1646,
+ "assignedCpu": 260.0,
+ "assignedMemOffHeap": 160.0
+ }]
+ "topologyStats": [
+ {
+ "windowPretty": "10m 0s",
+ "window": "600",
+ "emitted": 397960,
+ "transferred": 213380,
+ "completeLatency": "0.000",
+ "acked": 213460,
+ "failed": 0
+ },
+ {
+ "windowPretty": "3h 0m 0s",
+ "window": "10800",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ },
+ {
+ "windowPretty": "1d 0h 0m 0s",
+ "window": "86400",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ },
+ {
+ "windowPretty": "All time",
+ "window": ":all-time",
+ "emitted": 1190260,
+ "transferred": 638260,
+ "completeLatency": "0.000",
+ "acked": 638280,
+ "failed": 0
+ }
+ ],
+ "workers":[{
+ "topologyName":"WordCount3",
+ "topologyId":"WordCount3-1-1402960825",
+ "host":"192.168.10.237",
+ "supervisorId":"bdfe8eff-f1d8-4bce-81f5-9d3ae1bf432e-169.254.129.212",
+ "uptime":"2m 47s",
+ "uptimeSeconds":167,
+ "port":6707,
+ "workerLogLink":"http:\/\/192.168.10.237:8000\/log?file=WordCount3-1-1402960825%2F6707%2Fworker.log",
+ "componentNumTasks": {
+ "spout":5
+ },
+ "executorsTotal":8,
+ "assignedMemOnHeap":704.0,
+ "assignedCpu":130.0,
+ "assignedMemOffHeap":80.0
+ }],
+ "spouts": [
+ {
+ "executors": 5,
+ "emitted": 28880,
+ "completeLatency": "0.000",
+ "transferred": 28880,
+ "acked": 0,
+ "spoutId": "spout",
+ "tasks": 5,
+ "lastError": "",
+ "errorHost": "",
+ "errorPort": null,
+ "errorWorkerLogLink": "",
+ "errorTime": null,
+ "errorLapsedSecs": null,
+ "failed": 0
+ }
+ ],
+ "bolts": [
+ {
+ "executors": 12,
+ "emitted": 184580,
+ "transferred": 0,
+ "acked": 184640,
+ "executeLatency": "0.048",
+ "tasks": 12,
+ "executed": 184620,
+ "processLatency": "0.043",
+ "boltId": "count",
+ "lastError": "",
+ "errorHost": "",
+ "errorPort": null,
+ "errorWorkerLogLink": "",
+ "errorTime": null,
+ "errorLapsedSecs": null,
+ "capacity": "0.003",
+ "failed": 0
+ },
+ {
+ "executors": 8,
+ "emitted": 184500,
+ "transferred": 184500,
+ "acked": 28820,
+ "executeLatency": "0.024",
+ "tasks": 8,
+ "executed": 28780,
+ "processLatency": "2.112",
+ "boltId": "split",
+ "lastError": "java.lang.RuntimeException: Error here! at org.apache.storm.starter.bolt.WordCountBolt.nextTuple(WordCountBolt.java:50) at org.apache.storm.executor.bolt.BoltExecutor$2.call",
+ "errorHost": "192.168.10.237",
+ "errorPort": 6707,
+ "errorWorkerLogLink": "http://192.168.10.237:8000/api/v1/log?file=WordCount3-1-1402960825%2F6707%2Fworker.log",
+ "errorTime": 1597626060,
+ "errorLapsedSecs": 65,
+ "capacity": "0.000",
+ "failed": 0
+ }
+ ],
+ "configuration": {
+ "storm.id": "WordCount3-1-1402960825",
+ "dev.zookeeper.path": "/tmp/dev-storm-zookeeper",
+ "topology.tick.tuple.freq.secs": null,
+ "topology.builtin.metrics.bucket.size.secs": 60,
+ "topology.fall.back.on.java.serialization": false,
+ "topology.max.error.report.per.interval": 5,
+ "zmq.linger.millis": 5000,
+ "topology.skip.missing.kryo.registrations": false,
+ "storm.messaging.netty.client_worker_threads": 1,
+ "ui.childopts": "-Xmx768m",
+ "storm.zookeeper.session.timeout": 20000,
+ "nimbus.reassign": true,
+ "topology.trident.batch.emit.interval.millis": 500,
+ "storm.messaging.netty.flush.check.interval.ms": 10,
+ "nimbus.monitor.freq.secs": 10,
+ "logviewer.childopts": "-Xmx128m",
+ "java.library.path": "/usr/local/lib:/opt/local/lib:/usr/lib",
+ "topology.executor.send.buffer.size": 1024,
+ "storm.local.dir": "storm-local",
+ "storm.messaging.netty.buffer_size": 5242880,
+ "supervisor.worker.start.timeout.secs": 120,
+ "topology.enable.message.timeouts": true,
+ "nimbus.cleanup.inbox.freq.secs": 600,
+ "nimbus.inbox.jar.expiration.secs": 3600,
+ "drpc.worker.threads": 64,
+ "topology.worker.shared.thread.pool.size": 4,
+ "nimbus.seeds": [
+ "hw10843.local"
+ ],
+ "storm.messaging.netty.min_wait_ms": 100,
+ "storm.zookeeper.port": 2181,
+ "transactional.zookeeper.port": null,
+ "topology.executor.receive.buffer.size": 1024,
+ "transactional.zookeeper.servers": null,
+ "storm.zookeeper.root": "/storm",
+ "storm.zookeeper.retry.intervalceiling.millis": 30000,
+ "supervisor.enable": true,
+ "storm.messaging.netty.server_worker_threads": 1
+ },
+ "replicationCount": 1
+}
+```
+
+### /api/v1/topology/\<id\>/metrics
+
+Returns detailed metrics for topology for a topology whose id is substituted for \<id\>. It shows metrics per component, which are aggregated by stream.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|window |String. Default value :all-time| window duration for metrics in seconds|
+|sys |String. Values 1 or 0. Default value 0| Controls including sys stats part of the response|
+
+Response fields:
+
+|Field |Value |Description|
+|--- |--- |---
+|window |String. Default value ":all-time" | window duration for metrics in seconds|
+ |windowHint| String | window param value in "hh mm ss" format. Default value is "All Time"|
+|spouts| Array | Array of all the spout components in the topology|
+|spouts.id| String |Spout id|
+|spouts.emitted| Array | Array of all the output streams this spout emits messages |
+|spouts.emitted.stream_id| String | Stream id for this stream |
+|spouts.emitted.value| Long | Number of messages emitted in given window|
+|spouts.transferred | Array | Array of all the output streams this spout transfers messages |
+|spouts.transferred.stream_id| String | Stream id for this stream |
+|spouts.transferred.value| Long |Number messages transferred in given window|
+|spouts.acked| Array | Array of all the output streams this spout receives ack of messages |
+|spouts.acked.stream_id| String | Stream id for this stream |
+|spouts.acked.value| Long |Number of messages acked in given window|
+|spouts.failed| Array | Array of all the output streams this spout receives fail of messages |
+|spouts.failed.stream_id| String | Stream id for this stream |
+|spouts.failed.value| Long |Number of messages failed in given window|
+|spouts.complete_ms_avg| Array | Array of all the output streams this spout receives ack of messages |
+|spouts.complete_ms_avg.stream_id| String | Stream id for this stream |
+|spouts.complete_ms_avg.value| String (double value returned in String format) | Total latency for processing the message|
+|bolts| Array | Array of all the bolt components in the topology|
+|bolts.id| String |Bolt id|
+|bolts.emitted| Array | Array of all the output streams this bolt emits messages |
+|bolts.emitted.stream_id| String | Stream id for this stream |
+|bolts.emitted.value| Long | Number of messages emitted in given window|
+|bolts.transferred | Array | Array of all the output streams this bolt transfers messages |
+|bolts.transferred.stream_id| String | Stream id for this stream |
+|bolts.transferred.value| Long |Number messages transferred in given window|
+|bolts.acked| Array | Array of all the input streams this bolt acknowledges of messages |
+|bolts.acked.component_id| String | Component id for this stream |
+|bolts.acked.stream_id| String | Stream id for this stream |
+|bolts.acked.value| Long |Number of messages acked in given window|
+|bolts.failed| Array | Array of all the input streams this bolt receives fail of messages |
+|bolts.failed.component_id| String | Component id for this stream |
+|bolts.failed.stream_id| String | Stream id for this stream |
+|bolts.failed.value| Long |Number of messages failed in given window|
+|bolts.process_ms_avg| Array | Array of all the input streams this spout acks messages |
+|bolts.process_ms_avg.component_id| String | Component id for this stream |
+|bolts.process_ms_avg.stream_id| String | Stream id for this stream |
+|bolts.process_ms_avg.value| String (double value returned in String format) |Average time of the bolt to ack a message after it was received|
+|bolts.executed| Array | Array of all the input streams this bolt executes messages |
+|bolts.executed.component_id| String | Component id for this stream |
+|bolts.executed.stream_id| String | Stream id for this stream |
+|bolts.executed.value| Long |Number of messages executed in given window|
+|bolts.executed_ms_avg| Array | Array of all the output streams this spout receives ack of messages |
+|bolts.executed_ms_avg.component_id| String | Component id for this stream |
+|bolts.executed_ms_avg.stream_id| String | Stream id for this stream |
+|bolts.executed_ms_avg.value| String (double value returned in String format) | Average time to run the execute method of the bolt|
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics
+1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?sys=1
+2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/metrics?window=600
+```
+
+Sample response:
+
+```json
+{
+ "window":":all-time",
+ "window-hint":"All time",
+ "spouts":[
+ {
+ "id":"spout",
+ "emitted":[
+ {
+ "stream_id":"__metrics",
+ "value":20
+ },
+ {
+ "stream_id":"default",
+ "value":17350280
+ },
+ {
+ "stream_id":"__ack_init",
+ "value":17328160
+ },
+ {
+ "stream_id":"__system",
+ "value":20
+ }
+ ],
+ "transferred":[
+ {
+ "stream_id":"__metrics",
+ "value":20
+ },
+ {
+ "stream_id":"default",
+ "value":17350280
+ },
+ {
+ "stream_id":"__ack_init",
+ "value":17328160
+ },
+ {
+ "stream_id":"__system",
+ "value":0
+ }
+ ],
+ "acked":[
+ {
+ "stream_id":"default",
+ "value":17339180
+ }
+ ],
+ "failed":[
+
+ ],
+ "complete_ms_avg":[
+ {
+ "stream_id":"default",
+ "value":"920.497"
+ }
+ ]
+ }
+ ],
+ "bolts":[
+ {
+ "id":"count",
+ "emitted":[
+ {
+ "stream_id":"__metrics",
+ "value":120
+ },
+ {
+ "stream_id":"default",
+ "value":190748180
+ },
+ {
+ "stream_id":"__ack_ack",
+ "value":190718100
+ },
+ {
+ "stream_id":"__system",
+ "value":20
+ }
+ ],
+ "transferred":[
+ {
+ "stream_id":"__metrics",
+ "value":120
+ },
+ {
+ "stream_id":"default",
+ "value":0
+ },
+ {
+ "stream_id":"__ack_ack",
+ "value":190718100
+ },
+ {
+ "stream_id":"__system",
+ "value":0
+ }
+ ],
+ "acked":[
+ {
+ "component_id":"split",
+ "stream_id":"default",
+ "value":190733160
+ }
+ ],
+ "failed":[
+
+ ],
+ "process_ms_avg":[
+ {
+ "component_id":"split",
+ "stream_id":"default",
+ "value":"0.004"
+ }
+ ],
+ "executed":[
+ {
+ "component_id":"split",
+ "stream_id":"default",
+ "value":190733140
+ }
+ ],
+ "executed_ms_avg":[
+ {
+ "component_id":"split",
+ "stream_id":"default",
+ "value":"0.005"
+ }
+ ]
+ },
+ {
+ "id":"split",
+ "emitted":[
+ {
+ "stream_id":"__metrics",
+ "value":60
+ },
+ {
+ "stream_id":"default",
+ "value":190754740
+ },
+ {
+ "stream_id":"__ack_ack",
+ "value":17317580
+ },
+ {
+ "stream_id":"__system",
+ "value":20
+ }
+ ],
+ "transferred":[
+ {
+ "stream_id":"__metrics",
+ "value":60
+ },
+ {
+ "stream_id":"default",
+ "value":190754740
+ },
+ {
+ "stream_id":"__ack_ack",
+ "value":17317580
+ },
+ {
+ "stream_id":"__system",
+ "value":0
+ }
+ ],
+ "acked":[
+ {
+ "component_id":"spout",
+ "stream_id":"default",
+ "value":17339180
+ }
+ ],
+ "failed":[
+
+ ],
+ "process_ms_avg":[
+ {
+ "component_id":"spout",
+ "stream_id":"default",
+ "value":"0.051"
+ }
+ ],
+ "executed":[
+ {
+ "component_id":"spout",
+ "stream_id":"default",
+ "value":17339240
+ }
+ ],
+ "executed_ms_avg":[
+ {
+ "component_id":"spout",
+ "stream_id":"default",
+ "value":"0.052"
+ }
+ ]
+ }
+ ]
+}
+```
+
+### /api/v1/topology/\<id\>/component/\<component\> (GET)
+
+Returns detailed metrics and executor information for a topology whose id is substituted for \<id\> and a component whose id is substituted for \<component\>
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|component |String (required)| Component Id |
+|window |String. Default value :all-time| window duration for metrics in seconds|
+|sys |String. Values 1 or 0. Default value 0| controls including sys stats part of the response|
+
+Response fields:
+
+|Field |Value |Description|
+|--- |--- |---
+|user | String | Topology owner|
+|id | String | Component id|
+|encodedId | String | URL encoded component id|
+|name | String | Topology name|
+|executors| Integer |Number of executor tasks in the component|
+|tasks| Integer |Number of instances of component|
+|requestedMemOnHeap| Double|Requested On-Heap Memory by User (MB)
+|requestedMemOffHeap| Double|Requested Off-Heap Memory by User (MB)|
+|requestedCpu| Double|Requested CPU by User (%)|
+|schedulerDisplayResource| Boolean | Whether to display scheduler resource information|
+|topologyId| String | Topology id|
+|topologyStatus| String | Topology status|
+|encodedTopologyId| String | URL encoded topology id|
+|window |String. Default value "All Time" | window duration for metrics in seconds|
+|componentType | String | component type: SPOUT or BOLT|
+|windowHint| String | window param value in "hh mm ss" format. Default value is "All Time"|
+|debug| Boolean | If debug is enabled for the component|
+|samplingPct| Double| Controls downsampling of events before they are sent to event log (percentage)|
+|eventLogLink| String| URL viewer link to event log (debug mode)|
+|profilingAndDebuggingCapable| Boolean |true if there is support for Profiling and Debugging Actions|
+|profileActionEnabled| Boolean |true if worker profiling (Java Flight Recorder) is enabled|
+|profilerActive| Array |Array of currently active Profiler Actions|
+|componentErrors| Array of Errors | List of component errors|
+|componentErrors.errorTime| Long | Timestamp when the exception occurred (Prior to 0.11.0, this field was named 'time'.)|
+|componentErrors.errorHost| String | host name for the error|
+|componentErrors.errorPort| String | port for the error|
+|componentErrors.error| String |Shows the error happened in a component|
+|componentErrors.errorLapsedSecs| Integer | Number of seconds elapsed since the error happened in a component |
+|componentErrors.errorWorkerLogLink| String | Link to the worker log that reported the exception |
+|spoutSummary| Array | (only for spouts) Array of component stats, one element per window.|
+|spoutSummary.windowPretty| String |Duration passed in HH:MM:SS format|
+|spoutSummary.window| String | window duration for metrics in seconds|
+|spoutSummary.emitted| Long |Number of messages emitted in given window |
+|spoutSummary.completeLatency| String (double value returned in String format) |Total latency for processing the message|
+|spoutSummary.transferred| Long |Total number of messages transferred in given window|
+|spoutSummary.acked| Long |Number of messages acked|
+|spoutSummary.failed| Long |Number of messages failed|
+|boltStats| Array | (only for bolts) Array of component stats, one element per window.|
+|boltStats.windowPretty| String |Duration passed in HH:MM:SS format|
+|boltStats.window| String| window duration for metrics in seconds|
+|boltStats.transferred| Long |Total number of messages transferred in given window|
+|boltStats.processLatency| String (double value returned in String format) |Average time of the bolt to ack a message after it was received|
+|boltStats.acked| Long |Number of messages acked|
+|boltStats.failed| Long |Number of messages failed|
+|inputStats| Array | (only for bolts) Array of input stats|
+|inputStats.component| String |Component id|
+|inputStats.encodedComponentId| String |URL encoded component id|
+|inputStats.executeLatency| Long | The average time a tuple spends in the execute method|
+|inputStats.processLatency| Long | The average time it takes to ack a tuple after it is first received|
+|inputStats.executed| Long |The number of incoming tuples processed|
+|inputStats.acked| Long |Number of messages acked|
+|inputStats.failed| Long |Number of messages failed|
+|inputStats.stream| String |The name of the tuple stream given in the topology, or "default" if none specified|
+|outputStats| Array | Array of output stats|
+|outputStats.transferred| Long |Number of tuples emitted that sent to one ore more bolts|
+|outputStats.emitted| Long |Number of tuples emitted|
+|outputStats.stream| String |The name of the tuple stream given in the topology, or "default" if none specified|
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout
+2. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?sys=1
+3. http://ui-daemon-host-name:8080/api/v1/topology/WordCount3-1-1402960825/component/spout?window=600
+```
+
+Sample response:
+
+```json
+{
+ "name": "WordCount3",
+ "id": "spout",
+ "componentType": "spout",
+ "windowHint": "10m 0s",
+ "executors": 5,
+ "componentErrors":[{"errorTime": 1406006074000,
+ "errorHost": "10.11.1.70",
+ "errorPort": 6701,
+ "errorWorkerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log",
+ "errorLapsedSecs": 16,
+ "error": "java.lang.RuntimeException: java.lang.StringIndexOutOfBoundsException: Some Error\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:128)\n\tat org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)\n\tat org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)\n\tat backtype...more.."
+ }],
+ "topologyId": "WordCount3-1-1402960825",
+ "tasks": 5,
+ "window": "600",
+ "profilerActive": [
+ {
+ "host": "10.11.1.70",
+ "port": "6701",
+ "dumplink":"http:\/\/10.11.1.70:8000\/dumps\/ex-1-1452718803\/10.11.1.70%3A6701",
+ "timestamp":"576328"
+ }
+ ],
+ "profilingAndDebuggingCapable": true,
+ "profileActionEnabled": true,
+ "spoutSummary": [
+ {
+ "windowPretty": "10m 0s",
+ "window": "600",
+ "emitted": 28500,
+ "transferred": 28460,
+ "completeLatency": "0.000",
+ "acked": 0,
+ "failed": 0
+ },
+ {
+ "windowPretty": "3h 0m 0s",
+ "window": "10800",
+ "emitted": 127640,
+ "transferred": 127440,
+ "completeLatency": "0.000",
+ "acked": 0,
+ "failed": 0
+ },
+ {
+ "windowPretty": "1d 0h 0m 0s",
+ "window": "86400",
+ "emitted": 127640,
+ "transferred": 127440,
+ "completeLatency": "0.000",
+ "acked": 0,
+ "failed": 0
+ },
+ {
+ "windowPretty": "All time",
+ "window": ":all-time",
+ "emitted": 127640,
+ "transferred": 127440,
+ "completeLatency": "0.000",
+ "acked": 0,
+ "failed": 0
+ }
+ ],
+ "outputStats": [
+ {
+ "stream": "__metrics",
+ "emitted": 40,
+ "transferred": 0,
+ "completeLatency": "0",
+ "acked": 0,
+ "failed": 0
+ },
+ {
+ "stream": "default",
+ "emitted": 28460,
+ "transferred": 28460,
+ "completeLatency": "0",
+ "acked": 0,
+ "failed": 0
+ }
+ ],
+ "executorStats": [
+ {
+ "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log",
+ "emitted": 5720,
+ "port": 6701,
+ "completeLatency": "0.000",
+ "transferred": 5720,
+ "host": "10.11.1.7",
+ "acked": 0,
+ "uptime": "43m 4s",
+ "uptimeSeconds": 2584,
+ "id": "[24-24]",
+ "failed": 0
+ },
+ {
+ "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log",
+ "emitted": 5700,
+ "port": 6703,
+ "completeLatency": "0.000",
+ "transferred": 5700,
+ "host": "10.11.1.7",
+ "acked": 0,
+ "uptime": "42m 57s",
+ "uptimeSeconds": 2577,
+ "id": "[25-25]",
+ "failed": 0
+ },
+ {
+ "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6702.log",
+ "emitted": 5700,
+ "port": 6702,
+ "completeLatency": "0.000",
+ "transferred": 5680,
+ "host": "10.11.1.7",
+ "acked": 0,
+ "uptime": "42m 57s",
+ "uptimeSeconds": 2577,
+ "id": "[26-26]",
+ "failed": 0
+ },
+ {
+ "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6701.log",
+ "emitted": 5700,
+ "port": 6701,
+ "completeLatency": "0.000",
+ "transferred": 5680,
+ "host": "10.11.1.7",
+ "acked": 0,
+ "uptime": "43m 4s",
+ "uptimeSeconds": 2584,
+ "id": "[27-27]",
+ "failed": 0
+ },
+ {
+ "workerLogLink": "http://10.11.1.7:8000/log?file=worker-6703.log",
+ "emitted": 5680,
+ "port": 6703,
+ "completeLatency": "0.000",
+ "transferred": 5680,
+ "host": "10.11.1.7",
+ "acked": 0,
+ "uptime": "42m 57s",
+ "uptimeSeconds": 2577,
+ "id": "[28-28]",
+ "failed": 0
+ }
+ ]
+}
+```
+
+## Profiling and Debugging POST Operations
+
+### /api/v1/topology/\<id\>/profiling/start/\<host-port\>/\<timeout\> (POST)
+
+Request to start profiler on worker with timeout. Returns status and link to profiler artifacts for worker.
+Substitute appropriate values for \<id\>, \<host-port\> and \<timeout\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|host-port |String (required)| Worker Id |
+|timeout |String (required)| Time out for profiler to stop in minutes |
+
+Response fields:
+
+|Field |Value |Description|
+|----- |----- |-----------|
+|id | String | Worker id|
+|status | String | Response Status |
+|timeout | String | Requested timeout
+|dumplink | String | Link to logviewer URL for worker profiler documents.|
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/10
+2. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/5
+3. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/start/10.11.1.7:6701/20
+```
+
+Sample response:
+
+```json
+{
+ "status": "ok",
+ "id": "10.11.1.7:6701",
+ "timeout": "10",
+ "dumplink": "http:\/\/10.11.1.7:8000\/dumps\/wordcount-1-1446614150\/10.11.1.7%3A6701"
+}
+```
+
+### /api/v1/topology/\<id\>/profiling/dumpprofile/\<host-port\> (POST)
+
+Request to dump profiler recording on worker. Returns status and worker id for the request.
+Substitute for \<id\> and \<host-port\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|host-port |String (required)| Worker Id |
+
+Response fields:
+
+|Field |Value |Description|
+|----- |----- |-----------|
+|id | String | Worker id|
+|status | String | Response Status |
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpprofile/10.11.1.7:6701
+```
+
+Sample response:
+
+```json
+{
+ "status": "ok",
+ "id": "10.11.1.7:6701",
+}
+```
+
+### /api/v1/topology/\<id\>/profiling/stop/\<host-port\> (POST)
+
+Request to stop profiler on worker. Returns status and worker id for the request.
+Substitute for \<id\> and \<host-port\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|host-port |String (required)| Worker Id |
+
+Response fields:
+
+|Field |Value |Description|
+|----- |----- |-----------|
+|id | String | Worker id|
+|status | String | Response Status |
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/stop/10.11.1.7:6701
+```
+
+Sample response:
+
+```json
+{
+ "status": "ok",
+ "id": "10.11.1.7:6701",
+}
+```
+
+### /api/v1/topology/\<id\>/profiling/dumpjstack/\<host-port\> (POST)
+
+Request to dump jstack on worker. Returns status and worker id for the request.
+Substitute for \<id\> and \<host-port\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|host-port |String (required)| Worker Id |
+
+Response fields:
+
+|Field |Value |Description|
+|----- |----- |-----------|
+|id | String | Worker id|
+|status | String | Response Status |
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpjstack/10.11.1.7:6701
+```
+
+Sample response:
+
+```json
+{
+ "status": "ok",
+ "id": "10.11.1.7:6701",
+}
+```
+
+### /api/v1/topology/\<id\>/profiling/dumpheap/\<host-port\> (POST)
+
+Request to dump heap (jmap) on worker. Returns status and worker id for the request.
+Substitute for \<id\> and \<host-port\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|host-port |String (required)| Worker Id |
+
+Response fields:
+
+|Field |Value |Description|
+|----- |----- |-----------|
+|id | String | Worker id|
+|status | String | Response Status |
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/dumpheap/10.11.1.7:6701
+```
+
+Sample response:
+
+```json
+{
+ "status": "ok",
+ "id": "10.11.1.7:6701",
+}
+```
+
+### /api/v1/topology/\<id\>/profiling/restartworker/\<host-port\> (POST)
+
+Request to request the worker. Returns status and worker id for the request.
+Substitute for \<id\> and \<host-port\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|host-port |String (required)| Worker Id |
+
+Response fields:
+
+|Field |Value |Description|
+|----- |----- |-----------|
+|id | String | Worker id|
+|status | String | Response Status |
+
+Examples:
+
+```no-highlight
+1. http://ui-daemon-host-name:8080/api/v1/topology/wordcount-1-1446614150/profiling/restartworker/10.11.1.7:6701
+```
+
+Sample response:
+
+```json
+{
+ "status": "ok",
+ "id": "10.11.1.7:6701",
+}
+```
+
+## POST Operations
+
+### /api/v1/topology/\<id\>/activate (POST)
+
+Activates a topology. Substitute for \<id\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+
+Sample Response:
+
+```json
+{"topologyOperation":"activate","topologyId":"wordcount-1-1420308665","status":"success"}
+```
+
+
+### /api/v1/topology/\<id\>/deactivate (POST)
+
+Deactivates a topology. Substitute for \<id\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+
+Sample Response:
+
+```json
+{"topologyOperation":"deactivate","topologyId":"wordcount-1-1420308665","status":"success"}
+```
+
+
+### /api/v1/topology/\<id\>/rebalance/\<wait-time\> (POST)
+
+Rebalances a topology.
+Substitute for \<id\> and \<wait-time\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|wait-time |String (required)| Wait time before rebalance happens |
+|rebalanceOptions| Json (optional) | topology rebalance options |
+
+
+Sample rebalanceOptions json:
+
+```json
+{"rebalanceOptions" : {"numWorkers" : 2, "executors" : {"spout" :4, "count" : 10}}, "callback" : "foo"}
+```
+
+Examples:
+
+```no-highlight
+curl -i -b ~/cookiejar.txt -c ~/cookiejar.txt -X POST
+-H "Content-Type: application/json"
+-d '{"rebalanceOptions": {"numWorkers": 2, "executors": { "spout" : "5", "split": 7, "count": 5 }}, "callback":"foo"}'
+http://localhost:8080/api/v1/topology/wordcount-1-1420308665/rebalance/0
+```
+
+Sample Response:
+
+```json
+{"topologyOperation":"rebalance","topologyId":"wordcount-1-1420308665","status":"success"}
+```
+
+
+
+### /api/v1/topology/\<id\>/kill/\<wait-time\> (POST)
+
+Kills a topology.
+Substitute for \<id\> and \<wait-time\>.
+
+|Parameter |Value |Description |
+|----------|--------|-------------|
+|id |String (required)| Topology Id |
+|wait-time |String (required)| Wait time before rebalance happens |
+
+Caution: Small wait times (0-5 seconds) may increase the probability of triggering the bug reported in
+[STORM-112](https://issues.apache.org/jira/browse/STORM-112), which may result in broker Supervisor
+daemons.
+
+Sample Response:
+
+```json
+{"topologyOperation":"kill","topologyId":"wordcount-1-1420308665","status":"success"}
+```
+
+## API errors
+
+The API returns 500 HTTP status codes in case of any errors.
+
+# DRPC REST API
+
+If DRPC is configured with either an http or https port it will expose a REST endpoint. (See [Setting up a Storm cluster](Setting-up-a-Storm-cluster.html) for how to do that)
+
+In all of these commands `:func` is the DRPC function and `:args` is the arguments to it. The only difference is in how those arguments are supplied. In all cases the response
+is in the response's body.
+
+In all cases DRPC does not have state, so if your request times out or results in an error please retry the request, but preferably with an exponential backoff to avoid doing a
+DDOS on the DRPC servers.
+
+### /drpc/:func (POST)
+
+In this case the `:args` to the drpc request are in the body of the post.
+
+### /drpc/:func/:args (GET)
+
+In this case the `:args` are supplied as a part of the URL itself. There are limitations on URL lengths by many tools, so if this is above a hundred characters it is recomended
+to use the POST option instead.
+
+### /drpc/:func (GET)
+
+In some rare cases `:args` may not be needed by the DRPC command. If no `:args` section is given in the DRPC request and empty string `""` will be used for the arguments.
+
+
diff --git a/releases/2.8.8/Serialization-(prior-to-0.6.0).md b/releases/2.8.8/Serialization-(prior-to-0.6.0).md
new file mode 100644
index 0000000000..9e84482a5b
--- /dev/null
+++ b/releases/2.8.8/Serialization-(prior-to-0.6.0).md
@@ -0,0 +1,50 @@
+---
+layout: documentation
+---
+Tuples can be comprised of objects of any types. Since Storm is a distributed system, it needs to know how to serialize and deserialize objects when they're passed between tasks. By default Storm can serialize ints, shorts, longs, floats, doubles, bools, bytes, strings, and byte arrays, but if you want to use another type in your tuples, you'll need to implement a custom serializer.
+
+### Dynamic typing
+
+There are no type declarations for fields in a Tuple. You put objects in fields and Storm figures out the serialization dynamically. Before we get to the interface for serialization, let's spend a moment understanding why Storm's tuples are dynamically typed.
+
+Adding static typing to tuple fields would add large amount of complexity to Storm's API. Hadoop, for example, statically types its keys and values but requires a huge amount of annotations on the part of the user. Hadoop's API is a burden to use and the "type safety" isn't worth it. Dynamic typing is simply easier to use.
+
+Further than that, it's not possible to statically type Storm's tuples in any reasonable way. Suppose a Bolt subscribes to multiple streams. The tuples from all those streams may have different types across the fields. When a Bolt receives a `Tuple` in `execute`, that tuple could have come from any stream and so could have any combination of types. There might be some reflection magic you can do to declare a different method for every tuple stream a bolt subscribes to, but Storm opts for [...]
+
+Finally, another reason for using dynamic typing is so Storm can be used in a straightforward manner from dynamically typed languages like Clojure and JRuby.
+
+### Custom serialization
+
+Let's dive into Storm's API for defining custom serializations. There are two steps you need to take as a user to create a custom serialization: implement the serializer, and register the serializer to Storm.
+
+#### Creating a serializer
+
+Custom serializers implement the [ISerialization](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/backtype/storm/serialization/ISerialization.html) interface. Implementations specify how to serialize and deserialize types into a binary format.
+
+The interface looks like this:
+
+```java
+public interface ISerialization<T> {
+ public boolean accept(Class c);
+ public void serialize(T object, DataOutputStream stream) throws IOException;
+ public T deserialize(DataInputStream stream) throws IOException;
+}
+```
+
+Storm uses the `accept` method to determine if a type can be serialized by this serializer. Remember, Storm's tuples are dynamically typed so Storm determines what serializer to use at runtime.
+
+`serialize` writes the object out to the output stream in binary format. The field must be written in a way such that it can be deserialized later. For example, if you're writing out a list of objects, you'll need to write out the size of the list first so that you know how many elements to deserialize.
+
+`deserialize` reads the serialized object off of the stream and returns it.
+
+You can see example serialization implementations in the source for [SerializationFactory](https://github.com/apache/incubator-storm/blob/0.5.4/src/jvm/backtype/storm/serialization/SerializationFactory.java)
+
+#### Registering a serializer
+
+Once you create a serializer, you need to tell Storm it exists. This is done through the Storm configuration (See [Concepts](Concepts.html) for information about how configuration works in Storm). You can register serializations either through the config given when submitting a topology or in the storm.yaml files across your cluster.
+
+Serializer registrations are done through the Config.TOPOLOGY_SERIALIZATIONS config and is simply a list of serialization class names.
+
+Storm provides helpers for registering serializers in a topology config. The [Config](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/backtype/storm/Config.html) class has a method called `addSerialization` that takes in a serializer class to add to the config.
+
+There's an advanced config called Config.TOPOLOGY_SKIP_MISSING_SERIALIZATIONS. If you set this to true, Storm will ignore any serializations that are registered but do not have their code available on the classpath. Otherwise, Storm will throw errors when it can't find a serialization. This is useful if you run many topologies on a cluster that each have different serializations, but you want to declare all the serializations across all topologies in the `storm.yaml` files.
diff --git a/releases/2.8.8/Serialization.md b/releases/2.8.8/Serialization.md
new file mode 100644
index 0000000000..f4413f1d02
--- /dev/null
+++ b/releases/2.8.8/Serialization.md
@@ -0,0 +1,68 @@
+---
+title: Serialization
+layout: documentation
+documentation: true
+---
+This page is about how the serialization system in Storm works for versions 0.6.0 and onwards. Storm used a different serialization system prior to 0.6.0 which is documented on [Serialization (prior to 0.6.0)](Serialization-\(prior-to-0.6.0\).html).
+
+Tuples can be comprised of objects of any types. Since Storm is a distributed system, it needs to know how to serialize and deserialize objects when they're passed between tasks.
+
+Storm uses [Kryo](https://github.com/EsotericSoftware/kryo) for serialization. Kryo is a flexible and fast serialization library that produces small serializations.
+
+By default, Storm can serialize primitive types, strings, byte arrays, ArrayList, HashMap, and HashSet. If you want to use another type in your tuples, you'll need to register a custom serializer.
+
+### Dynamic typing
+
+There are no type declarations for fields in a Tuple. You put objects in fields and Storm figures out the serialization dynamically. Before we get to the interface for serialization, let's spend a moment understanding why Storm's tuples are dynamically typed.
+
+Adding static typing to tuple fields would add large amount of complexity to Storm's API. Hadoop, for example, statically types its keys and values but requires a huge amount of annotations on the part of the user. Hadoop's API is a burden to use and the "type safety" isn't worth it. Dynamic typing is simply easier to use.
+
+Further than that, it's not possible to statically type Storm's tuples in any reasonable way. Suppose a Bolt subscribes to multiple streams. The tuples from all those streams may have different types across the fields. When a Bolt receives a `Tuple` in `execute`, that tuple could have come from any stream and so could have any combination of types. There might be some reflection magic you can do to declare a different method for every tuple stream a bolt subscribes to, but Storm opts for [...]
+
+Finally, another reason for using dynamic typing is so Storm can be used in a straightforward manner from dynamically typed languages like Clojure and JRuby.
+
+### Custom serialization
+
+As mentioned, Storm uses Kryo for serialization. To implement custom serializers, you need to register new serializers with Kryo. It's highly recommended that you read over [Kryo's home page](https://github.com/EsotericSoftware/kryo) to understand how it handles custom serialization.
+
+Adding custom serializers is done through the "topology.kryo.register" property in your topology config or through a ServiceLoader described later. The config takes a list of registrations, where each registration can take one of two forms:
+
+1. The name of a class to register. In this case, Storm will use Kryo's `FieldsSerializer` to serialize the class. This may or may not be optimal for the class -- see the Kryo docs for more details.
+2. A map from the name of a class to register to an implementation of [com.esotericsoftware.kryo.Serializer](https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/Serializer.java).
+
+Let's look at an example.
+
+```
+topology.kryo.register:
+ - com.mycompany.CustomType1
+ - com.mycompany.CustomType2: com.mycompany.serializer.CustomType2Serializer
+ - com.mycompany.CustomType3
+```
+
+`com.mycompany.CustomType1` and `com.mycompany.CustomType3` will use the `FieldsSerializer`, whereas `com.mycompany.CustomType2` will use `com.mycompany.serializer.CustomType2Serializer` for serialization.
+
+Storm provides helpers for registering serializers in a topology config. The [Config](https://javadoc.io/doc/org.apache.storm/storm-client/2.8.8/org/apache/storm/Config.html) class has a method called `registerSerialization` that takes in a registration to add to the config.
+
+There's an advanced config called `Config.TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS`. If you set this to true, Storm will ignore any serializations that are registered but do not have their code available on the classpath. Otherwise, Storm will throw errors when it can't find a serialization. This is useful if you run many topologies on a cluster that each have different serializations, but you want to declare all the serializations across all topologies in the `storm.yaml` files.
+
+#### SerializationRegister Service Loader
+
+If you want to provide language bindings to storm, have a library that you want to interact cleanly with storm or have some other reason to provide serialization bindings and don't want to force the user to update their configs you can use the org.apache.storm.serialization.SerializationRegister service loader.
+
+You may use this like any other service loader and storm will register the bindings without forceing users to update their configs. The storm-clojure package uses this to provide transparent support for clojure types.
+
+### Java serialization
+
+When `Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION` is set true, if Storm encounters a type for which it doesn't have a serialization registered, it will use Java serialization if possible. If the object can't be serialized with Java serialization, then Storm will throw an error.
+
+Beware that Java serialization is extremely expensive, both in terms of CPU cost as well as the size of the serialized object. It is highly recommended that you register custom serializers when you put the topology in production. The Java serialization behavior is there so that it's easy to prototype new topologies.
+
+You can turn on/off the behavior to fall back on Java serialization by setting the `Config.TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION` config to true/false. The default value is false for security reasons.
+
+### Component-specific serialization registrations
+
+Storm 0.7.0 lets you set component-specific configurations (read more about this at [Configuration](Configuration.html)). Of course, if one component defines a serialization that serialization will need to be available to other bolts -- otherwise they won't be able to receive messages from that component!
+
+When a topology is submitted, a single set of serializations is chosen to be used by all components in the topology for sending messages. This is done by merging the component-specific serializer registrations with the regular set of serialization registrations. If two components define serializers for the same class, one of the serializers is chosen arbitrarily.
+
+To force a serializer for a particular class if there's a conflict between two component-specific registrations, just define the serializer you want to use in the topology-specific configuration. The topology-specific configuration has precedence over component-specific configurations for serialization registrations.
diff --git a/releases/2.8.8/Serializers.md b/releases/2.8.8/Serializers.md
new file mode 100644
index 0000000000..071c885117
--- /dev/null
+++ b/releases/2.8.8/Serializers.md
@@ -0,0 +1,4 @@
+---
+layout: documentation
+---
+* [storm-json](https://github.com/rapportive-oss/storm-json): Simple JSON serializer for Storm
diff --git a/releases/2.8.8/Setting-up-a-Storm-cluster.md b/releases/2.8.8/Setting-up-a-Storm-cluster.md
new file mode 100644
index 0000000000..558defa5ed
--- /dev/null
+++ b/releases/2.8.8/Setting-up-a-Storm-cluster.md
@@ -0,0 +1,135 @@
+---
+title: Setting up a Storm Cluster
+layout: documentation
+documentation: true
+---
+This page outlines the steps for getting a Storm cluster up and running.
+
+If you run into difficulties with your Storm cluster, first check for a solution is in the [Troubleshooting](Troubleshooting.html) page. Otherwise, email the mailing list.
+
+Here's a summary of the steps for setting up a Storm cluster:
+
+1. Set up a Zookeeper cluster
+2. Install dependencies on Nimbus and worker machines
+3. Download and extract a Storm release to Nimbus and worker machines
+4. Fill in mandatory configurations into storm.yaml
+5. Launch daemons under supervision using "storm" script and a supervisor of your choice
+6. Setup DRPC servers (Optional)
+
+### Set up a Zookeeper cluster
+
+Storm uses Zookeeper for coordinating the cluster. Zookeeper **is not** used for message passing, so the load Storm places on Zookeeper is quite low. Single node Zookeeper clusters should be sufficient for most cases, but if you want failover or are deploying large Storm clusters you may want larger Zookeeper clusters. Instructions for deploying Zookeeper are [here](http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html).
+
+A few notes about Zookeeper deployment:
+
+1. It's critical that you run Zookeeper under supervision, since Zookeeper is fail-fast and will exit the process if it encounters any error case. See [here](http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_supervision) for more details.
+2. It's critical that you set up a cron to compact Zookeeper's data and transaction logs. The Zookeeper daemon does not do this on its own, and if you don't set up a cron, Zookeeper will quickly run out of disk space. See [here](http://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_maintenance) for more details.
+
+### Install dependencies on Nimbus and worker machines
+
+Next you need to install Storm's dependencies on Nimbus and the worker machines. These are:
+
+1. Java 11+ (Apache Storm 2.x is tested through GitHub actions against Java 11, Java 17 and Java 21)
+2. Python 3.x
+
+These are the versions of the dependencies that have been tested with Storm. Storm may or may not work with different versions of Java and/or Python.
+
+
+### Download and extract a Storm release to Nimbus and worker machines
+
+Next, download a Storm release and extract the zip file somewhere on Nimbus and each of the worker machines. The Storm releases can be downloaded [from here](../../downloads.html).
+
+### Fill in mandatory configurations into storm.yaml
+
+The Storm release contains a file at `conf/storm.yaml` that configures the Storm daemons. You can see the default configuration values [here]({{page.git-blob-base}}/conf/defaults.yaml). storm.yaml overrides anything in defaults.yaml. There's a few configurations that are mandatory to get a working cluster:
+
+1) **storm.zookeeper.servers**: This is a list of the hosts in the Zookeeper cluster for your Storm cluster. It should look something like:
+
+```yaml
+storm.zookeeper.servers:
+ - "111.222.333.444"
+ - "555.666.777.888"
+```
+
+If the port that your Zookeeper cluster uses is different than the default, you should set **storm.zookeeper.port** as well.
+
+2) **storm.local.dir**: The Nimbus and Supervisor daemons require a directory on the local disk to store small amounts of state (like jars, confs, and things like that).
+ You should create that directory on each machine, give it proper permissions, and then fill in the directory location using this config. For example:
+
+```yaml
+storm.local.dir: "/mnt/storm"
+```
+If you run storm on windows, it could be:
+
+```yaml
+storm.local.dir: "C:\\storm-local"
+```
+If you use a relative path, it will be relative to where you installed storm(STORM_HOME).
+You can leave it empty with default value `$STORM_HOME/storm-local`
+
+3) **nimbus.seeds**: The worker nodes need to know which machines are the candidate of master in order to download topology jars and confs. For example:
+
+```yaml
+nimbus.seeds: ["111.222.333.44"]
+```
+You're encouraged to fill out the value to list of **machine's FQDN**. If you want to set up Nimbus H/A, you have to address all machines' FQDN which run nimbus. You may want to leave it to default value when you just want to set up 'pseudo-distributed' cluster, but you're still encouraged to fill out FQDN.
+
+4) **supervisor.slots.ports**: For each worker machine, you configure how many workers run on that machine with this config. Each worker uses a single port for receiving messages, and this setting defines which ports are open for use. If you define five ports here, then Storm will allocate up to five workers to run on this machine. If you define three ports, Storm will only run up to three. By default, this setting is configured to run 4 workers on the ports 6700, 6701, 6702, and 6703. F [...]
+
+```yaml
+supervisor.slots.ports:
+ - 6700
+ - 6701
+ - 6702
+ - 6703
+```
+
+5) **drpc.servers**: If you want to setup DRPC servers they need to specified so that the workers can find them. This should be a list of the DRPC servers. For example:
+
+```yaml
+drpc.servers: ["111.222.333.44"]
+```
+
+### Monitoring Health of Supervisors
+
+Storm provides a mechanism by which administrators can configure the supervisor to run administrator supplied scripts periodically to determine if a node is healthy or not. Administrators can have the supervisor determine if the node is in a healthy state by performing any checks of their choice in scripts located in storm.health.check.dir. If a script detects the node to be in an unhealthy state, it must return a non-zero exit code. In pre-Storm 2.x releases, a bug considered a script e [...]
+
+If the supervisor is running with supervision "/bin/storm node-health-check" can be called to determine if the supervisor should be launched or if the node is unhealthy.
+
+The health check directory location can be configured with:
+
+```yaml
+storm.health.check.dir: "healthchecks"
+
+```
+The scripts must have execute permissions.
+The time to allow any given healthcheck script to run before it is marked failed due to timeout can be configured with:
+
+```yaml
+storm.health.check.timeout.ms: 5000
+```
+
+### Configure external libraries and environment variables (optional)
+
+If you need support from external libraries or custom plugins, you can place such jars into the extlib/ and extlib-daemon/ directories. Note that the extlib-daemon/ directory stores jars used only by daemons (Nimbus, Supervisor, DRPC, UI, Logviewer), e.g., HDFS and customized scheduling libraries. Accordingly, two environment variables STORM_EXT_CLASSPATH and STORM_EXT_CLASSPATH_DAEMON can be configured by users for including the external classpath and daemon-only external classpath. See [...]
+
+
+### Launch daemons under supervision using "storm" script and a supervisor of your choice
+
+The last step is to launch all the Storm daemons. It is critical that you run each of these daemons under supervision. Storm is a __fail-fast__ system which means the processes will halt whenever an unexpected error is encountered. Storm is designed so that it can safely halt at any point and recover correctly when the process is restarted. This is why Storm keeps no state in-process -- if Nimbus or the Supervisors restart, the running topologies are unaffected. Here's how to run the Sto [...]
+
+1. **Nimbus**: Run the command `bin/storm nimbus` under supervision on the master machine.
+2. **Supervisor**: Run the command `bin/storm supervisor` under supervision on each worker machine. The supervisor daemon is responsible for starting and stopping worker processes on that machine.
+3. **UI**: Run the Storm UI (a site you can access from the browser that gives diagnostics on the cluster and topologies) by running the command "bin/storm ui" under supervision. The UI can be accessed by navigating your web browser to http://{ui host}:8080.
+
+As you can see, running the daemons is very straightforward. The daemons will log to the logs/ directory in wherever you extracted the Storm release.
+
+### Setup DRPC servers (Optional)
+
+Just like with nimbus or the supervisors you will need to launch the drpc server. To do this run the command `bin/storm drpc` on each of the machines that you configured as a part of the `drpc.servers` config.
+
+#### DRPC Http Setup
+
+DRPC optionally offers a REST API as well. To enable this set teh config `drpc.http.port` to the port you want to run on before launching the DRPC server. See the [REST documentation](STORM-UI-REST-API.html) for more information on how to use it.
+
+It also supports SSL by setting `drpc.https.port` along with the keystore and optional truststore similar to how you would configure the UI.
diff --git a/releases/2.8.8/Setting-up-a-Storm-project-in-Eclipse.md b/releases/2.8.8/Setting-up-a-Storm-project-in-Eclipse.md
new file mode 100644
index 0000000000..5137cd9e32
--- /dev/null
+++ b/releases/2.8.8/Setting-up-a-Storm-project-in-Eclipse.md
@@ -0,0 +1 @@
+- fill me in
\ No newline at end of file
diff --git a/releases/2.8.8/Setting-up-development-environment.md b/releases/2.8.8/Setting-up-development-environment.md
new file mode 100644
index 0000000000..72e34727ed
--- /dev/null
+++ b/releases/2.8.8/Setting-up-development-environment.md
@@ -0,0 +1,33 @@
+---
+title: Setting Up a Development Environment
+layout: documentation
+documentation: true
+---
+This page outlines what you need to do to get a Storm development environment set up. In summary, the steps are:
+
+1. Download a [Storm release](../../downloads.html) , unpack it, and put the unpacked `bin/` directory on your PATH
+2. To be able to start and stop topologies on a remote cluster, put the cluster information in `~/.storm/storm.yaml`
+
+More detail on each of these steps is below.
+
+### What is a development environment?
+
+Storm has two modes of operation: local mode and remote mode. In local mode, you can develop and test topologies completely in process on your local machine. In remote mode, you submit topologies for execution on a cluster of machines.
+
+A Storm development environment has everything installed so that you can develop and test Storm topologies in local mode, package topologies for execution on a remote cluster, and submit/kill topologies on a remote cluster.
+
+Let's quickly go over the relationship between your machine and a remote cluster. A Storm cluster is managed by a master node called "Nimbus". Your machine communicates with Nimbus to submit code (packaged as a jar) and topologies for execution on the cluster, and Nimbus will take care of distributing that code around the cluster and assigning workers to run your topology. Your machine uses a command line client called `storm` to communicate with Nimbus. The `storm` client is only used f [...]
+
+### Installing a Storm release locally
+
+If you want to be able to submit topologies to a remote cluster from your machine, you should install a Storm release locally. Installing a Storm release will give you the `storm` client that you can use to interact with remote clusters. To install Storm locally, download a release [from here](https://github.com/apache/storm/releases) and unzip it somewhere on your computer. Then add the unpacked `bin/` directory onto your `PATH` and make sure the `bin/storm` script is executable.
+
+Installing a Storm release locally is only for interacting with remote clusters. For developing and testing topologies in local mode, it is recommended that you use Maven to include Storm as a dev dependency for your project. You can read more about using Maven for this purpose on [Maven](Maven.html).
+
+### Starting and stopping topologies on a remote cluster
+
+The previous step installed the `storm` client on your machine which is used to communicate with remote Storm clusters. Now all you have to do is tell the client which Storm cluster to talk to. To do this, all you have to do is put the host address of the master in the `~/.storm/storm.yaml` file. It should look something like this:
+
+```
+nimbus.seeds: ["123.45.678.890"]
+```
diff --git a/releases/2.8.8/Spout-implementations.md b/releases/2.8.8/Spout-implementations.md
new file mode 100644
index 0000000000..f52e662dc0
--- /dev/null
+++ b/releases/2.8.8/Spout-implementations.md
@@ -0,0 +1,10 @@
+---
+title: Spout Implementations
+layout: documentation
+documentation: true
+---
+* [storm-kestrel](https://github.com/nathanmarz/storm-kestrel): Adapter to use Kestrel as a spout
+* [storm-amqp-spout](https://github.com/rapportive-oss/storm-amqp-spout): Adapter to use AMQP source as a spout
+* [storm-jms](https://github.com/ptgoetz/storm-jms): Adapter to use a JMS source as a spout
+* [storm-redis-pubsub](https://github.com/sorenmacbeth/storm-redis-pubsub): A spout that subscribes to a Redis pubsub stream
+* [storm-beanstalkd-spout](https://github.com/haitaoyao/storm-beanstalkd-spout): A spout that subscribes to a beanstalkd queue
diff --git a/releases/2.8.8/State-checkpointing.md b/releases/2.8.8/State-checkpointing.md
new file mode 100644
index 0000000000..687ea818f8
--- /dev/null
+++ b/releases/2.8.8/State-checkpointing.md
@@ -0,0 +1,289 @@
+---
+title: Storm State Management
+layout: documentation
+documentation: true
+---
+# State support in core storm
+Storm core has abstractions for bolts to save and retrieve the state of its operations. There is a default in-memory
+based state implementation and also a Redis backed implementation that provides state persistence.
+
+## State management
+Bolts that requires its state to be managed and persisted by the framework should implement the `IStatefulBolt` interface or
+extend the `BaseStatefulBolt` and implement `void initState(T state)` method. The `initState` method is invoked by the framework
+during the bolt initialization with the previously saved state of the bolt. This is invoked after prepare but before the bolt starts
+processing any tuples.
+
+Currently the only kind of `State` implementation that is supported is `KeyValueState` which provides key-value mapping.
+
+For example a word count bolt could use the key value state abstraction for the word counts as follows.
+
+1. Extend the BaseStatefulBolt and type parameterize it with KeyValueState which would store the mapping of word to count.
+2. The bolt gets initialized with its previously saved state in the init method. This will contain the word counts
+last committed by the framework during the previous run.
+3. In the execute method, update the word count.
+
+```java
+ public class WordCountBolt extends BaseStatefulBolt<KeyValueState<String, Long>> {
+ private KeyValueState<String, Long> wordCounts;
+ private OutputCollector collector;
+ ...
+ @Override
+ public void prepare(Map stormConf, TopologyContext context, OutputCollector collector) {
+ this.collector = collector;
+ }
+ @Override
+ public void initState(KeyValueState<String, Long> state) {
+ wordCounts = state;
+ }
+ @Override
+ public void execute(Tuple tuple) {
+ String word = tuple.getString(0);
+ Integer count = wordCounts.get(word, 0);
+ count++;
+ wordCounts.put(word, count);
+ collector.emit(tuple, new Values(word, count));
+ collector.ack(tuple);
+ }
+ ...
+ }
+```
+
+4. The framework periodically checkpoints the state of the bolt (default every second). The frequency
+can be changed by setting the storm config `topology.state.checkpoint.interval.ms`
+5. For state persistence, use a state provider that supports persistence by setting the `topology.state.provider` in the
+storm config. E.g. for using Redis based key-value state implementation set `topology.state.provider: org.apache.storm.redis.state.RedisKeyValueStateProvider`
+in storm.yaml. The provider implementation jar should be in the class path, which in this case means adding `storm-redis`
+to dependency of your topology.
+6. The state provider properties can be overridden by setting `topology.state.provider.config`. For Redis state this is a
+json config with the following properties.
+
+```
+{
+ "keyClass": "Optional fully qualified class name of the Key type.",
+ "valueClass": "Optional fully qualified class name of the Value type.",
+ "keySerializerClass": "Optional Key serializer implementation class.",
+ "valueSerializerClass": "Optional Value Serializer implementation class.",
+ "jedisPoolConfig": {
+ "host": "localhost",
+ "port": 6379,
+ "timeout": 2000,
+ "database": 0,
+ "password": "xyz"
+ }
+}
+```
+
+For Redis Cluster state this is a json config with the following properties.
+
+```
+ {
+ "keyClass": "Optional fully qualified class name of the Key type.",
+ "valueClass": "Optional fully qualified class name of the Value type.",
+ "keySerializerClass": "Optional Key serializer implementation class.",
+ "valueSerializerClass": "Optional Value Serializer implementation class.",
+ "jedisClusterConfig": {
+ "nodes": ["localhost:7379", "localhost:7380", "localhost:7381"],
+ "timeout": 2000,
+ "maxRedirections": 5
+ }
+ }
+```
+
+NOTE: If you used Redis state with Storm version 1.1.0 or earlier, you would need to also migrate your state since the representation of state has changed
+from Base64-encoded string to binary to reduce huge overhead. Storm provides a migration tool to help, which is placed on `storm-redis-example` module.
+
+Please download the source from download page or clone the project, and type below command:
+
+```
+mvn clean install -DskipTests
+cd examples/storm-redis-examples
+<storm-installation-dir>/bin/storm jar target/storm-redis-examples-*.jar org.apache.storm.redis.tools.Base64ToBinaryStateMigrationUtil [options]
+```
+
+Supported options are listed here:
+
+```
+ -d,--dbnum <arg> Redis DB number (default: 0)
+ -h,--host <arg> Redis hostname (default: localhost)
+ -n,--namespace <arg> REQUIRED the list of namespace to migrate.
+ -p,--port <arg> Redis port (default: 6379)
+ --password <arg> Redis password (default: no password)
+```
+
+You can provide multiple `namespace` options to migrate multiple namespaces at once.
+(e.g.: `--namespace total-7 --namespace partialsum-3`)
+Other options are not mandatory.
+Please note that you need to also migrate the key starting with "$checkpointspout-" since it's internal namespace of state.
+
+## Checkpoint mechanism
+Checkpoint is triggered by an internal checkpoint spout at the specified `topology.state.checkpoint.interval.ms`. If there is
+at-least one `IStatefulBolt` in the topology, the checkpoint spout is automatically added by the topology builder . For stateful topologies,
+the topology builder wraps the `IStatefulBolt` in a `StatefulBoltExecutor` which handles the state commits on receiving the checkpoint tuples.
+The non stateful bolts are wrapped in a `CheckpointTupleForwarder` which just forwards the checkpoint tuples so that the checkpoint tuples
+can flow through the topology DAG. The checkpoint tuples flow through a separate internal stream namely `$checkpoint`. The topology builder
+wires the checkpoint stream across the whole topology with the checkpoint spout at the root.
+
+```
+ default default default
+[spout1] ---------------> [statefulbolt1] ----------> [bolt1] --------------> [statefulbolt2]
+ | ----------> -------------->
+ | ($chpt) ($chpt)
+ |
+[$checkpointspout] _______| ($chpt)
+```
+
+At checkpoint intervals the checkpoint tuples are emitted by the checkpoint spout. On receiving a checkpoint tuple, the state of the bolt
+is saved and then the checkpoint tuple is forwarded to the next component. Each bolt waits for the checkpoint to arrive on all its input
+streams before it saves its state so that the state represents a consistent state across the topology. Once the checkpoint spout receives
+ACK from all the bolts, the state commit is complete and the transaction is recorded as committed by the checkpoint spout.
+
+The state checkpointing does not currently checkpoint the state of the spout. Yet, once the state of all bolts are checkpointed, and once the checkpoint tuples are acked, the tuples emitted by the spout are also acked.
+It also implies that `topology.state.checkpoint.interval.ms` is lower than `topology.message.timeout.secs`.
+
+The state commit works like a three phase commit protocol with a prepare and commit phase so that the state across the topology is saved
+in a consistent and atomic manner.
+
+### Recovery
+The recovery phase is triggered when the topology is started for the first time. If the previous transaction was not successfully
+prepared, a `rollback` message is sent across the topology so that if a bolt has some prepared transactions it can be discarded.
+If the previous transaction was prepared successfully but not committed, a `commit` message is sent across the topology so that
+the prepared transactions can be committed. After these steps are complete, the bolts are initialized with the state.
+
+The recovery is also triggered if one of the bolts fails to acknowledge the checkpoint message or say a worker crashed in
+the middle. Thus when the worker is restarted by the supervisor, the checkpoint mechanism makes sure that the bolt gets
+initialized with its previous state and the checkpointing continues from the point where it left off.
+
+### Guarantee
+Storm relies on the acking mechanism to replay tuples in case of failures. It is possible that the state is committed
+but the worker crashes before acking the tuples. In this case the tuples are replayed causing duplicate state updates.
+Also currently the StatefulBoltExecutor continues to process the tuples from a stream after it has received a checkpoint
+tuple on one stream while waiting for checkpoint to arrive on other input streams for saving the state. This can also cause
+duplicate state updates during recovery.
+
+The state abstraction does not eliminate duplicate evaluations and currently provides only at-least once guarantee.
+
+In order to provide the at-least once guarantee, all bolts in a stateful topology are expected to anchor the tuples
+while emitting and ack the input tuples once its processed. For non-stateful bolts, the anchoring/acking can be automatically
+managed by extending the `BaseBasicBolt`. Stateful bolts are expected to anchor tuples while emitting and ack the tuple
+after processing like in the `WordCountBolt` example in the State management section above.
+
+### IStateful bolt hooks
+IStateful bolt interface provides hook methods where in the stateful bolts could implement some custom actions.
+
+```java
+ /**
+ * This is a hook for the component to perform some actions just before the
+ * framework commits its state.
+ */
+ void preCommit(long txid);
+
+ /**
+ * This is a hook for the component to perform some actions just before the
+ * framework prepares its state.
+ */
+ void prePrepare(long txid);
+
+ /**
+ * This is a hook for the component to perform some actions just before the
+ * framework rolls back the prepared state.
+ */
+ void preRollback();
+```
+
+This is optional and stateful bolts are not expected to provide any implementation. This is provided so that other
+system level components can be built on top of the stateful abstractions where we might want to take some actions before the
+stateful bolt's state is prepared, committed or rolled back.
+
+## Providing custom state implementations
+Currently the only kind of `State` implementation supported is `KeyValueState` which provides key-value mapping.
+
+Custom state implementations should provide implementations for the methods defined in the `org.apache.storm.State` interface.
+These are the `void prepareCommit(long txid)`, `void commit(long txid)`, `rollback()` methods. `commit()` method is optional
+and is useful if the bolt manages the state on its own. This is currently used only by the internal system bolts,
+for e.g. the CheckpointSpout to save its state.
+
+`KeyValueState` implementation should also implement the methods defined in the `org.apache.storm.state.KeyValueState` interface.
+
+### State provider
+The framework instantiates the state via the corresponding `StateProvider` implementation. A custom state should also provide
+a `StateProvider` implementation which can load and return the state based on the namespace. Each state belongs to a unique namespace.
+The namespace is typically unique per task so that each task can have its own state. The StateProvider and the corresponding
+State implementation should be available in the class path of Storm (by placing them in the extlib directory).
+
+
+### Supported State Backends
+
+#### Redis
+
+* State provider class name (`topology.state.provider`)
+
+`org.apache.storm.redis.state.RedisKeyValueStateProvider`
+
+* Provider config (`topology.state.provider.config`)
+
+```
+ {
+ "keyClass": "Optional fully qualified class name of the Key type.",
+ "valueClass": "Optional fully qualified class name of the Value type.",
+ "keySerializerClass": "Optional Key serializer implementation class.",
+ "valueSerializerClass": "Optional Value Serializer implementation class.",
+ "jedisPoolConfig": {
+ "host": "localhost",
+ "port": 6379,
+ "timeout": 2000,
+ "database": 0,
+ "password": "xyz"
+ }
+ }
+ ```
+
+* Artifacts to add (`--artifacts`)
+
+`org.apache.storm:storm-redis:<storm-version>`
+
+#### HBase
+
+In order to make state scalable, HBaseKeyValueState stores state KV to a row. This introduces `non-atomic` commit phase and guarantee
+eventual consistency on HBase side. It doesn't matter in point of state's view because HBaseKeyValueState can still provide not-yet-committed value.
+Even if worker crashes at commit phase, after restart it will read pending-commit states (stored atomically) from HBase and states will be stored eventually.
+
+NOTE: HBase state provider uses pre-created table and column family, so users need to create and provide one to the provider config.
+
+You can simply create table via `create 'state', 'cf'` in `hbase shell` but in production you may want to give some more properties.
+
+* State provider class name (`topology.state.provider`)
+
+`org.apache.storm.hbase.state.HBaseKeyValueStateProvider`
+
+* Provider config (`topology.state.provider.config`)
+
+```
+ {
+ "keyClass": "Optional fully qualified class name of the Key type.",
+ "valueClass": "Optional fully qualified class name of the Value type.",
+ "keySerializerClass": "Optional Key serializer implementation class.",
+ "valueSerializerClass": "Optional Value Serializer implementation class.",
+ "hbaseConfigKey": "config key to load hbase configuration from storm root configuration. (similar to storm-hbase)",
+ "tableName": "Pre-created table name for state.",
+ "columnFamily": "Pre-created column family for state."
+ }
+ ```
+
+If you want to initialize HBase state provider from codebase, please see below example:
+
+```
+Config conf = new Config();
+ Map<String, Object> hbConf = new HashMap<String, Object>();
+ hbConf.put("hbase.rootdir", "file:///tmp/hbase");
+ conf.put("hbase.conf", hbConf);
+ conf.put("topology.state.provider", "org.apache.storm.hbase.state.HBaseKeyValueStateProvider");
+ conf.put("topology.state.provider.config", "{" +
+ " \"hbaseConfigKey\": \"hbase.conf\"," +
+ " \"tableName\": \"state\"," +
+ " \"columnFamily\": \"cf\"" +
+ " }");
+```
+
+* Artifacts to add (`--artifacts`)
+
+`org.apache.storm:storm-hbase:<storm-version>`
\ No newline at end of file
diff --git a/releases/2.8.8/Storm-Scheduler.md b/releases/2.8.8/Storm-Scheduler.md
new file mode 100644
index 0000000000..4b9807ea29
--- /dev/null
+++ b/releases/2.8.8/Storm-Scheduler.md
@@ -0,0 +1,27 @@
+---
+title: Scheduler
+layout: documentation
+documentation: true
+---
+
+Storm now has 4 kinds of built-in schedulers: [DefaultScheduler]({{page.git-blob-base}}/storm-server/src/main/java/org/apache/storm/scheduler/DefaultScheduler.java), [IsolationScheduler]({{page.git-blob-base}}/storm-server/src/main/java/org/apache/storm/scheduler/IsolationScheduler.java), [MultitenantScheduler]({{page.git-blob-base}}/storm-server/src/main/java/org/apache/storm/scheduler/multitenant/MultitenantScheduler.java), [ResourceAwareScheduler](Resource_Aware_Scheduler_overview.html).
+
+## Pluggable scheduler
+You can implement your own scheduler to replace the default scheduler to assign executors to workers. You configure the class to use the "storm.scheduler" config in your storm.yaml, and your scheduler must implement the [IScheduler]({{page.git-blob-base}}/storm-client/src/jvm/org/apache/storm/scheduler/IScheduler.java) interface.
+
+## Isolation Scheduler
+The isolation scheduler makes it easy and safe to share a cluster among many topologies. The isolation scheduler lets you specify which topologies should be "isolated", meaning that they run on a dedicated set of machines within the cluster where no other topologies will be running. These isolated topologies are given priority on the cluster, so resources will be allocated to isolated topologies if there's competition with non-isolated topologies, and resources will be taken away from no [...]
+
+You can configure the isolation scheduler in the Nimbus configuration by setting "storm.scheduler" to "org.apache.storm.scheduler.IsolationScheduler". Then, use the "isolation.scheduler.machines" config to specify how many machines each topology should get. This configuration is a map from topology name to the number of isolated machines allocated to this topology. For example:
+
+```
+isolation.scheduler.machines:
+ "my-topology": 8
+ "tiny-topology": 1
+ "some-other-topology": 3
+```
+
+Any topologies submitted to the cluster not listed there will not be isolated. Note that there is no way for a user of Storm to affect their isolation settings – this is only allowed by the administrator of the cluster (this is very much intentional).
+
+The isolation scheduler solves the multi-tenancy problem – avoiding resource contention between topologies – by providing full isolation between topologies. The intention is that "productionized" topologies should be listed in the isolation config, and test or in-development topologies should not. The remaining machines on the cluster serve the dual role of failover for isolated topologies and for running the non-isolated topologies.
+
diff --git a/releases/2.8.8/Storm-multi-language-protocol-(versions-0.7.0-and-below).md b/releases/2.8.8/Storm-multi-language-protocol-(versions-0.7.0-and-below).md
new file mode 100644
index 0000000000..1d4422f7b0
--- /dev/null
+++ b/releases/2.8.8/Storm-multi-language-protocol-(versions-0.7.0-and-below).md
@@ -0,0 +1,122 @@
+---
+layout: documentation
+---
+This page explains the multilang protocol for versions 0.7.0 and below. The protocol changed in version 0.7.1.
+
+# Storm Multi-Language Protocol
+
+## The ShellBolt
+
+Support for multiple languages is implemented via the ShellBolt class. This
+class implements the IBolt interfaces and implements the protocol for
+executing a script or program via the shell using Java's ProcessBuilder class.
+
+## Output fields
+
+Output fields are part of the Thrift definition of the topology. This means that when you multilang in Java, you need to create a bolt that extends ShellBolt, implements IRichBolt, and declared the fields in `declareOutputFields`.
+You can learn more about this on [Concepts](Concepts.html)
+
+## Protocol Preamble
+
+A simple protocol is implemented via the STDIN and STDOUT of the executed
+script or program. A mix of simple strings and JSON encoded data are exchanged
+with the process making support possible for pretty much any language.
+
+# Packaging Your Stuff
+
+To run a ShellBolt on a cluster, the scripts that are shelled out to must be
+in the `resources/` directory within the jar submitted to the master.
+
+However, During development or testing on a local machine, the resources
+directory just needs to be on the classpath.
+
+## The Protocol
+
+Notes:
+* Both ends of this protocol use a line-reading mechanism, so be sure to
+trim off newlines from the input and to append them to your output.
+* All JSON inputs and outputs are terminated by a single line contained "end".
+* The bullet points below are written from the perspective of the script writer's
+STDIN and STDOUT.
+
+
+* Your script will be executed by the Bolt.
+* STDIN: A string representing a path. This is a PID directory.
+Your script should create an empty file named with it's pid in this directory. e.g.
+the PID is 1234, so an empty file named 1234 is created in the directory. This
+file lets the supervisor know the PID so it can shutdown the process later on.
+* STDOUT: Your PID. This is not JSON encoded, just a string. ShellBolt will log the PID to its log.
+* STDIN: (JSON) The Storm configuration. Various settings and properties.
+* STDIN: (JSON) The Topology context
+* The rest happens in a while(true) loop
+* STDIN: A tuple! This is a JSON encoded structure like this:
+
+```
+{
+ // The tuple's id
+ "id": -6955786537413359385,
+ // The id of the component that created this tuple
+ "comp": 1,
+ // The id of the stream this tuple was emitted to
+ "stream": 1,
+ // The id of the task that created this tuple
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["snow white and the seven dwarfs", "field2", 3]
+}
+```
+
+* STDOUT: The results of your bolt, JSON encoded. This can be a sequence of acks, fails, emits, and/or logs. Emits look like:
+
+```
+{
+ "command": "emit",
+ // The ids of the tuples this output tuples should be anchored to
+ "anchors": [1231231, -234234234],
+ // The id of the stream this tuple was emitted to. Leave this empty to emit to default stream.
+ "stream": 1,
+ // If doing an emit direct, indicate the task to sent the tuple to
+ "task": 9,
+ // All the values in this tuple
+ "tuple": ["field1", 2, 3]
+}
+```
+
+An ack looks like:
+
+```
+{
+ "command": "ack",
+ // the id of the tuple to ack
+ "id": 123123
+}
+```
+
... 16451 lines suppressed ...