You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@solr.apache.org by ja...@apache.org on 2026/05/22 00:53:26 UTC
(solr-orbit) 03/06: Add documentation site
This is an automated email from the ASF dual-hosted git repository.
janhoy pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/solr-orbit.git
commit abff1eb365019508f82141f14e300fe701346b12
Author: Jan Høydahl <ja...@cominvent.com>
AuthorDate: Fri May 22 00:54:56 2026 +0200
Add documentation site
Complete documentation overhaul for the Solr port:
- Replace OpenSearch-focused Jekyll docs site with Solr-specific content covering
installation, configuration, running benchmarks, workload creation, and the
new OpenSearch→Solr converter
- Remove legacy docs/api/ (OpenSearch API reference) and docs/user-guides/
in favor of the new docs/user-guide/ and docs/reference/ structure
- Update README.md, DEVELOPER_GUIDE.md, PYTHON_SUPPORT_GUIDE.md,
CREATE_WORKLOAD_GUIDE.md to use Solr terminology and solr-benchmark CLI
- Add TODO.md: incubation checklist and known remaining work
- Add it/README.md: integration test setup instructions
- Remove opensearch_benchmark.png splash image
Part of #3
---
CREATE_WORKLOAD_GUIDE.md | 184 ++++-----
DEVELOPER_GUIDE.md | 302 +++++----------
PYTHON_SUPPORT_GUIDE.md | 118 +++---
README.md | 118 +++---
TODO.md | 96 +++++
docker/README.md | 54 +--
docs/.gitignore | 6 +
docs/.ruby-version | 1 +
docs/Gemfile | 4 +
docs/README.md | 139 +++++++
docs/_config.yml | 75 ++++
docs/_includes/footer_custom.html | 9 +
docs/about.md | 31 ++
docs/api/client-options.md | 6 -
docs/api/index.md | 6 -
docs/api/kill-running-process.md | 6 -
docs/api/pipeline.md | 6 -
docs/api/run-test.md | 102 -----
docs/api/target-hosts.md | 6 -
docs/api/test-mode.md | 6 -
docs/api/workload.md | 6 -
docs/cluster-config/available-configs.md | 214 +++++++++++
docs/cluster-config/index.md | 46 +++
docs/converter/index.md | 33 ++
docs/converter/usage.md | 66 ++++
docs/converter/what-converts.md | 61 +++
docs/faq.md | 58 +++
docs/get-started.md | 6 -
docs/glossary.md | 65 ++++
docs/index.md | 28 ++
docs/quickstart.md | 191 ++++++++++
docs/reference/commands/aggregate.md | 94 +++++
docs/reference/commands/benchmarkd.md | 71 ++++
docs/reference/commands/command-flags.md | 195 ++++++++++
docs/reference/commands/compare.md | 49 +++
docs/reference/commands/create-workload.md | 57 +++
docs/reference/commands/download.md | 51 +++
docs/reference/commands/generate-data.md | 58 +++
docs/reference/commands/index.md | 39 ++
docs/reference/commands/info.md | 48 +++
docs/reference/commands/install.md | 81 ++++
docs/reference/commands/list.md | 70 ++++
docs/reference/commands/run.md | 164 ++++++++
docs/reference/commands/start.md | 53 +++
docs/reference/commands/stop.md | 42 +++
docs/reference/commands/visualize.md | 56 +++
docs/reference/index.md | 14 +
docs/reference/metrics/filesystem-metrics-store.md | 98 +++++
docs/reference/metrics/index.md | 64 ++++
docs/reference/metrics/metric-records.md | 82 ++++
docs/reference/metrics/metrics-reference.md | 309 +++++++++++++++
docs/reference/summary-report.md | 85 +++++
docs/reference/telemetry.md | 291 ++++++++++++++
docs/reference/workloads/collections.md | 62 +++
docs/reference/workloads/corpora.md | 72 ++++
docs/reference/workloads/index.md | 10 +
docs/reference/workloads/operations.md | 274 ++++++++++++++
docs/reference/workloads/test-procedures.md | 75 ++++
docs/user-guide/concepts.md | 120 ++++++
docs/user-guide/index.md | 18 +
.../install-and-configure/configuring.md | 148 ++++++++
docs/user-guide/install-and-configure/index.md | 10 +
.../user-guide/install-and-configure/installing.md | 181 +++++++++
.../optimizing-benchmarks/best-practices.md | 108 ++++++
docs/user-guide/optimizing-benchmarks/index.md | 17 +
.../optimizing-benchmarks/randomizing-queries.md | 112 ++++++
.../running-distributed-load.md | 105 ++++++
.../optimizing-benchmarks/target-throughput.md | 79 ++++
docs/user-guide/understanding-results/index.md | 10 +
.../understanding-results/summary-reports.md | 136 +++++++
docs/user-guide/understanding-results/telemetry.md | 94 +++++
.../anatomy-of-a-workload.md | 419 +++++++++++++++++++++
.../understanding-workloads/choosing-a-workload.md | 82 ++++
.../understanding-workloads/common-operations.md | 184 +++++++++
docs/user-guide/understanding-workloads/index.md | 10 +
.../contributing-workloads.md | 18 +
.../creating-custom-workloads.md | 287 ++++++++++++++
.../working-with-workloads/finetune-workloads.md | 205 ++++++++++
docs/user-guide/working-with-workloads/index.md | 17 +
.../working-with-workloads/running-workloads.md | 216 +++++++++++
docs/user-guides/create-pipeline.md | 129 -------
docs/user-guides/index.md | 0
it/README.md | 13 +
opensearch_benchmark.png | Bin 16845 -> 0 bytes
84 files changed, 6633 insertions(+), 698 deletions(-)
diff --git a/CREATE_WORKLOAD_GUIDE.md b/CREATE_WORKLOAD_GUIDE.md
index 80d2b9c8..676cc26b 100644
--- a/CREATE_WORKLOAD_GUIDE.md
+++ b/CREATE_WORKLOAD_GUIDE.md
@@ -1,114 +1,120 @@
# Create Workload Guide
-This guide explores how users can use the `create-workload` subcommand in OpenSearch Benchmark to create a workload based on pre-existing data in a cluster.
-
-### Create a Workload from Pre-Existing Indices in a Cluster
+This guide explains how to create a Solr Benchmark workload.
+
+## Option 1: Write a workload from scratch
+
+A workload is a JSON file with the following structure:
+
+```json
+{
+ "name": "my-workload",
+ "description": "Benchmark description",
+ "collections": [
+ {
+ "name": "my-collection",
+ "configset": "my-configset",
+ "configset-path": "/path/to/configset/dir",
+ "num-shards": 1,
+ "replication-factor": 1
+ }
+ ],
+ "challenges": [
+ {
+ "name": "default",
+ "description": "Default challenge",
+ "schedule": [
+ {"operation": {"operation-type": "delete-collection", "ignore-missing": true}},
+ {"operation": {"operation-type": "create-collection"}},
+ {"operation": {"operation-type": "bulk-index", "bulk-size": 500}},
+ {"operation": {"operation-type": "search", "q": "*:*", "rows": 10}},
+ {"operation": {"operation-type": "optimize"}},
+ {"operation": {"operation-type": "delete-collection"}}
+ ]
+ }
+ ]
+}
+```
-**Prerequisites:**
-* OpenSearch cluster with data ingested into it in an index. Ensure that index has 1000+ docs. If not, a workload will be created but users cannot run the workload with `--test-mode`.
-* Ensure that your cluster is permissive.
+Run the workload with:
-Create a workload with the following command:
+```bash
+solr-benchmark run \
+ --pipeline=benchmark-only \
+ --workload-path=/path/to/workload.json \
+ --target-host="localhost:8983"
```
-$ opensearch-benchmark create-workload \
---workload="<WORKLOAD NAME>" \
---target-hosts="<CLUSTER ENDPOINT>" \
---client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" \
---indices="<INDICES TO GENERATE WORKLOAD FROM>" \
---output-path="<LOCAL DIRECTORY PATH TO STORE WORKLOAD>"
-```
-Note that:
-* `--indices` can be 1+ indices specified in a comma-separated list.
-* If the cluster uses basic authentication and has TLS enabled, users will need to provide them through `--client-options`.
-The following is an example output of when a user creates a workload from an index called movies that contains 2000 docs.
+## Option 2: Migrate an existing OSB workload
+
+If you have a workload written for OpenSearch Benchmark, use the migration
+utility to translate it to Solr format:
+```bash
+python -m osbenchmark.tools.migrate_workload \
+ --input osb-workload.json \
+ --output solr-workload.json
```
- ____ _____ __ ____ __ __
- / __ \____ ___ ____ / ___/___ ____ ___________/ /_ / __ )___ ____ _____/ /_ ____ ___ ____ ______/ /__
- / / / / __ \/ _ \/ __ \\__ \/ _ \/ __ `/ ___/ ___/ __ \ / __ / _ \/ __ \/ ___/ __ \/ __ `__ \/ __ `/ ___/ //_/
-/ /_/ / /_/ / __/ / / /__/ / __/ /_/ / / / /__/ / / / / /_/ / __/ / / / /__/ / / / / / / / / /_/ / / / ,<
-\____/ .___/\___/_/ /_/____/\___/\__,_/_/ \___/_/ /_/ /_____/\___/_/ /_/\___/_/ /_/_/ /_/ /_/\__,_/_/ /_/|_|
- /_/
-[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
-[INFO] Connected to OpenSearch cluster [380d8fd64dd85b5f77c0ad81b0799e1e] version [1.1.0].
+The tool will:
+- Rename `indices` → `collections`
+- Translate `bulk` → `bulk-index`, `force-merge` → `optimize`,
+ `create-index` → `create-collection`, `delete-index` → `delete-collection`
+- Rename the `index` operation parameter → `collection`
+- Mark unsupported operations with a `_migration_todo` note — **no operation
+ is ever silently dropped**
-Extracting documents for index [movies] for test mode... 1000/1000 docs [100.0% done]
-Extracting documents for index [movies]... 2000/2000 docs [100.0% done]
+Review the output file and resolve any `_migration_todo` items before running.
-[INFO] Workload movies has been created. Run it with: opensearch-benchmark --workload-path=/Users/hoangia/Desktop/workloads/movies
+## Supported operation types
--------------------------------
-[INFO] SUCCESS (took 2 seconds)
--------------------------------
-```
+| Operation type | Description |
+|---|---|
+| `bulk-index` | Index documents from an NDJSON corpus |
+| `search` | Run a Solr query (classic params or JSON DSL `body`) |
+| `commit` | Issue a hard or soft commit |
+| `optimize` | Merge segments |
+| `create-collection` | Upload configset then create collection |
+| `delete-collection` | Delete collection (and optionally its configset) |
+| `raw-request` | Issue an arbitrary Solr V2 API request |
-By default, workloads created will come with the following operations run in the following order:
-* **delete-index**: Deletes any pre-existing indices with the same name(s) as the indices provided in `--indices`
-* **create-index**: Creates the index with the same name(s) as the indices provided in `--indices`
-* **cluster-health**: Verifies that cluster health is green before proceeding with the ingestion
-* **bulk**: Ingests documents collected from the indices specified in `--indices`
-* **default**: Runs a match-all query on the index for a number of iterations
+## Corpus format
+
+Document corpora use NDJSON (newline-delimited JSON), with alternating action
+and document lines — the same format as OpenSearch/Elasticsearch bulk API:
-To invoke the newly created workload, run the following:
```
-$ opensearch-benchmark run \
---pipeline="benchmark-only" \
---workload-path="<PATH OUTPUTTED IN THE OUTPUT OF THE CREATE-WORKLOAD COMMAND>" \
---target-host="<CLUSTER ENDPOINT>" \
---client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'"
+{"index": {"_id": "1", "_index": "my-collection"}}
+{"title": "Hello world", "body": "..."}
+{"index": {"_id": "2"}}
+{"title": "Second doc", "body": "..."}
```
-Users have the options to specify a subset of documents from the index or override the default match_all query. See the following sections for more information on how.
+The `_id` field is mapped to Solr's `id` field. `_index` and `_type` are used
+for routing/logging only and are not stored in the document.
-### Adding Custom Queries
-Add `--custom-queries` to the `create-workload` command. This parameter takes in a JSON filepath. This overrides the default match_all query with the queries present in the input file.
+## Custom queries
-Requirements:
-* Ensure that queries are properly formatted and adhere to JSON schema
-* Ensure that all queries are contained within a list. Exception: If providing only a single query, it does not have to be in a list.
+For search operations using Solr's JSON Query DSL, pass a `body` dict:
-Adding to the previous example, a user wants to override default query with the following two custom queries in a JSON file.
-```
-[
- {
- "name": "default",
- "operation-type": "search",
- "body": {
- "query": {
- "match_all": {}
- }
- }
- },
- {
- "name": "term",
- "operation-type": "search",
- "body": {
- "query": {
- "term": {
- "director": "Ian"
- }
- }
- }
+```json
+{
+ "operation-type": "search",
+ "body": {
+ "query": {"field": {"title": "hello"}},
+ "limit": 10
}
-]
+}
```
-To do this, the user can provide the JSON filepath to `--custom-queries` parameter:
-```
-$ opensearch-benchmark create-workload \
---workload="<WORKLOAD NAME>" \
---target-hosts="<CLUSTER ENDPOINT>" \
---client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" \
---indices="<INDICES TO GENERATE WORKLOAD FROM>" \
---output-path="<LOCAL DIRECTORY PATH TO STORE WORKLOAD>" \
---custom-queries="<JSON filepath containing queries>"
-```
+For classic Solr query syntax, use `q`, `fl`, `rows`, `fq`, and `sort`
+parameters directly:
-### Common Errors
-When adding custom queries, users might experience the following error will occur if the queries do not adhere to JSON schema standards or are not in a list.
+```json
+{
+ "operation-type": "search",
+ "q": "title:hello",
+ "rows": 10,
+ "fl": "id,title,score"
+}
```
-[INFO] You did not provide an explicit timeout in the client options. Assuming default of 10 seconds.
-[ERROR] Cannot create-workload. Ensure JSON schema is valid and queries are contained in a list: Extra data: line 9 column 2 (char 113)
-```
\ No newline at end of file
diff --git a/DEVELOPER_GUIDE.md b/DEVELOPER_GUIDE.md
index f4ee51b0..00c87790 100644
--- a/DEVELOPER_GUIDE.md
+++ b/DEVELOPER_GUIDE.md
@@ -1,264 +1,158 @@
# Developer Guide
-This document will walk you through on what's needed to start contributing code to OpenSearch Benchmark.
+This document walks you through what's needed to start contributing code to
+Apache Solr Benchmark.
### Table of Contents
-- [Installation](#installation)
- - [Prerequisites](#prerequisites)
- - [Setup](#setup)
+- [Prerequisites](#prerequisites)
+- [Setup](#setup)
- [Importing the project into an IDE](#importing-the-project-into-an-ide)
-- [Setting Up a Local OpenSearch Cluster For OSB Development (Optional)](#setting-up-a-local-opensearch-cluster-for-osb-development-optional)
-- [running tests](#running-tests)
- - [Unit tests](#unit-tests)
- - [Integration tests](#integration-tests)
-- [Submitting your changes for a pull request](#submitting-your-changes-for-a-pull-request)
-- [Developing breaking changes](#developing-breaking-changes)
+- [Setting Up a Local Solr Instance (Optional)](#setting-up-a-local-solr-instance-optional)
+- [Running Tests](#running-tests)
+- [Submitting a Pull Request](#submitting-a-pull-request)
+- [Developing Breaking Changes](#developing-breaking-changes)
- [Miscellaneous](#miscellaneous)
- - [git-secrets](#git-secrets)
- - [Adding new major and minor Python versions to OpenSearch Benchmark](#adding-new-major-and-minor-python-versions-to-opensearch-benchmark)
- - [Debugging OpenSearch Benchmark in Developer Mode](#debugging-opensearch-benchmark-in-developer-mode)
- - [Debugging unittests in Visual Studio Code](#debugging-unittests-in-visual-studio-code)
-## Installation
+## Prerequisites
-### Prerequisites
+- **Python 3.9+**: Use [pyenv](https://github.com/pyenv/pyenv) to manage
+ Python versions.
- - **pyenv**: Install `pyenv` and follow the instructions in the output of `pyenv init` to set up your shell and restart it before proceeding.
- For more details please refer to the [pyenv installation instructions](https://github.com/pyenv/pyenv#installation).
-
- `pyenv` requires that the C compiler and development libraries be installed, so that the specified Python versions can be build from source. The installation instructions vary from platform to platform.
-
- For Debian-based systems, install the following modules to continue with the next steps:
- ```
- sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
+ Debian/Ubuntu:
+ ```bash
+ sudo apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \
libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \
xz-utils tk-dev libffi-dev liblzma-dev git
- ```
-
- For Amazon Linux 2023, run the following command:
- ```
- sudo yum install -y gcc zlib-devel bzip2 bzip2-devel patch readline-devel sqlite sqlite-devel openssl-devel tk-devel libffi-devel xz-devel git
-
- git clone https://github.com/pyenv/pyenv.git ~/.pyenv
-
- echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
- echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
- echo 'eval "$(pyenv init -)"' >> ~/.bash_profile
-
- source ~/.bash_profile
- ```
-
- On the Mac platform, XCode needs to be installed as well as some additional required libraries:
- ```
- xcode-select --install
- brew install pyenv jq zlib xz
- ```
+ ```
- - **JDK**: Although OSB is a Python application, it optionally builds and provisions OpenSearch clusters. JDK version 21 is required to build the current version of OpenSearch. Please refer to the [build setup requirements](https://github.com/opensearch-project/OpenSearch/blob/ca564fd04f5059cf9e3ce8aba442575afb3d99f1/DEVELOPER_GUIDE.md#install-prerequisites).
- Note that the `javadoc` executable should be available in the JDK installation.
+ macOS:
+ ```bash
+ xcode-select --install
+ brew install pyenv jq zlib xz
+ ```
- If OSB is asked to run with an earlier version of OpenSearch, an older version of the JDK such as version 17 will be required. Version 21 has eliminated support for the Security Manager which is used by these OpenSearch releases.
+- **Docker** (optional): required for the `docker` pipeline.
+ Install Docker and confirm `docker ps` works.
- The integration tests exercise both older versions of OpenSearch and the latest version as part of the test suite. Therefore, two versions of the JDK will need to be installed and the appropriate environment variables set, like so:
+- **Git 1.9+**
- ```
- export JAVA21_HOME=/path/to/JDK21
- export JAVA17_HOME=/path/to/JDK17
+## Setup
- ```
+Fork and clone the repository, then install in development mode:
- - **Docker**: Install Docker and `docker-compose`. Start the Docker server. The user running the integration tests should have the permissions required to run docker commands. Test by running `docker ps`.
-
- - **Git** : supports versions 1.9+
-
-### Setup
-
-To develop OSB properly, it is recommended that you fork the official OpenSearch Benchmark repository.
-
-For those working on WSL2, it is recommended to clone the repository and set up the working environment within the Linux subsystem. Refer to the guide for setting up WSL2 on [Visual Studio Code](https://code.visualstudio.com/docs/remote/wsl) or [PyCharm](https://www.jetbrains.com/help/pycharm/using-wsl-as-a-remote-interpreter.html#create-wsl-interpreter).
-
-After you git cloned the forked copy of OpenSearch Benchmark, use the following command-line instructions to set up OpenSearch Benchmark for development:
-```
-cd opensearch-benchmark
+```bash
+cd solr-benchmark # (or your fork directory)
make develop
```
-Depending on the platform and shell you have, use the following command to activate the virtual environment:
+Activate the virtual environment:
-| Platform | Shell | Command used to activate the virtual environment |
-| --------------- | --------- | ------------------------------------- |
-| Posix| bash/zsh | source .venv/bin/activate |
-| | fish | source .venv/bin/activate.fish |
-| | csh/tcsh | source .venv/bin/activate.csh |
-| | PowerShell Core | .venv/bin/Activate.ps1 |
-| Windows | cmd.exe| C:\> <venv>\Scripts\activate.bat |
-| | PowerShell | PS C:\> <venv>\Scripts\Activate.ps1 |
+| Platform | Shell | Command |
+|----------|-------|---------|
+| Posix | bash/zsh | `source .venv/bin/activate` |
+| | fish | `source .venv/bin/activate.fish` |
+| | csh/tcsh | `source .venv/bin/activate.csh` |
+| Windows | cmd.exe | `.venv\Scripts\activate.bat` |
+| | PowerShell | `.venv\Scripts\Activate.ps1` |
-For more information regarding activating virtual environments, please see https://docs.python.org/3/library/venv.html.
+## Importing the Project into an IDE
-## Importing the project into an IDE
+The project uses a virtualenv created by `make develop`. In PyCharm:
-OpenSearch Benchmark builds using virtualenv. When importing into an IDE, such as PyCharm IDE, you will need to define an appropriate Python SDK, which is provided by virtualenv.
-Refer to IDE documentation for more details on defining a Python SDK. We recommend using the Python SDK that `make prereq` creates.
-This is typically created in PyCharm IDE by visiting the `Python Interpreter`, selecting either `Virtualenv Environment` or `Existing Environment`, and pointing interpreter to `.venv/bin/python3` within the OpenSearch Benchmark source directory.
-`
-In order to run tests within the PyCharm IDE, ensure the `Python Integrated Tools` / `Testing` / `Default Test Runner` is set to `pytest`.
+1. Go to *Settings → Python Interpreter*.
+2. Select *Existing Environment*.
+3. Point the interpreter to `.venv/bin/python3` inside the repository root.
+4. In *Python Integrated Tools → Testing*, set the default runner to `pytest`.
-## Setting Up a Local OpenSearch Cluster For OSB Development (Optional)
+## Setting Up a Local Solr Instance (Optional)
-### OpenSearch Installation
+Download the latest Solr release from https://solr.apache.org/downloads.html:
-Download the latest release of OpenSearch from https://opensearch.org/downloads.html. If you are using WSL, make sure to download it into your `/home/<user>` directory instead of `/mnt/c`.
-```
-wget https://artifacts.opensearch.org/releases/bundle/opensearch/<x.x.x>/opensearch-<x.x.x>-linux-x64.tar.gz
-tar -xf opensearch-x.x.x-linux-x64.tar.gz
-cd opensearch-x.x.x
+```bash
+wget https://downloads.apache.org/solr/solr/<version>/solr-<version>.tgz
+tar -xf solr-<version>.tgz
+cd solr-<version>
+bin/solr start -c # SolrCloud mode (recommended)
```
-NOTE: Have Docker running in the background for the next steps. Refer to the installation instructions [here](https://docs.docker.com/compose/install/).
-### OpenSearch Cluster setup
+Verify Solr is running:
-Add the following settings to the `opensearch.yml` file under the config directory
+```bash
+curl http://localhost:8983/api/node/system | python3 -m json.tool
```
-vim config/opensearch.yml
-```
-```
-#
-discovery.type: single-node
-plugins.security.disabled: true
-#
-```
-Run the opensearch-tar-install.sh script to install and setup a cluster for our use.
-```
-bash opensearch-tar-install.sh
-```
-Check the output of `curl.exe "http://localhost:9200/_cluster/health?pretty"`. Output should be similar to this:
-```
-{
- "cluster_name" : "<name>",
- "status" : "green",
- "timed_out" : false,
- "number_of_nodes" : 1,
- "number_of_data_nodes" : 1,
- "discovered_master" : true,
- "discovered_cluster_manager" : true,
- "active_primary_shards" : 3,
- "active_shards" : 3,
- "relocating_shards" : 0,
- "initializing_shards" : 0,
- "unassigned_shards" : 0,
- "delayed_unassigned_shards" : 0,
- "number_of_pending_tasks" : 0,
- "number_of_in_flight_fetch" : 0,
- "task_max_waiting_in_queue_millis" : 0,
- "active_shards_percent_as_number" : 100.0
-}
-```
-Now, you have a local cluster running! You can connect to this and run the workload for the next step.
-### Running Workloads on a locally installed Cluster
+### Running a workload against a local Solr cluster
-Here's a sample run of the geonames benchmark which can be found from the [workloads](https://github.com/opensearch-project/opensearch-benchmark-workloads) repo.
-```
-opensearch-benchmark run --pipeline=benchmark-only --workload=geonames --target-host=127.0.0.1:9200 --test-mode --workload-params '{"number_of_shards":"1","number_of_replicas":"0"}'
+```bash
+solr-benchmark execute-test \
+ --pipeline=benchmark-only \
+ --workload=<your-workload> \
+ --target-host=localhost:8983
```
-And we're done! You should be seeing the performance metrics soon enough!
+Logs are written to `~/.benchmark/logs/benchmark.log`.
-### Debugging
+## Running Tests
-**If you are not seeing any results, it should be an indicator that there is an issue with your cluster setup or the way the manager is accessing it**. Use the command below to view the logs.
-```
-tail -f ~/.benchmark/logs/benchmark.log
-```
+### Unit tests
-If you run into below error:
-```json
-[ERROR] ❌ Cannot run. Error in test run orchestrator (No value for mandatory configuration: section='reporting', key='datastore.type')
-```
-This normally happens because you are using the old config cached in `~/.benchmark` and `~/.osb`. You can remove them to force using the latest config.
-```json
-rm -rf ~/.benchmark
-rm -rf ~/.osb
-```
-
-## running tests
-
-Once setup is complete, you may run the unit and integration tests.
-
-### Unit Tests
-Invoke unit tests by running the following command within the root directory of the repository:
-
-```
+```bash
make test
+# or directly:
+python -m pytest tests/unit/solr/ -v
```
-### Integration Tests
+### Integration tests
-Integration tests are expected to run for approximately **20-30 mins** and can be run on the following operating systems:
- * RedHat
- * CentOS
- * Ubuntu (and WSL)
- * Amazon Linux 2
- * MacOS
+Integration tests require a running Solr instance (local or Docker).
-Integration tests run against the standard [OpenSearch Benchmark workloads](https://github.com/opensearch-project/opensearch-benchmark-workloads). Sometimes, it may be necessary to run integration tests against a modified forked copy of these workloads. In that case, please follow the instructions [here](https://github.com/opensearch-project/opensearch-benchmark-workloads/blob/main/README.md#testing-the-workload).
-
-Invoke integration tests by running the following command within the root directory of the repository:
-
-```
+```bash
make it
```
+## Submitting a Pull Request
-## Submitting your changes for a pull request
-
-Once your changes and tests are ready to submit for review:
+1. **Run tests**: `make test` (and `make it` if applicable).
+2. **Sign commits** with DCO: `git commit -s`.
+3. **Rebase** onto the latest `main` before opening a PR.
+4. Open the PR, referencing the related issue (`Closes #123`).
+5. Respond to review comments; squash commits if asked.
-1. Test your changes
-
- Run the test suite to make sure that nothing is broken: `make it`.
-
-2. Sign the Developer Certificate of Origin
+## Developing Breaking Changes
- Please make sure you have signed the DCO certificate. Include the `--signoff` argument as part of your `git commit`
+Develop breaking changes in a dedicated feature branch. Rebase onto `main`
+before the next release and merge at that point.
-3. Rebase your changes
+## Miscellaneous
- Update your local repository with the most recent code from the main OpenSearch Benchmark repository, and rebase your branch on top of the latest master branch. We prefer your initial changes to be squashed into a single commit. Later, if we ask you to make changes, add them as separate commits. This makes them easier to review. As a final step before merging we will either ask you to squash all commits yourself or we'll do it for you.
+### Avoiding secrets in commits
-4. Submit a pull request
+Install [git-secrets](https://github.com/awslabs/git-secrets) to prevent
+accidentally committing credentials:
- Push your local changes to your forked copy of the repository and [submit a pull request](https://help.github.com/articles/using-pull-requests). In the pull request, choose a title which sums up the changes that you have made, and in the body provide more details about what your changes do. Also mention the number of the issue where discussion has taken place, eg "Closes #123".
+```bash
+git clone https://github.com/awslabs/git-secrets.git
+cd git-secrets && make install
+```
-## Developing Breaking Changes
-Breaking changes should not be directly added to the `main` branch. These should be developed in their own feature branch. Prior to a new release this feature branch should be rebased onto the latest changes from `main`. `main` can then `pull` or `cherry-pick` the breaking changes from the feature branch.
+### Developer mode (quick iteration)
-## Miscellaneous
+```bash
+python3 -m pip install -e .
+```
-### git-secrets
-Security is our top priority. Avoid checking in credentials, install awslabs/git-secrets.
+Changes to source files are reflected immediately on the next run.
-```
-git clone https://github.com/awslabs/git-secrets.git
-cd git-secrets
-make install
-```
-### Adding New Major and Minor Python Versions to OpenSearch-Benchmark
-To streamline the process, please refer to [this guide](https://github.com/opensearch-project/opensearch-benchmark/blob/main/PYTHON_SUPPORT_GUIDE.md)
+### Debugging unit tests in Visual Studio Code
-### Debugging OpenSearch Benchmark in Developer Mode
-Many users find that the simplest way to debug OpenSearch Benchmark is by using developer mode. Users can activate developer mode by running `python3 -m pip install -e .` within the cloned OpenSearch Benchmark repository. Any changes made and saved will be reflected when OpenSearch Benchmark is run. Users can add loggers or print statements and see the changes reflected in subsequent runs.
+Add to your `launch.json`:
-### Debugging Unittests in Visual Studio Code
-To run and debug unittests in Visual Studio Code, add the following configuration to the Python Debugger `launch.json` file. See [the official Visual Studio Code documentation](https://code.visualstudio.com/docs/editor/debugging) for more information on setting up and accessing `launch.json` file.
-```
- {
- "name": "Python: Module",
- "type": "python",
- "request": "launch",
- "module": "pytest",
- "args": ["-k ${file}"]
- }
+```json
+{
+ "name": "pytest (current file)",
+ "type": "python",
+ "request": "launch",
+ "module": "pytest",
+ "args": ["-k", "${file}"]
+}
```
-With this, users can easily run and debug unittests within Visual Studio Code without invoking pytest manually on the command line.
diff --git a/PYTHON_SUPPORT_GUIDE.md b/PYTHON_SUPPORT_GUIDE.md
index 7327d524..71aa7780 100644
--- a/PYTHON_SUPPORT_GUIDE.md
+++ b/PYTHON_SUPPORT_GUIDE.md
@@ -1,77 +1,89 @@
# Python Support Guide
-This document walks developers through how to add support for new major & minor Python versions in OpenSearch-Benchmark.
+This document walks developers through how to add support for new major and
+minor Python versions in Apache Solr Benchmark.
-### Update Python versions supported in OpenSearch Benchmark
+## Update Python versions supported
-Make changes to the following files and open a PR titled "Update Python versions supported to `<list of Python versions this PR plans to support>`".
+Make changes to the following files and open a PR titled
+"Update Python versions supported to `<list of versions>`".
* `.ci/variables.json`: Update Python variables and `MIN_PY_VER` as needed.
- * For example: If OSB needs to add support for Python `3.12`, ensure there is a `PY312` variable and make it set to the latest patch release of Python `3.12`, such as `3.12.11`.
-* `.github/workflows/integ-tests.yml`: Update supported Python versions in the `python-versions` section
-* `setup.py`: Update supported Python versions in `supported_python_versions`.
-* `tox.ini`: Update supported Python versions in `env_list`
-* `Makefile`: If updating the minimum supported Python version, ensure that the minimum Python version environment variable, the `pyinst<MINIMUM_SUPPORTED_PYTHON_VERSION>` section, and `check-pip` section have been updated.
- * For example: If changing the minimum supported Python version to Python `3.10`, ensure the following lines have been updated to use 3.10
-```
-VERSION310 = $(shell jq -r '.python_versions | .[]' .ci/variables.json | sed '$$d' | grep 3\.10)
+ For example: to add Python `3.13`, ensure there is a `PY313` variable set
+ to the latest patch release such as `3.13.1`.
+* `.github/workflows/integ-tests.yml`: Update supported Python versions in
+ the `python-versions` section.
+* `setup.py`: Update `supported_python_versions`.
+* `tox.ini`: Update `env_list`.
+* `Makefile`: If updating the minimum supported Python version, update the
+ `pyinst<MIN_VERSION>` target and `check-pip`:
-pyinst310:
- pyenv install --skip-existing $(VERSION310)
- pyenv local $(VERSION310)
+ ```makefile
+ VERSION310 = $(shell jq -r '.python_versions | .[]' .ci/variables.json | sed '$$d' | grep 3\.10)
-check-pip:
- @if ! $(PIP) > /dev/null 2>&1 || ! $(PIP) install pip > /dev/null 2>&1; then make pyinst310; fi
-```
-* `osbenchmark/__init__.py`: If updating the minimum supported Python version, ensure the <MINIMUM_SUPPORTED_PYTHON_VERSION> has been updated in the following error statement:
-```
-raise RuntimeError("OSB requires at least Python <MINIMUM_SUPPORTED_PYTHON_VERSION> but you are using:\n\nPython %s" % str(sys.version))
-```
+ pyinst310:
+ pyenv install --skip-existing $(VERSION310)
+ pyenv local $(VERSION310)
-For an example, please see the reference the following:
-* [Update Python versions supported to 3.10 - 3.13](https://github.com/opensearch-project/opensearch-benchmark/pull/961/files)
+ check-pip:
+ @if ! $(PIP) > /dev/null 2>&1 || ! $(PIP) install pip > /dev/null 2>&1; then make pyinst310; fi
+ ```
-### Testing New Python Versions After Adding to OpenSearch Benchmark
+* `osbenchmark/__init__.py`: Update the minimum version in the error message:
-1. Setup a fresh testing environment for each of the following operating systems: MacOS, Ubuntu, and Amazon Linux 2
-2. Install dependencies, git clone the repository, and install the Python version you are trying to add (whether that’s through pyenv or directly from source). If you are using pyenv, to switch to a Python version, run `pyenv local <PYTHON VERSION>`.
-3. Verify the Python version your python3 alias is pointing to by running `python3 --version`. This should output the Python version you are trying to add support to OpenSearch Benchmark.
-4. Run the following tests:
+ ```python
+ raise RuntimeError("Solr Benchmark requires at least Python <MIN_VERSION> but you are using:\n\nPython %s" % str(sys.version))
+ ```
-**Basic OpenSearch Benchmark command with distribution version and test mode**
-```
-opensearch-benchmark run --distribution-version=1.0.0 --workload=geonames --test-mode
-```
+## Testing new Python versions
-**OpenSearch Benchmark command running test on target-host in test mode**
-```
-opensearch-benchmark run --workload=geonames --pipeline=benchmark-only --target-host="<OPENSEARCH CLUSTER ENDPOINT>" --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'" --test-mode"
-```
+1. Set up a fresh environment on each supported OS: macOS, Ubuntu, Amazon Linux 2.
+2. Install the new Python version (via pyenv or from source). Switch to it:
+ ```bash
+ pyenv local <PYTHON VERSION>
+ python3 --version # confirm
+ ```
-**OpenSearch-Benchmark command running test on target-host without test mode**
+3. Run the following tests:
+
+**Basic run with a local Solr instance (benchmark-only pipeline):**
+```bash
+solr-benchmark run \
+ --pipeline=benchmark-only \
+ --workload=<YOUR_WORKLOAD> \
+ --target-host="localhost:8983" \
+ --test-mode
```
-opensearch-benchmark run --workload=geonames --pipeline=benchmark-only --target-host="<OPENSEARCH CLUSTER ENDPOINT>" --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'"
+
+**Run without test mode:**
+```bash
+solr-benchmark run \
+ --pipeline=benchmark-only \
+ --workload=<YOUR_WORKLOAD> \
+ --target-host="<SOLR HOST:PORT>"
```
-To ensure that users are using the correct python versions, install the repository with `python3 -m pip install -e .` and run `which opensearch-benchmark` to get the path. Pre-append this path to each of the three commands above and re-run them in the command line.
+**Run with distribution provisioning:**
+```bash
+solr-benchmark run \
+ --pipeline=solr-from-distribution \
+ --distribution-version=9.7.0 \
+ --workload=<YOUR_WORKLOAD> \
+ --test-mode
+```
-Keep in mind the file path outputted differs for each operating system and might point to a shell script or a Python script. Depending on which it is, make sure you can invoke it with the proper prefix such as `bash` or `python3`.
+4. To test the installed binary path explicitly:
+ ```bash
+ which solr-benchmark # e.g. /home/user/.pyenv/shims/solr-benchmark
+ bash /home/user/.pyenv/shims/solr-benchmark run --pipeline=benchmark-only ...
+ ```
-- For example: When running `which opensearch-benchmark` on an Ubuntu environment, the commad line outputs `/home/ubuntu/.pyenv/shims/opensearch-benchmark`. On closer inspection, the path points to a shell script. Thus, to invoke OpenSearch Benchmark, pre-=append the OpenSearch Benchmark command with `bash` and the path outputted earlier:
-```
-bash -x /home/ubuntu/.pyenv/shims/opensearch-benchmark run --workload=geonames --pipeline=benchmark-only --target-host="<OPENSEARCH CLUSTER ENDPOINT>" --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'"
-```
+## Creating a pull request
-- Another example: When running `which opensearch-benchmark` on an Amazon Linux 2 environment, the command line outputs `~/.local/bin/opensearch-benchmark`. On closer inspection, the path points to a Python script. Thus, to invoke OpenSearch Benchmark, pre-append the OpenSearch Benchmark command with `python3` and the path outputted earlier:
-```
-python3 ~/.local/bin/opensearch-benchmark run --workload=geonames --pipeline=benchmark-only --target-host="<OPENSEARCH CLUSTER ENDPOINT>" --client-options="basic_auth_user:'<USERNAME>',basic_auth_password:'<PASSWORD>'"
-```
+After testing, open a PR. Once merged, create and push a version tag to
+trigger the release pipeline:
-### Creating a Pull Request After Adding Changes and Testing Them Out
-After adding Python versions and testing them in environments, make a pull request on Github. Once that’s been merged, push a tag and get it approved by a maintainer. After this has been approved, Jenkins will publish to OpenSearch Benchmark’s Pypi account with the new version you updated it to.
-```
-# Create Tag
+```bash
git tag <NEW MAJOR.MINOR.PATCH VERSION> main
-# Push Tag
git push origin <NEW MAJOR.MINOR.PATCH VERSION>
```
diff --git a/README.md b/README.md
index f72e6d38..d4444f5d 100644
--- a/README.md
+++ b/README.md
@@ -1,67 +1,93 @@
-[](https://github.com/opensearch-project/opensearch-benchmark/actions/workflows/main.yml)
-[](https://github.com/opensearch-project/opensearch-benchmark/actions/workflows/manual-integ.yml)
-[](https://github.com/opensearch-project/opensearch-benchmark/actions/workflows/publish-release.yml)
-[](https://pypi.org/project/opensearch-benchmark/)
-
-[](https://forum.opensearch.org/categories)
-
+# Apache Solr Benchmark
-<img src="https://github.com/opensearch-project/opensearch-benchmark/blob/main/opensearch_benchmark.png?raw=true" height="64px" alt="OpenSearch Benchmark">
+Apache Solr Benchmark is a macrobenchmarking framework for [Apache Solr](https://solr.apache.org/).
-OpenSearch Benchmark is the macrobenchmarking framework for OpenSearch.
+It is a fork/port of [Rally](https://github.com/elastic/rally)/[Opensearch Benchmark](https://github.com/opensearch-project/opensearch-benchmark), ported to work with Apache Solr.
-What is OpenSearch Benchmark?
------------------------------
+## Documentation
-If you are looking to performance test OpenSearch, then OpenSearch Benchmark is for you. It can help you with the following tasks:
+Full documentation is available in [`docs/`](docs/) folder of this repository. Build the docs with jekyll.
+A public documentation site is available at [https://janhoy.github.io/solr-benchmark/](https://janhoy.github.io/solr-benchmark/).
+
+**This is a Work in Progress**
+
+## What is Apache Solr Benchmark?
+
+If you are looking to performance test Apache Solr, this tool can help you with:
* Running performance benchmarks and recording results
-* Setting up and tearing down OpenSearch clusters for benchmarking
-* Managing benchmark data and specifications across OpenSearch versions
-* Discovering performance problems by attaching so-called telemetry devices
-* Comparing performance results
-* Creating customized workloads
+* Setting up and tearing down Solr clusters for benchmarking (local distribution, build-from-source or Docker, including nightly builds)
+* Managing benchmark workloads (collections, configsets, search operations)
+* Run same workload against multiple Solr versions or multiple cluster-configurations (heap size, GC settings, etc.)
+* Collecting JVM, node, and collection metrics via telemetry devices
+* Output results for each run in JSON format, suitable for analysis and dashboarding
+* Assist in converting existing OpenSearch Benchmark workloads to Solr format
+
+## Quick Start
+
+### Install
+
+**NOTE**: We do not offer the tool as a python package yet
+
+```bash
+pip install -e .
+```
+
+### Run a benchmark against a Solr version in Docker
+
+```bash
+solr-benchmark run \
+ --pipeline=docker \
+ --distribution-version=9.10.1 \
+ --workload=nyc_taxis \
+ --test-mode
+```
-We have also put considerable effort into OpenSearch Benchmark to ensure that benchmarking data are reproducible.
+**Note**: Defaults to cloud mode (SolrCloud with embedded ZooKeeper).
-Documentation
--------------
+### Provision Solr locally, then benchmark
-Official documentation for OpenSearch Benchmark is [available online](https://opensearch.org/docs/latest/benchmark/index/).
+```bash
+solr-benchmark run \
+ --pipeline=from-distribution \
+ --distribution-version=9.10.1 \
+ --workload=nyc_taxis \
+ --test-mode
+```
-Quick Start
------------
+**Note**: Always uses cloud mode (SolrCloud with embedded ZooKeeper).
-Want to get started with OpenSearch Benchmark quickly? See [OpenSearch Benchmark's Quick Start guide in the documentation](https://opensearch.org/docs/latest/benchmark/index/).
+### Provision Solr from source, then benchmark
-Getting help
-------------
+```bash
+solr-benchmark run \
+ --pipeline=from-sources \
+ --distribution-version=9.10.1 \
+ --workload=nyc_taxis \
+ --test-mode
+```
-* Quick help: ``opensearch-benchmark --help``
-* Want to contribute? Look at [OpenSearch Benchmark's Developer Guide](<https://github.com/opensearch-project/OpenSearch-Benchmark/blob/main/DEVELOPER_GUIDE.md>) for more information
-* For any questions or answers, visit our [community forum](<https://discuss.opendistrocommunity.dev/>) and Slack community channel [#performance-benchmarking](https://opensearch.slack.com/archives/C082PLA3VPW).
-* File improvements or bug reports in our [Github repo](<https://github.com/opensearch-project/OpenSearch-Benchmark/issues>).
+## Workload format
-How to Contribute
------------------
+See [Workload Reference](https://janhoy.github.io/solr-benchmark/reference/workloads/) in the documentation for the full `workload.json` format, including `collections`, `corpora`, `operations`, and `test-procedures`.
-See all details in the [contributor guidelines](<https://github.com/opensearch-project/OpenSearch-Benchmark/blob/main/CONTRIBUTING.md>).
+Pre-built workloads are available at [https://github.com/janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads). Feel free to
+contribute your own with a pull request!
-License
--------
+## Result output
-This software is licensed under the Apache License, version 2 ("ALv2"), quoted below.
+Each test-run outputs a **test_run.json**, a complete canonical record of the benchmark run including:
+ - Benchmark metadata (version, environment, pipeline, user tags)
+ - Workload and test procedure information
+ - Cluster configuration specification (heap size, GC settings, all variables)
+ - Detailed operation metrics (throughput, latency, error rates)
+ - System metrics (GC times, merge times, segment counts, etc.)
-Copyright 2015-2022 OpenSearch <https://opensearch.org/>
+This output can be used for further analysis, comparison and dashboarding.
-Licensed under the Apache License, Version 2.0 (the "License"); you may not
-use this file except in compliance with the License. You may obtain a copy of
-the License at
+## License
- http://www.apache.org/licenses/LICENSE-2.0
+Apache License, Version 2.0. See [LICENSE](LICENSE) for the full text.
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
-WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
-License for the specific language governing permissions and limitations under
-the License.
\ No newline at end of file
+This product includes software developed by the OpenSearch Contributors, and
+prior to that by Elasticsearch (Rally). Full attribution is in [NOTICE](NOTICE).
diff --git a/TODO.md b/TODO.md
new file mode 100644
index 00000000..7238d35a
--- /dev/null
+++ b/TODO.md
@@ -0,0 +1,96 @@
+# Solr Benchmark TODO List
+
+## Testing
+
+- [ ] Test telemetry with multi-node SolrCloud clusters
+- [ ] Add integration tests for full workload execution
+- [ ] Faceting/aggregation result validation
+
+## ASF Donation
+
+Steps to donate this repository to the Apache Software Foundation under the Solr PMC:
+
+- [ ] Hold a procedural vote on the Solr dev@ mailing list to adopt Solr Orbit as an Apache Solr
+ sub-project
+- [ ] Fill in and submit the [ASF IP Clearance form](https://incubator.apache.org/ip-clearance/)
+ to the incubator (lazy consensus; conducted by the Solr PMC)
+- [ ] Order the new `apache/solr-orbit*` repositories via the Apache self-service repo tooling
+- [ ] Import the codebase — starting from the last OpenSearch Benchmark commit that was forked —
+ into the new repos, establishing a clean starting point in Apache git history
+- [ ] Request a trademark check from trademarks@apache.org for the "Solr Orbit" name (not
+ required by policy, but advisable before public announcement)
+- [ ] Open one PR per `apache/solr-orbit*` repo carrying all porting work accumulated in this
+ repo (telemetry, Solr client, workloads, docs, CI, etc.)
+
+## Documentation
+
+- [ ] Decide on where to host the public documentation after moving to the `apache/` GitHub
+ organisation — GitHub Pages is already set up; confirm it stays or migrate — and update all
+ internal and external links accordingly
+- [ ] Document telemetry usage and configuration in DEVELOPER_GUIDE.md
+- [ ] Document the SolrClient pattern in DEVELOPER_GUIDE.md
+- [ ] Create troubleshooting guide
+- [ ] Add examples of custom native Solr workloads
+
+## Telemetry
+
+- [ ] SolrShardStats (per-shard statistics)
+- [ ] SolrReplicationStats (replication lag tracking)
+- [ ] SolrSegmentStats (detailed segment breakdown including deleted docs)
+- [ ] StartupTime device (node startup duration tracking)
+- [ ] DiskIo device (OS-level disk stats)
+- [ ] Heapdump on demand (telemetry device)
+
+## Workload
+
+- [ ] Improve schema auto-generation for complex field types
+- [ ] Support for nested documents (child docs in Solr)
+- [ ] Support for Solr's streaming expressions
+- [ ] Support for Solr SQL queries
+
+## Codebase Hygiene
+
+- [ ] Rename the product from "Solr Benchmark" to "Solr Orbit" after the porting PRs land in the
+ new Apache repos (step 7 of the ASF donation process) — touches the project name, CLI entry
+ points (`solr-benchmark` → `solr-orbit`), `setup.py`, README, all docs, and PyPI package name
+- [ ] Rename the `osbenchmark` package and folder to match the new product name — the current
+ name is an artefact of the OpenSearch Benchmark fork; touches imports across the entire
+ codebase, entry points in `setup.py`, and all references in docs and CI scripts
+
+## Metrics Store
+
+- [ ] Implement native Solr metrics store (currently only local filesystem JSON/CSV is used)
+
+## Performance
+
+- [ ] Profile memory usage with large workloads
+
+## Release
+
+- [ ] Publish the tool to PyPI as `solr-benchmark` (or `solr-orbit` once renamed) so users can
+ install with `pip install solr-benchmark` rather than cloning the repo; requires a PyPI
+ project, a CI publish workflow triggered on tags, and a `twine`/`build` release step
+ (tooling already present in `develop_require`)
+- [ ] Author a release guide documenting the end-to-end release process; evaluate
+ [Apache Testing and Release (ATR)](https://release-test.apache.org/docs/) as the
+ release tooling — ATR is the ASF's modern replacement for the manual svn-based release
+ process and handles signing, staging, and voting workflows
+
+## ASF Compliance
+
+- [ ] Replace pylint with `ruff` for linting and formatting — Apache Airflow uses ruff as its
+ primary tool (`[tool.ruff]` in `pyproject.toml`); much faster, broader rule coverage,
+ and handles formatting (replacing black/isort) in one tool
+- [ ] Add `pre-commit` framework with `insert-license` hook (Lucas-C/pre-commit-hooks) to
+ enforce the SPDX license header on every source file at commit time — Apache Airflow
+ uses this to cover `.py`, `.sh`, `.yaml`, `.xml`, `.md` and more via per-type templates
+ in `scripts/ci/license-templates/`
+- [ ] Add Apache RAT with a `.rat-excludes` exclusion list for release-time license audit —
+ Apache Airflow maintains ~334 exclusion patterns; RAT is required by ASF release policy
+ and is already available since JDK 21 is a prerequisite
+- [ ] Add `liccheck` with an authorised-license allowlist in `setup.cfg` to gate CI on
+ dependency license compliance — Apache Superset uses liccheck to flag GPL/LGPL
+ transitive dependencies automatically; will also catch certifi (MPL-2.0, Category B)
+- [ ] Add `pip-licenses` target (`make licenses`) to regenerate NOTICE / THIRD_PARTY
+ attribution from installed packages — ensures the attribution file stays current as
+ dependencies change
diff --git a/docker/README.md b/docker/README.md
index aa9d0e66..f78830d6 100644
--- a/docker/README.md
+++ b/docker/README.md
@@ -1,42 +1,42 @@
-# OpenSearch Benchmark Docker Image
+# Apache Solr Benchmark Docker Image
-This Docker image allows users to spin up a Docker container preloaded with essential dependencies and get started with OpenSearch Benchmark faster.
+This Docker image provides a container with all dependencies pre-installed so you can run [Apache Solr Benchmark](https://github.com/janhoy/solr-benchmark) without a local Python setup.
-# Running the OpenSearch Benchmark Image
-**Prerequisite:** Ensure that Docker is installed. If not, refer to [this guide to download Docker Desktop](https://docs.docker.com/get-docker/) or [this guide to download Docker Engine](https://docs.docker.com/engine/install/).
+**Prerequisite:** [Docker Desktop](https://docs.docker.com/get-docker/) or [Docker Engine](https://docs.docker.com/engine/install/) must be installed.
-To run the image in a Docker container, invoke the following command:
-```
-docker run opensearchproject/opensearch-benchmark [OSB ARGS]
-```
+## Running the Image
-For instance, using `-h` for the arguments will print the OSB help information. Once the OSB process completes, the Docker container is automatically terminated.
+```bash
+docker run janhoy/solr-benchmark [ARGS]
+```
-To run in interactive mode, run `docker run --entrypoint bash -it opensearchproject/opensearch-benchmark -c /bin/bash`. This will place you into a shell to interact with the container where you can invoke opensearch-benchmark with any desired subcommands or options. When you are finished, exit from the shell to terminate the container.
+For example, passing `-h` prints the help text. The container exits automatically when the process finishes.
+To run interactively:
-The 0.4.1 release of OSB includes a Beta feature that permits increasing the size of the data corpus associated with the _http_logs_ workload. For more details, run the command below:
-```
-docker run -it --entrypoint expand-data-corpus.py opensearchstaging/opensearch-benchmark:0.4.1 -h
+```bash
+docker run --entrypoint bash -it janhoy/solr-benchmark -c /bin/bash
```
+This drops you into a shell inside the container where you can invoke `solr-benchmark` with any subcommand or option. Exit the shell to terminate the container.
-# Building a Copy of OpenSearch Benchmark Image
-1. Git clone OpenSearch Benchmark Github repository
-2. Make changes locally
-3. Run `docker build --build-arg VERSION=<PYPI VERSION> -t <docker image name>:<optional tag> -f ./docker/Dockerfile .`
+## Building a Local Image
-Note: `Dockerfile` uses the argument `VERSION` to install a specific version of OSB that exists on PyPi.
+1. Clone the repository:
+ ```bash
+ git clone https://github.com/janhoy/solr-benchmark.git
+ cd solr-benchmark
+ ```
-### Difference between Dockerfile and Dockerfile-development
-The build command in step 3 uses `Dockerfile`, which installs OSB from Pypi. However, if users want an image with OSB installed from the source code, replace `Dockerfile` in step 3's command with `Dockerfile-development`. When using `Dockerfile-development`, users can omit the `--build-arg VERSION=<PYPI VERSION>` since this argument is not used when installing OSB from source code.
+2. Make any local changes.
+3. Build the image:
+ ```bash
+ docker build --build-arg VERSION=<version> -t solr-benchmark:<tag> -f docker/Dockerfile .
+ ```
-# Publishing a New Version of OpenSearch Benchmark Image
-After updating [version.txt](../version.txt) with the newest major, minor, and patch version and before publishing a tag to the repository, ensure that [docker.yml](../.github/workflows/docker.yml) has the tags updated. The last tag in the list should be updated to the new major, minor, and patch version found in [version.txt](../version.txt).
-```
-tags: opensearchstaging/opensearch-benchmark:latest,opensearchstaging/opensearch-benchmark:<MAJOR.MINOR.PATCH VERSION FROM VERSION.TXT>
-```
+ The `VERSION` build argument is embedded in the image labels. You can set it to any string that identifies your build (e.g. `dev` or a semantic version).
+
+## Dockerfile Best Practices
-# Applying Best Practices for Building Dockerfiles
-Follow the best practices guidelines found [here](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) when updating OpenSearch Benchmark's Dockerfile.
\ No newline at end of file
+Follow the [Dockerfile best practices guide](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/) when modifying the `Dockerfile`.
diff --git a/docs/.gitignore b/docs/.gitignore
new file mode 100644
index 00000000..75391cf5
--- /dev/null
+++ b/docs/.gitignore
@@ -0,0 +1,6 @@
+_site/
+.jekyll-cache/
+.jekyll-metadata
+.bundle/
+vendor/
+Gemfile.lock
diff --git a/docs/.ruby-version b/docs/.ruby-version
new file mode 100644
index 00000000..15a27998
--- /dev/null
+++ b/docs/.ruby-version
@@ -0,0 +1 @@
+3.3.0
diff --git a/docs/Gemfile b/docs/Gemfile
new file mode 100644
index 00000000..b3fbcd5e
--- /dev/null
+++ b/docs/Gemfile
@@ -0,0 +1,4 @@
+source "https://rubygems.org"
+
+gem "jekyll", "~> 4.4.1"
+gem "just-the-docs", "0.12.0"
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 00000000..dfa579dd
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,139 @@
+# Documentation Maintenance Guide
+
+This directory contains the source for the [Apache Solr Benchmark documentation site](https://janhoy.github.io/solr-benchmark/),
+built with [Jekyll](https://jekyllrb.com/) and the [just-the-docs](https://just-the-docs.com/) theme.
+
+## Prerequisites
+
+- **Ruby 3.3+** — The system Ruby on macOS is often too old. Use [rbenv](https://github.com/rbenv/rbenv) or [RVM](https://rvm.io/):
+ ```
+ rbenv install 3.3.0
+ rbenv local 3.3.0 # sets .ruby-version in docs/
+ ```
+- **Bundler** — comes with modern Ruby: `gem install bundler`
+
+## First-time Setup
+
+```bash
+cd docs
+bundle install
+```
+
+This installs Jekyll 4.4.1 and the just-the-docs 0.12.0 gem into the local bundle.
+
+## Local Development
+
+Start a live-reload development server:
+
+```bash
+cd docs
+bundle exec jekyll serve
+```
+
+The site will be available at `http://localhost:4000/`. Jekyll watches for file changes and rebuilds automatically.
+
+To verify the build before pushing:
+
+```bash
+bundle exec jekyll build
+```
+
+The generated site is written to `docs/_site/` (git-ignored).
+
+### Sass deprecation warnings
+
+You will see many warnings like:
+
+```
+DEPRECATION WARNING [import]: Sass @import rules are deprecated and will be removed in Dart Sass 3.0.0.
+```
+
+These come from the just-the-docs theme gem's own SCSS files and are **harmless** — the CSS compiles correctly. They are a known issue upstream and can be ignored until just-the-docs migrates its SCSS from `@import` to `@use`.
+
+## Adding or Editing Pages
+
+Each page is a Markdown file with a YAML front matter block. The minimal required front matter is:
+
+```yaml
+---
+title: My Page Title
+nav_order: 10
+---
+```
+
+For pages nested under a section, add `parent` and optionally `grand_parent`:
+
+```yaml
+---
+title: Running a Workload
+parent: Working with Workloads
+grand_parent: User Guide
+nav_order: 9
+---
+```
+
+To create a section that has child pages, add `has_children: true` to its `index.md`.
+
+Navigation order is controlled by `nav_order` — lower numbers appear first in the sidebar.
+
+### Internal links
+
+Jekyll generates `.html` files (not directory-style pretty URLs), so internal cross-page links must use the `.html` extension:
+
+```markdown
+See [Available Configs](available-configs.html) for details.
+See [Telemetry Devices](../../reference/telemetry.html) for full documentation.
+```
+
+Do **not** use `.md` extensions in links — they resolve to the source file, not the built page.
+
+### Important: Jinja2 / Liquid Escaping
+
+Jekyll processes Liquid template tags (`{{ }}` and `{% %}`) in all Markdown files, including
+inside fenced code blocks. When documenting workload files that use Jinja2 syntax, wrap the
+code block with `{% raw %}` and `{% endraw %}`:
+
+````
+{% raw %}
+```json
+{ "bulk_size": {{ bulk_size | default(500) }} }
+```
+{% endraw %}
+````
+
+## Site Structure
+
+```
+docs/
+├── _config.yml ← Site title, theme, nav settings
+├── Gemfile ← Ruby gem dependencies
+├── index.md ← Home page
+├── quickstart.md
+├── glossary.md
+├── faq.md
+├── about.md ← License, attribution, trademarks
+├── _includes/
+│ └── footer_custom.html ← ASF copyright footer (applied to every page)
+├── user-guide/
+├── reference/
+├── cluster-config/
+└── converter/
+```
+
+## Publishing
+
+The documentation is published automatically to **GitHub Pages** via the GitHub Actions workflow
+at `.github/workflows/docs.yml`.
+
+**Trigger**: every push to the `main` branch.
+
+**Steps in the workflow**:
+1. Check out the repository.
+2. Install Ruby 3.3 and run `bundle install` (gems are cached between runs).
+3. `actions/configure-pages` injects the correct `baseurl` for GitHub Pages.
+4. `bundle exec jekyll build` generates the static site.
+5. The `docs/_site/` directory is uploaded as a Pages artifact and deployed.
+
+The live site URL is: **https://janhoy.github.io/solr-benchmark/**
+
+No manual deployment step is needed — merge to `main` and the site updates within a few minutes.
diff --git a/docs/_config.yml b/docs/_config.yml
new file mode 100644
index 00000000..f9ac9642
--- /dev/null
+++ b/docs/_config.yml
@@ -0,0 +1,75 @@
+title: Apache Solr Benchmark
+description: >-
+ A performance benchmarking tool for Apache Solr clusters,
+ derived from OpenSearch Benchmark.
+
+theme: just-the-docs
+
+url: https://janhoy.github.io
+# baseurl is injected automatically by actions/configure-pages during CI
+
+aux_links:
+ "GitHub": https://github.com/janhoy/solr-benchmark
+ "Apache Solr": https://solr.apache.org
+ "Apache Software Foundation": https://www.apache.org
+
+aux_links_new_tab: true
+
+search_enabled: true
+
+search:
+ heading_level: 2
+ previews: 3
+ preview_words_before: 5
+ preview_words_after: 10
+
+# Callout blocks
+callouts:
+ note:
+ title: Note
+ color: blue
+ warning:
+ title: Warning
+ color: yellow
+ important:
+ title: Important
+ color: red
+
+# Back to top link
+back_to_top: true
+back_to_top_text: "Back to top"
+
+# Footer
+last_edit_timestamp: true
+last_edit_time_format: "%b %e %Y at %I:%M %p"
+
+# Heading anchors
+heading_anchors: true
+
+# External navigation links
+nav_external_links:
+ - title: Apache Solr
+ url: https://solr.apache.org
+ - title: ASF
+ url: https://www.apache.org
+
+# Color scheme (light by default)
+color_scheme: light
+
+# Apply the just-the-docs layout to every page by default
+defaults:
+ - scope:
+ path: ""
+ values:
+ layout: "default"
+
+# Suppress @import deprecation warnings from the just-the-docs gem's SCSS
+sass:
+ quiet_deps: true
+
+# Exclude from processing
+exclude:
+ - Gemfile
+ - Gemfile.lock
+ - .gitignore
+ - README.md
diff --git a/docs/_includes/footer_custom.html b/docs/_includes/footer_custom.html
new file mode 100644
index 00000000..a5637d42
--- /dev/null
+++ b/docs/_includes/footer_custom.html
@@ -0,0 +1,9 @@
+<p>
+ Copyright © 2026 <a href="https://www.apache.org/">The Apache Software Foundation</a>.
+ Licensed under the <a href="https://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+</p>
+<p>
+ Apache, Apache Solr, and the Apache logo are trademarks of
+ The Apache Software Foundation.
+ See <a href="{{ '/about' | relative_url }}">About / Credits</a> for full attribution and trademark notices.
+</p>
diff --git a/docs/about.md b/docs/about.md
new file mode 100644
index 00000000..3f1bf1ab
--- /dev/null
+++ b/docs/about.md
@@ -0,0 +1,31 @@
+---
+title: About / Credits
+nav_order: 102
+---
+
+# About / Credits
+
+## License
+
+Apache Solr Benchmark is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
+
+## Attribution
+
+Apache Solr Benchmark is derived from [OpenSearch Benchmark](https://github.com/opensearch-project/opensearch-benchmark) (Copyright 2022 OpenSearch Contributors, licensed under Apache 2.0), which in turn derives from [Elasticsearch Rally](https://github.com/elastic/rally) (Copyright Elasticsearch bv, licensed under Apache 2.0).
+
+Full attribution details are in the [NOTICE](https://github.com/janhoy/solr-benchmark/blob/main/NOTICE) file in the source repository.
+
+## Trademarks
+
+Apache, Apache Solr, and the Apache feather logo are trademarks of [The Apache Software Foundation](https://www.apache.org/).
+
+OpenSearch® is a registered trademark of the OpenSearch Software Foundation.
+
+Elasticsearch is a trademark of Elasticsearch bv.
+
+## Links
+
+- [The Apache Software Foundation](https://www.apache.org/)
+- [Apache Solr](https://solr.apache.org)
+- [OpenSearch Benchmark](https://github.com/opensearch-project/opensearch-benchmark) (upstream project)
+- [Elasticsearch Rally](https://github.com/elastic/rally) (upstream project)
diff --git a/docs/api/client-options.md b/docs/api/client-options.md
deleted file mode 100644
index cc439be6..00000000
--- a/docs/api/client-options.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Client Options
-parent: OSB API
-nav_order: 12
----
\ No newline at end of file
diff --git a/docs/api/index.md b/docs/api/index.md
deleted file mode 100644
index e28f4c08..00000000
--- a/docs/api/index.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: OSB API
-nav_order: 2
-has_children: true
----
\ No newline at end of file
diff --git a/docs/api/kill-running-process.md b/docs/api/kill-running-process.md
deleted file mode 100644
index 287dea3d..00000000
--- a/docs/api/kill-running-process.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Kill Running Process
-parent: OSB API
-nav_order: 14
----
\ No newline at end of file
diff --git a/docs/api/pipeline.md b/docs/api/pipeline.md
deleted file mode 100644
index 36077cc4..00000000
--- a/docs/api/pipeline.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Pipeline
-parent: OSB API
-nav_order: 11
----
\ No newline at end of file
diff --git a/docs/api/run-test.md b/docs/api/run-test.md
deleted file mode 100644
index 338bd136..00000000
--- a/docs/api/run-test.md
+++ /dev/null
@@ -1,102 +0,0 @@
----
-layout: default
-title: Run Test
-parent: OSB API
-nav_order: 10
----
-
-The `run` command of OpenSearch Benchmark runs tests against your OpenSearch cluster.
-
-
-## Syntax
-
-```bash
-opensearch-benchmark run <arguments>
-```
-
-
-### Common Arguments
-
-See [All Arguments](#all-arguments) for an exhaustive list of arguments
-
-Argument | Description | Required
-:--- | :--- |:---
-`workload` | The dataset and operations that run during a test. See [OpenSearch Benchmark Workloads repository](https://github.com/opensearch-project/opensearch-benchmark-workloads) for more details on workloads. | Yes
-`workload-params` | Parameters defined within each workload that can be overwritten. These parameters are outlined in the README of each workload. You can find an example of the parameters for the eventdata workload [here](https://github.com/opensearch-project/opensearch-benchmark-workloads/tree/main/eventdata#parameters). | No
-`test_procedure` | Test Procedures define the sequence of operations and parameters for a specific workload. When no `test_procedure` is specified, OSB selects the default for the workload. You can find an example test procedure [here](https://github.com/opensearch-project/opensearch-benchmark-workloads/blob/main/eventdata/test_procedures/default.json). | No
-`client-options` | Options for the [OpenSearch Python client](https://opensearch.org/docs/latest/clients/python/). Required if testing against a cluster with security enabled. | No
-`pipeline` | Steps required to run a test, including provisioning an OpenSearch from source code or a specified distribution. Defaults to `from-sources` which provisions an OpenSearch cluster from source code. | No
-`distribution-version` | The OpenSearch version to use for a given test. Defining a version can be useful when using a `pipeline` that includes provisioning. When using a `pipeline` without provisioning, OSB will automatically determine the version | No
-`target-hosts` | The OpenSearch endpoint(s) to run a test against. This should only be specified with `--pipeline=benchmark-only` | No
-`test-mode` | Run a single iteration of each operation in the test procedure. The test provides a quick way for sanity checking a testing configuration. Therefore, do not use `test-mode` for actual benchmarking. | No
-`kill-running-processes` | Kill any running OpenSearch Benchmark processes on the local machine before the test runs. | No
-
-*Example 1*
-
-```
-opensearch-benchmark run --workload eventdata --test-mode
-```
-
-Provision an OpenSearch node on the local machine based on the latest source code in Github and run the `eventdata` workload in test mode.
-
-*Example 2*
-
-```
-opensearch-benchmark run --workload http_logs --pipeline benchmark-only --target-hosts <endpoint> --workload-params "bulk_indexing_clients:1,ingest_percentage:10"
-```
-
-Run the `http_logs` workload against an existing OpenSearch cluster but only use one client for indexing and only ingest 10% of the total data corpus.
-
-*Example 3*
-
-```
-opensearch-benchmark run --workload nyc_taxis --pipeline benchmark-only --target-hosts <endpoint> --client-options "verify_certs:false,use_ssl:true,basic_auth_user:admin,basic_auth_password:admin"
-```
-
-Run the `nyc_taxis` workload against an existing OpenSearch cluster with the security plugin enabled.
-
-
-### All Arguments
-
-Argument | Description | Required
-:--- | :--- |:---
-`distribution-version` | Define the version of the OpenSearch distribution to download. Check https://opensearch.org/docs/version-history/ for released versions. | No
-`cluster-config-path` | Define the path to the cluster-configs and plugin configurations to use. | No
-`cluster-config-repository` | Define repository from where OSB will load cluster-configs (default: `default`). | No
-`cluster-config-revision` | Define a specific revision in the cluster-config repository that OSB should use. | No
-`test-run-id` | Define a unique id for this test_run. | No
-`pipeline` | Select the pipeline to run. | No
-`revision` | Define the source code revision for building the benchmark candidate. 'current' uses the source tree as is, 'latest' fetches the latest version on main. It is also possible to specify a commit id or an ISO timestamp. The timestamp must be specified as: "@ts" where "ts" must be a valid ISO 8601 timestamp, e.g. "@2013-07-27T10:37:00Z" (default: `current`). | No
-`workload-repository` | Define the repository from where OSB will load workloads (default: `default`). | No
-`workload-path` | Define the path to a workload. | No
-`workload-revision` | Define a specific revision in the workload repository that OSB should use. | No
-`workload` | Define the workload to use. List possible workloads with `opensearch-benchmark list workloads`. | No
-`workload-params` | Define a comma-separated list of key:value pairs that are injected verbatim to the workload as variables. | No
-`test-procedure` | Define the test_procedure to use. List possible test_procedures for workloads with `opensearch-benchmark list workloads`. | No
-`cluster-config` | Define the cluster-config to use. List possible cluster-configs with `opensearch-benchmark list cluster-configs` (default: `defaults`). | No
-`cluster-config-params` | Define a comma-separated list of key:value pairs that are injected verbatim as variables for the cluster-config. | No
-`runtime-jdk` | The major version of the runtime JDK to use. | No
-`opensearch-plugins` | Define the OpenSearch plugins to install. (default: install no plugins). | No
-`plugin-params` | Define a comma-separated list of key:value pairs that are injected verbatim to all plugins as variables. | No
-`target-hosts` | Define a comma-separated list of host:port pairs which should be targeted if using the pipeline 'benchmark-only' (default: `localhost:9200`). | No
-`worker-ips` | Define a comma-separated list of hosts which should generate load (default: `localhost`). | No
-`client-options` | Define a comma-separated list of client options to use. The options will be passed to the OpenSearch Python client (default: `timeout:60`). | No
-`on-error` | Controls how OSB behaves on response errors. Options are `continue` and `abort` (default: `continue`). | No
-`telemetry` | Enable the provided telemetry devices, provided as a comma-separated list. List possible telemetry devices with `opensearch-benchmark list telemetry`. | No
-`telemetry-params` | Define a comma-separated list of key:value pairs that are injected verbatim to the telemetry devices as parameters. | No
-`distribution-repository` | Define the repository from where the OpenSearch distribution should be downloaded (default: `release`). | No
-`include-tasks` | Defines a comma-separated list of tasks to run. By default all tasks of a test_procedure are run. | No
-`exclude-tasks` | Defines a comma-separated list of tasks not to run. By default all tasks of a test_procedure are run. | No
-`user-tag` | Define a user-specific key-value pair (separated by ':'). It is added to each metric record as meta info. Example: intention:baseline-ticket-12345 | No
-`results-format` | Define the output format for the command line results. Options are `markdown` and `csv` (default: `markdown`). | No
-`results-numbers-align` | Define the output column number alignment for the command line results. Options are `right`, `center`, `left` and `decimal` (default: right). | No
-`show-in-results` | Define which values are shown in the summary publish. Options are `available`, `all-percentiles` and `all` (default: `available`). | No
-`results-file` | Write the command line results also to the provided file. | No
-`preserve-install` | Keep the benchmark candidate and its index. (default: false). | No
-`test-mode` | Runs the given workload in 'test mode'. Meant to check a workload for errors but not for real benchmarks (default: false). | No
-`enable-worker-coordinator-profiling` | Enables a profiler for analyzing the performance of calls in OSB's worker coordinator (default: false). | No
-`enable-assertions` | Enables assertion checks for tasks (default: false). | No
-`kill-running-processes` | If any processes is running, it is going to kill them and allow OSB to continue to run. | No
-`quiet` | Suppress as much as output as possible (default: false). | No
-`offline` | Assume that OSB has no connection to the Internet (default: false). | No
-
diff --git a/docs/api/target-hosts.md b/docs/api/target-hosts.md
deleted file mode 100644
index 905fc7a7..00000000
--- a/docs/api/target-hosts.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Target Hosts
-parent: OSB API
-nav_order: 15
----
\ No newline at end of file
diff --git a/docs/api/test-mode.md b/docs/api/test-mode.md
deleted file mode 100644
index c781aab3..00000000
--- a/docs/api/test-mode.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Test Mode
-parent: OSB API
-nav_order: 13
----
\ No newline at end of file
diff --git a/docs/api/workload.md b/docs/api/workload.md
deleted file mode 100644
index 9d127882..00000000
--- a/docs/api/workload.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Run Test
-parent: OSB API
-nav_order: 16
----
\ No newline at end of file
diff --git a/docs/cluster-config/available-configs.md b/docs/cluster-config/available-configs.md
new file mode 100644
index 00000000..ff16ab80
--- /dev/null
+++ b/docs/cluster-config/available-configs.md
@@ -0,0 +1,214 @@
+---
+title: Available Configs
+parent: Cluster Config
+nav_order: 2
+---
+
+# Available Cluster Configs
+
+## defaults
+
+Sets the Java heap to 1 GB (`heap_size=1g`). This is the baseline configuration that all other heap-size configs extend.
+
+```
+heap_size: 1g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config defaults ...
+```
+
+---
+
+## 1gheap
+
+Sets the Solr JVM heap to 1 GB. Suitable for small workloads and testing.
+
+```
+heap_size: 1g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config 1gheap ...
+```
+
+---
+
+## 2gheap
+
+Sets the Solr JVM heap to 2 GB.
+
+```
+heap_size: 2g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config 2gheap ...
+```
+
+---
+
+## 4gheap
+
+Sets the Solr JVM heap to 4 GB. Suitable for larger workloads.
+
+```
+heap_size: 4g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config 4gheap ...
+```
+
+---
+
+## 8gheap
+
+Sets the Solr JVM heap to 8 GB.
+
+```
+heap_size: 8g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config 8gheap ...
+```
+
+---
+
+## 16gheap
+
+Sets the Solr JVM heap to 16 GB.
+
+```
+heap_size: 16g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config 16gheap ...
+```
+
+---
+
+## 24gheap
+
+Sets the Solr JVM heap to 24 GB.
+
+```
+heap_size: 24g
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config 24gheap ...
+```
+
+---
+
+## g1gc
+
+Enables the G1 garbage collector. Recommended for latency-sensitive benchmarks.
+
+```
+use_g1_gc: true
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config g1gc ...
+```
+
+---
+
+## parallelgc
+
+Enables the Parallel (throughput-optimized) garbage collector.
+
+```
+use_cms_gc: false
+use_g1_gc: false
+```
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config parallelgc ...
+```
+
+Note: `parallelgc` is available in the `main` cluster config bundle but not in `1.0`.
+
+---
+
+## vanilla
+
+The base cluster configuration. All other heap and GC configs extend `vanilla`. Use this when you want to run Solr with no heap or GC overrides beyond ASB defaults.
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config vanilla ...
+```
+
+---
+
+## ea
+
+Mixin that enables Java assertions (`-ea`). Useful for debugging workload runs.
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config ea ...
+```
+
+---
+
+## fp
+
+Mixin that preserves JVM frame pointers. Required for accurate async-profiler CPU profiles.
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config fp ...
+```
+
+---
+
+## debug-non-safepoints
+
+Mixin that enables more accurate CPU profiling by recording non-safepoint debug information.
+
+**Usage:**
+```bash
+solr-benchmark run --cluster-config debug-non-safepoints ...
+```
+
+---
+
+## Comparing configs
+
+To compare G1GC vs Parallel GC on the same workload:
+
+```bash
+# Run 1: G1GC
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --cluster-config g1gc
+
+# Run 2: Parallel GC
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --cluster-config parallelgc
+
+# Compare results
+solr-benchmark compare \
+ --baseline <g1gc-run-id> \
+ --contender <parallelgc-run-id>
+```
diff --git a/docs/cluster-config/index.md b/docs/cluster-config/index.md
new file mode 100644
index 00000000..a02bf8e5
--- /dev/null
+++ b/docs/cluster-config/index.md
@@ -0,0 +1,46 @@
+---
+title: Cluster Config
+nav_order: 27
+has_children: true
+---
+
+# Cluster Config
+
+The `--cluster-config` flag selects a preset JVM and garbage collector configuration applied to Solr nodes provisioned by Apache Solr Benchmark.
+
+{: .note }
+> `--cluster-config` is only valid with provisioning pipelines: `docker`, `from-distribution`, and `from-sources`. It is **not** applicable to the `benchmark-only` pipeline, which connects to an already-running cluster without modifying it.
+
+## Usage
+
+```bash
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --cluster-config 4gheap
+```
+
+## How it works
+
+Cluster configs are predefined sets of environment variables injected into the Solr startup environment:
+
+| Variable | Description |
+|----------|-------------|
+| `SOLR_HEAP` | JVM heap size (e.g., `4g`) |
+| `GC_TUNE` | JVM garbage collector arguments |
+| `SOLR_OPTS` | Additional JVM options |
+
+Apache Solr Benchmark applies these variables when starting a provisioned Solr node so you can benchmark different JVM configurations without modifying the Solr installation.
+
+## Available configs
+
+See [Available Configs](available-configs.html) for the full list of built-in configurations.
+
+## Why use cluster configs?
+
+Comparing the same workload across different JVM configurations helps you:
+
+- Find the optimal heap size for your workload
+- Compare G1GC vs Parallel GC performance
+- Identify GC-related latency spikes via telemetry
diff --git a/docs/converter/index.md b/docs/converter/index.md
new file mode 100644
index 00000000..96a641ea
--- /dev/null
+++ b/docs/converter/index.md
@@ -0,0 +1,33 @@
+---
+title: Converter Tool
+nav_order: 28
+has_children: true
+---
+
+# Converter Tool
+
+Apache Solr Benchmark includes a `convert-workload` command that translates OpenSearch Benchmark workloads into Apache Solr Benchmark format.
+
+{: .note }
+> The converter performs a **best-effort** translation. Most workloads require some manual review after conversion. Check the generated `CONVERTED.md` summary file for a list of items that may need attention.
+
+## When to use the converter
+
+Use `convert-workload` when you want to run an existing OpenSearch Benchmark workload against Apache Solr. Pre-built Solr workloads that need no conversion are available at [https://github.com/janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads).
+
+## Quick example
+
+```bash
+solr-benchmark convert-workload \
+ --workload-path /path/to/osb-workload \
+ --output-path /path/to/solr-workload
+```
+
+The converted workload is ready to run with:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path /path/to/solr-workload
+```
diff --git a/docs/converter/usage.md b/docs/converter/usage.md
new file mode 100644
index 00000000..4f80ccf6
--- /dev/null
+++ b/docs/converter/usage.md
@@ -0,0 +1,66 @@
+---
+title: Usage
+parent: Converter Tool
+nav_order: 2
+---
+
+# Using the Converter
+
+## Basic usage
+
+```bash
+solr-benchmark convert-workload \
+ --workload-path /path/to/osb-workload \
+ --output-path /path/to/solr-workload
+```
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--workload-path` | Path to the source OpenSearch Benchmark workload directory |
+| `--output-path` | Destination directory for the converted workload |
+| `--force` | Overwrite the output directory if it already exists |
+
+## Output
+
+The converter produces a copy of the workload in the output directory with the following changes applied:
+
+- `"indices"` keys renamed to `"collections"` in `workload.json`
+- `create-index` / `delete-index` operation types replaced with `create-collection` / `delete-collection`
+- OpenSearch JSON DSL search bodies translated to Solr JSON query format
+- Date range filters converted from custom formats (e.g., `dd/MM/yyyy`) to ISO 8601
+- Aggregations translated to Solr facet syntax
+
+A `CONVERTED.md` file is written to the output directory summarizing what was converted and flagging any items that require manual review.
+
+## Example output
+
+```
+Converted workload saved to: /path/to/solr-workload
+
+Summary:
+ - workload.json: renamed 2 "indices" key(s) to "collections"
+ - operations/default.json: translated 5 search bodies
+ - operations/default.json: converted 3 date range filters
+ - operations/default.json: translated 4 aggregations to facets
+
+Items requiring manual review (see CONVERTED.md):
+ - 1 script_score query (not auto-translated)
+ - 2 complex nested aggregations
+```
+
+## Idempotent re-runs
+
+If the output directory already contains a `CONVERTED.md` file, the converter will skip it unless `--force` is passed. This makes re-runs safe after partial edits:
+
+```bash
+# Re-run without losing manual edits
+solr-benchmark convert-workload \
+ --workload-path /path/to/osb-workload \
+ --output-path /path/to/solr-workload \
+ --force
+```
+
+{: .warning }
+> Using `--force` overwrites all files in the output directory, including any manual edits you have made.
diff --git a/docs/converter/what-converts.md b/docs/converter/what-converts.md
new file mode 100644
index 00000000..f7b064ef
--- /dev/null
+++ b/docs/converter/what-converts.md
@@ -0,0 +1,61 @@
+---
+title: What Gets Converted
+parent: Converter Tool
+nav_order: 3
+---
+
+# What Gets Converted
+
+The table below summarizes which OpenSearch Benchmark constructs are automatically converted, which require manual review, and which are skipped with a TODO comment.
+
+## Automatically converted
+
+| Construct | Conversion |
+|-----------|------------|
+| `"indices"` key in workload.json | Renamed to `"collections"` |
+| `create-index` operation type | → `create-collection` |
+| `delete-index` operation type | → `delete-collection` |
+| Simple `match` queries | Converted to Solr JSON DSL equivalent |
+| `term` / `terms` queries | Converted to Solr query string or `{!terms}` local params |
+| `range` queries | Converted to Solr range filter with ISO 8601 dates |
+| `bool` queries (`must` / `filter` / `should` / `must_not`) | Converted to Solr `bool` query parser |
+| Date ranges with `format` parameter | Dates parsed per format and written as ISO 8601 |
+| `terms` aggregations | Converted to Solr `terms` facet |
+| `date_histogram` aggregations | Converted to Solr `range` facet with calendar gap |
+| `avg` / `sum` / `min` / `max` aggregations | Converted to Solr function query stats |
+
+## Requires manual review
+
+These constructs are flagged in `CONVERTED.md` but not automatically converted:
+
+| Construct | Reason |
+|-----------|--------|
+| `script_score` queries | No direct Solr equivalent; requires a custom function query |
+| Complex nested aggregations (3+ levels) | Auto-translation may be incomplete; verify output |
+| Custom analyzers in mapping | Configset `schema.xml` must be created manually |
+| `multi_match` queries | Partially converted; review field list for Solr equivalents |
+
+## Skipped (with TODO comment)
+
+These operations are not meaningful in Apache Solr and are replaced with a `// TODO` comment in the output:
+
+| Construct | Action |
+|-----------|--------|
+| `cluster-health` | Not applicable to Solr |
+| OpenSearch ML Commons operations | Not applicable to Solr |
+| `close-index` / `open-index` | Not applicable to Solr |
+| Vector search operations (`knn`, `approximate_knn`) | Solr uses different vector search syntax; manual conversion required |
+
+## Date format mapping
+
+When OpenSearch range queries include a `format` parameter, the converter maps date format strings to Python `strptime` patterns:
+
+| OpenSearch format | Python pattern |
+|-------------------|---------------|
+| `dd/MM/yyyy` | `%d/%m/%Y` |
+| `MM/dd/yyyy` | `%m/%d/%Y` |
+| `yyyy-MM-dd` | `%Y-%m-%d` |
+| `dd-MM-yyyy` | `%d-%m-%Y` |
+| `yyyyMMdd` | `%Y%m%d` |
+
+Dates are converted to Solr ISO 8601 format: `YYYY-MM-DDTHH:MM:SSZ`.
diff --git a/docs/faq.md b/docs/faq.md
new file mode 100644
index 00000000..aa3a8018
--- /dev/null
+++ b/docs/faq.md
@@ -0,0 +1,58 @@
+---
+title: FAQ
+nav_order: 101
+---
+
+# Frequently Asked Questions
+
+**What is Apache Solr Benchmark?**
+Apache Solr Benchmark is a macro-benchmarking framework for [Apache Solr](https://solr.apache.org) clusters. It is a fork of OpenSearch Benchmark, adapted to work with Solr instead of OpenSearch.
+
+**What Solr versions are supported?**
+Apache Solr Benchmark supports Solr 9.x and Solr 10.x. Some metrics collection endpoints differ between versions and are detected automatically at runtime.
+
+**Where can I find pre-built workloads?**
+Pre-built workloads for Apache Solr Benchmark are available at [https://github.com/janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads).
+
+**How do I convert an OpenSearch Benchmark workload?**
+Use the `convert-workload` command:
+
+```bash
+solr-benchmark convert-workload \
+ --workload-path /path/to/osb-workload \
+ --output-path /path/to/solr-workload
+```
+
+See [Converter Tool](converter/) for details on what gets converted automatically.
+
+**How is a collection different from an index?**
+In Apache Solr, the equivalent of an OpenSearch index is a *collection*. A collection is a logical grouping of documents distributed across shards. See [Glossary](glossary.html) for a full terminology mapping.
+
+**Can I benchmark a multi-node Solr cluster?**
+Yes. Pass multiple hosts to `--target-hosts`, separated by commas:
+
+```bash
+solr-benchmark run --target-hosts node1:8983,node2:8983,node3:8983 ...
+```
+
+**What pipelines are available?**
+- `benchmark-only` — run against an existing cluster (no provisioning)
+- `docker` — start a Solr cluster via Docker, then benchmark, then tear down
+- `from-distribution` — download and install Solr, then benchmark, then tear down
+- `from-sources` — build Solr from source, install, benchmark, then tear down
+
+**How do I run in test mode?**
+Pass `--test-mode` to limit the workload to a small subset of documents (at most 1,000) for quick validation:
+
+```bash
+solr-benchmark run --test-mode ...
+```
+
+**Where are results stored?**
+Results from individual benchmark runs are stored in `~/.solr-benchmark/benchmarks/test-runs/<run-id>/`. Aggregated results (from the `aggregate` command or multi-iteration runs) go to `~/.solr-benchmark/benchmarks/aggregated_results/<id>/`.
+
+**Does Apache Solr Benchmark support distributed (multi-machine) benchmarking?**
+Yes. Use the daemon mode (`solr-benchmarkd`) on worker nodes and the coordinator on the driver node. See [Working with Workloads](user-guide/working-with-workloads/) for details.
+
+**What is `--cluster-config` for?**
+The `--cluster-config` flag selects a preset JVM/GC configuration for provisioned Solr nodes (used with `docker`, `from-distribution`, and `from-sources` pipelines). It is not applicable to the `benchmark-only` pipeline. See [Cluster Config](cluster-config/) for available presets.
diff --git a/docs/get-started.md b/docs/get-started.md
deleted file mode 100644
index b9e2ad7f..00000000
--- a/docs/get-started.md
+++ /dev/null
@@ -1,6 +0,0 @@
----
-layout: default
-title: Get Started
-parent: OSB
-nav_order: 1
----
\ No newline at end of file
diff --git a/docs/glossary.md b/docs/glossary.md
new file mode 100644
index 00000000..6819a0fa
--- /dev/null
+++ b/docs/glossary.md
@@ -0,0 +1,65 @@
+---
+title: Glossary
+nav_order: 100
+---
+
+# Glossary
+
+## Core concepts
+
+**Workload**
+A workload describes a benchmarking scenario: the data to load and the operations to run. A workload is defined by a `workload.json` file plus associated data, operation, and configset files. See [Understanding Workloads](user-guide/understanding-workloads/).
+
+**Test Procedure**
+A specific sequence of operations within a workload. A workload can define multiple test procedures (e.g., `append-no-conflicts`, `bulk-update`). Each test procedure specifies which operations to run and their parameters. Select a test procedure with `--test-procedure` when running the benchmark.
+
+**Pipeline**
+A sequence of phases that a benchmark run executes. Built-in pipelines:
+- `benchmark-only` — run against an existing cluster
+- `docker` — spin up a Solr cluster via Docker then benchmark
+- `from-distribution` — download and install Solr then benchmark
+- `from-sources` — build Solr from source then benchmark
+
+**Schedule**
+Defines how an operation is executed: number of iterations, target throughput (operations per second), warmup iterations, and parallel client count.
+
+**Operation**
+A single benchmarking action, such as `bulk-index`, `search`, `commit`, or `optimize`. Operations are referenced from schedules in a test procedure.
+
+**Corpora**
+The dataset used by a workload. Corpora are defined in the workload and reference data files (typically gzip-compressed NDJSON).
+
+## Apache Solr concepts
+
+**Collection**
+The Solr equivalent of an OpenSearch index. A collection is a logical grouping of documents, distributed across one or more shards. In OSB terminology: *index*.
+
+**Configset**
+A named set of Solr configuration files (`schema.xml`, `solrconfig.xml`) stored in ZooKeeper. Collections reference a configset by name.
+
+**Shard Leader**
+The primary replica of a shard responsible for accepting writes and coordinating replication. In OSB terminology: *primary shard*.
+
+**NRT Replica**
+A near-real-time replica that receives updates directly from the shard leader and participates in leader election.
+
+**TLOG Replica**
+A transaction-log replica that receives updates via replication from the shard leader and buffers changes in a transaction log.
+
+**Pull Replica**
+A read-only replica that only receives index segments from the shard leader, not transaction log entries.
+
+**Facets**
+Solr's aggregation mechanism for computing counts, statistics, and groupings over search results. In OSB terminology: *aggregations*.
+
+## Terminology mapping
+
+| OpenSearch Benchmark Term | Apache Solr Benchmark Canonical Term |
+|---------------------------|--------------------------------------|
+| index | collection |
+| indices | collections |
+| create-index | create-collection |
+| delete-index | delete-collection |
+| primary shard | shard leader |
+| aggregation / aggregations | facet / facets |
+| mapping | schema |
diff --git a/docs/index.md b/docs/index.md
index e69de29b..1d9da303 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -0,0 +1,28 @@
+---
+title: Apache Solr Benchmark
+nav_order: 1
+---
+
+# Apache Solr Benchmark
+
+Apache Solr Benchmark is a performance benchmarking tool for [Apache Solr](https://solr.apache.org) clusters, derived from [OpenSearch Benchmark](https://github.com/opensearch-project/opensearch-benchmark).
+
+Use Apache Solr Benchmark to measure the performance of your Apache Solr clusters across a variety of workloads — from simple keyword queries to complex faceted search and aggregations.
+
+## Getting started
+
+- [Quickstart](quickstart.html) — install the tool and run your first benchmark in minutes
+- [User Guide](user-guide/) — deeper guides on workloads, pipelines, and results
+- [Reference](reference/) — complete CLI and workload format reference
+
+## Key features
+
+- **Multiple pipelines** — benchmark an existing cluster (`benchmark-only`), or let Apache Solr Benchmark provision one via Docker (`docker`), direct installation (`from-distribution`), or source build (`from-sources`)
+- **Flexible workloads** — load pre-built workloads from [janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads) or create your own
+- **Workload converter** — convert existing OpenSearch Benchmark workloads to Solr format with the `convert-workload` command
+- **Telemetry** — collect JVM, node, and collection-level metrics from Solr during a run
+- **Result storage** — results saved as JSON and CSV to `~/.solr-benchmark/benchmarks/test-runs/`
+
+## Source code
+
+Apache Solr Benchmark is hosted at [https://github.com/janhoy/solr-benchmark](https://github.com/janhoy/solr-benchmark).
diff --git a/docs/quickstart.md b/docs/quickstart.md
new file mode 100644
index 00000000..b7bac927
--- /dev/null
+++ b/docs/quickstart.md
@@ -0,0 +1,191 @@
+---
+title: Quickstart
+nav_order: 2
+---
+
+# Apache Solr Benchmark quickstart
+
+This page outlines how to quickly install Apache Solr Benchmark and run your first benchmark workload.
+
+## Prerequisites
+
+To perform the quickstart steps, you'll need the following:
+
+- Git 2.3 or later
+- Python 3.10 or later
+- An active Apache Solr cluster, **or** Docker (to let Solr Benchmark start one for you)
+
+## Set up a Solr cluster
+
+If you don't already have a running Solr cluster, the easiest way to start one is with Docker:
+
+```bash
+docker run -d --name solr-benchmark -p 8983:8983 solr:9 solr-demo
+```
+
+Verify that Solr is running by opening [http://localhost:8983/solr/](http://localhost:8983/solr/) in your browser, or with:
+
+```bash
+curl http://localhost:8983/solr/admin/info/system?wt=json
+```
+
+With your cluster running, you can now install Apache Solr Benchmark.
+
+## Installing Apache Solr Benchmark
+
+{: .note }
+Apache Solr Benchmark is not yet published on PyPI. Install it directly from the source repository.
+
+Clone the repository and install in editable mode:
+
+```bash
+git clone https://github.com/janhoy/solr-benchmark.git
+cd solr-benchmark
+pip install -e .
+```
+
+After installation completes, verify that Solr Benchmark is running:
+
+```bash
+solr-benchmark --version
+```
+
+If successful, Solr Benchmark prints its version.
+
+## Running your first benchmark
+
+You can now run your first benchmark. The following example uses the built-in `nyc_taxis` workload.
+
+### Understanding workload command flags
+
+Benchmarks are run using the [`run`](reference/commands/run.html) command. Some commonly used flags:
+
+For the full list of flags, see the [run command reference](reference/commands/run.html) and [command flags reference](reference/commands/command-flags.html).
+{: .tip }
+
+- `--pipeline=benchmark-only` — tells Solr Benchmark that you are providing your own Solr cluster; Solr Benchmark will not start or stop one.
+- `--pipeline=docker` — tells Solr Benchmark to start a Solr cluster in Docker before the run and stop it afterwards.
+- `--pipeline=from-distribution,` - tells Solr Benchmark to download and install Solr from a distribution.
+- `--workload=nyc_taxis` — the name of the workload to run.
+- `--target-hosts=localhost:8983` — the host and port of the Solr cluster to benchmark, if using `benchmark-only` pipeline.
+- `--distribution-version=9.10.1` — the Solr version to use when provisioning a cluster.
+- `--test-mode` — runs an abbreviated version of the workload (first 1,000 documents per task) for a quick sanity check. Metrics produced in test mode are not meaningful for production analysis.
+
+### Running the workload
+
+To run the `nyc_taxis` workload against an existing Solr cluster:
+
+```bash
+solr-benchmark run \
+ --pipeline=benchmark-only \
+ --target-hosts=localhost:8983 \
+ --workload=nyc_taxis \
+ --test-mode
+```
+
+To instead let Solr Benchmark start a Solr cluster for you using Docker:
+
+```bash
+solr-benchmark run \
+ --pipeline=docker \
+ --distribution-version=9.10.1 \
+ --workload=nyc_taxis \
+ --test-mode
+```
+
+When the `run` command executes, all tasks and operations in the `nyc_taxis` workload run sequentially.
+
+### Validating the test
+
+After the benchmark completes, verify it ran correctly:
+
+- Check that a collection named `nyc_taxis` was created in your Solr cluster (visible in the Solr Admin UI at [http://localhost:8983/solr/](http://localhost:8983/solr/)).
+- In test mode, only a subset of documents are indexed. For a full run (without `--test-mode`), compare the document count in the collection against the expected count in the workload's `workload.json` file.
+
+### Understanding the results
+
+Once the benchmark completes, Solr Benchmark prints a summary table to the console:
+
+```
+------------------------------------------------------
+ _______ __ _____
+ / ____(_)___ ____ _/ / / ___/_________ ________
+ / /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
+ / __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
+/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
+------------------------------------------------------
+
+| Metric | Task | Value | Unit |
+|------------------------------:|--------------------------:|--------:|-------:|
+| Min Throughput | index | 3697.64 | docs/s |
+| Mean Throughput | index | 3697.64 | docs/s |
+| Median Throughput | index | 3697.64 | docs/s |
+| Max Throughput | index | 3697.64 | docs/s |
+| 50th percentile latency | index | 197.368 | ms |
+| 100th percentile latency | index | 214.59 | ms |
+| 50th percentile service time | index | 197.368 | ms |
+| 100th percentile service time | index | 214.59 | ms |
+| error rate | index | 0 | % |
+| Min Throughput | match-all | 18 | ops/s |
+| Mean Throughput | match-all | 18 | ops/s |
+| Median Throughput | match-all | 18 | ops/s |
+| Max Throughput | match-all | 18 | ops/s |
+| 100th percentile latency | match-all | 67.3283 | ms |
+| 100th percentile service time | match-all | 10.5466 | ms |
+| error rate | match-all | 0 | % |
+| Min Throughput | range | 35.23 | ops/s |
+| Mean Throughput | range | 35.23 | ops/s |
+| Median Throughput | range | 35.23 | ops/s |
+| Max Throughput | range | 35.23 | ops/s |
+| 100th percentile latency | range | 38.3106 | ms |
+| 100th percentile service time | range | 9.70796 | ms |
+| error rate | range | 0 | % |
+| Min Throughput | asc_sort_passenger_count | 75.87 | ops/s |
+| Mean Throughput | asc_sort_passenger_count | 75.87 | ops/s |
+| Median Throughput | asc_sort_passenger_count | 75.87 | ops/s |
+| Max Throughput | asc_sort_passenger_count | 75.87 | ops/s |
+| 100th percentile latency | asc_sort_passenger_count | 22.2872 | ms |
+| 100th percentile service time | asc_sort_passenger_count | 8.95417 | ms |
+| error rate | asc_sort_passenger_count | 0 | % |
+[...]
+
+----------------------------------
+[INFO] ✅ SUCCESS (took 38 seconds)
+----------------------------------
+```
+
+Each task in the summary corresponds to a specific operation that was performed against the Solr cluster during the benchmark. Each task produces the following metrics:
+
+- **Throughput** — the number of successful Solr operations completed per second (`ops/s`) or documents per second (`docs/s`) for indexing tasks.
+- **Latency** — the total time from when the benchmark submitted the request until it received a complete response, including any time the operation spent waiting in the internal scheduling queue.
+- **Service time** — the time from when the request was sent to Solr until the response was received, excluding scheduling queue wait time. When no target throughput is set, service time and latency are equal.
+- **Error rate** — the percentage of operations that returned an error or timed out. A value of `0` means all operations completed successfully.
+
+For more details on how the summary report is generated, see [Summary report](reference/summary-report.html).
+
+The computed results are also saved to `~/.solr-benchmark/benchmarks/test-runs/<run-id>/test_run.json` for later comparison with [`solr-benchmark compare`](reference/commands/compare.html).
+
+## Running Solr Benchmark on your own cluster
+
+Now that you're familiar with running a benchmark, you can run Solr Benchmark against your own Solr cluster. Use the same `run` command and adjust the following settings:
+
+- Replace `localhost:8983` with your cluster's host and port.
+- Remove `--test-mode` to run the full workload rather than the abbreviated test.
+- Use `--workload-params` to override workload parameters such as the number of clients or the target throughput.
+- Use `--include-tasks` or `--exclude-tasks` to run only the operations you care about.
+
+```bash
+solr-benchmark run \
+ --pipeline=benchmark-only \
+ --target-hosts=<your-solr-host>:8983 \
+ --workload=nyc_taxis
+```
+
+## Next steps
+
+See the following resources to learn more about Apache Solr Benchmark:
+
+- [User guide](user-guide/) — understand workloads, pipelines, challenges, and how to interpret results.
+- [Converter tool](converter/) — convert an existing OpenSearch Benchmark workload to Solr format.
+- [Command reference](reference/commands/) — complete CLI reference for all subcommands and flags.
+- [Metrics reference](reference/metrics/) — understand what every metric key means and how raw samples are stored.
diff --git a/docs/reference/commands/aggregate.md b/docs/reference/commands/aggregate.md
new file mode 100644
index 00000000..3251e451
--- /dev/null
+++ b/docs/reference/commands/aggregate.md
@@ -0,0 +1,94 @@
+---
+title: aggregate
+parent: Command Reference
+grand_parent: Reference
+nav_order: 10
+---
+
+# aggregate
+
+Combines results from multiple benchmark runs into a single aggregated result. This is useful when you want statistical confidence in your benchmark numbers — running a workload three or more times and aggregating the results reduces the influence of any single outlier run.
+
+## Two modes of aggregation
+
+### Automatic aggregation (via `run`)
+
+Pass `--test-iterations` and `--aggregate` directly to the `run` command to execute the workload multiple times and aggregate automatically:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --test-iterations 3 \
+ --aggregate true \
+ --sleep-timer 30
+```
+
+| Flag | Description | Default |
+|------|-------------|---------|
+| `--test-iterations` | Number of times to run the workload | `1` |
+| `--aggregate` | Aggregate results after all iterations | `true` |
+| `--sleep-timer` | Seconds to wait between runs | `5` |
+| `--cancel-on-error` | Stop all remaining iterations on first error | `false` |
+
+### Manual aggregation
+
+Run benchmarks separately, then combine specific runs by their test run IDs:
+
+```bash
+# First, list recent test runs to get IDs
+solr-benchmark list test-runs
+
+# Then aggregate the runs you want
+solr-benchmark aggregate \
+ --test-runs 20260101T120000Z,20260102T120000Z,20260103T120000Z
+```
+
+## Syntax
+
+```bash
+solr-benchmark aggregate --test-runs ID1,ID2[,...] [OPTIONS]
+```
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--test-runs` | Comma-separated list of test run IDs to aggregate |
+| `--test-runs-id` | Custom ID for the aggregated result (auto-generated if omitted) |
+| `--results-file` | Path to write the aggregated results JSON |
+
+## Output
+
+The aggregated result includes additional statistical fields compared to a single run:
+
+```json
+{
+ "task": "index",
+ "throughput": {
+ "overall_min": 3820.5,
+ "mean": 4105.3,
+ "median": 4098.7,
+ "overall_max": 4390.1,
+ "unit": "docs/s",
+ "mean_rsd": 3.8
+ }
+}
+```
+
+| Field | Description |
+|-------|-------------|
+| `overall_min` | True minimum value across all runs |
+| `mean` | Arithmetic mean across all runs |
+| `median` | Median value across all runs |
+| `overall_max` | True maximum value across all runs |
+| `mean_rsd` | Mean relative standard deviation (%) — lower is better; indicates how consistent the runs were |
+
+Aggregated results are saved to `~/.solr-benchmark/benchmarks/aggregated_results/<aggregated-id>/`.
+
+## See also
+
+- [compare](compare.html) — compare two individual runs
+- [list](list.html) — list test runs and their IDs
+- [Understanding Results](../../user-guide/understanding-results/)
diff --git a/docs/reference/commands/benchmarkd.md b/docs/reference/commands/benchmarkd.md
new file mode 100644
index 00000000..c6761cde
--- /dev/null
+++ b/docs/reference/commands/benchmarkd.md
@@ -0,0 +1,71 @@
+---
+title: solr-benchmarkd
+parent: Command Reference
+grand_parent: Reference
+nav_order: 15
+---
+
+# solr-benchmarkd
+
+`solr-benchmarkd` is the daemon binary used for distributed load generation. It must be running on every **worker** machine before starting a distributed benchmark from the coordinator.
+
+## Syntax
+
+```bash
+solr-benchmarkd start --node-ip IP --coordinator-ip IP
+solr-benchmarkd stop --node-ip IP --coordinator-ip IP
+solr-benchmarkd restart --node-ip IP --coordinator-ip IP
+solr-benchmarkd status
+```
+
+## Subcommands
+
+| Subcommand | Description |
+|-----------|-------------|
+| `start` | Start the daemon and register with the coordinator |
+| `stop` | Stop the running daemon |
+| `restart` | Stop and restart the daemon |
+| `status` | Print whether the daemon is currently running |
+
+## Options
+
+| Option | Required | Description |
+|--------|----------|-------------|
+| `--node-ip` | Yes (`start`, `stop`, `restart`) | IP address of this worker machine |
+| `--coordinator-ip` | Yes (`start`, `stop`, `restart`) | IP address of the coordinator machine running `solr-benchmark` |
+
+## Examples
+
+Start daemons on two worker machines, then run the benchmark from the coordinator:
+
+```bash
+# On worker-1 (192.168.1.10):
+solr-benchmarkd start --node-ip 192.168.1.10 --coordinator-ip 192.168.1.1
+
+# On worker-2 (192.168.1.11):
+solr-benchmarkd start --node-ip 192.168.1.11 --coordinator-ip 192.168.1.1
+
+# On the coordinator (192.168.1.1):
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts solr1:8983,solr2:8983 \
+ --worker-ips 192.168.1.10,192.168.1.11
+```
+
+Check daemon status on a worker:
+
+```bash
+solr-benchmarkd status
+```
+
+Stop a worker daemon when the benchmark is complete:
+
+```bash
+solr-benchmarkd stop --node-ip 192.168.1.10 --coordinator-ip 192.168.1.1
+```
+
+## See also
+
+- [Running Distributed Load](../../user-guide/optimizing-benchmarks/running-distributed-load.html)
+- [run command](run.html)
diff --git a/docs/reference/commands/command-flags.md b/docs/reference/commands/command-flags.md
new file mode 100644
index 00000000..90563f46
--- /dev/null
+++ b/docs/reference/commands/command-flags.md
@@ -0,0 +1,195 @@
+---
+title: Command Flags
+parent: Command Reference
+grand_parent: Reference
+nav_order: 150
+---
+
+# Command Flags
+
+Complete reference of all `solr-benchmark` command-line flags.
+
+## Global flags
+
+Accepted by all subcommands.
+
+| Flag | Short | Description |
+|------|-------|-------------|
+| `--help` | `-h` | Display help text for the current command and exit |
+| `--offline` | — | Run without network access; disables workload repository fetching and update checks |
+| `--version` | `-v` | Show version and exit |
+| `--quiet` | — | Suppress console output (except errors) |
+
+## run flags
+
+### Workload selection
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--workload` | string | — | Named workload (fetched from workloads repository) |
+| `--workload-path` | path | — | Local workload directory path |
+| `--workload-repository` | string | `"default"` | Git URL for the workloads repository (the string `"default"` resolves to the URL configured in `benchmark.ini`) |
+| `--workload-revision` | string | — | Git revision (branch, tag, or commit) of the workloads repository. If omitted, the branch is selected automatically based on `--distribution-version` (e.g., `10.0.0` → branch `10`); falls back to `main` if no matching branch exists. |
+| `--workload-params` | string | — | Comma-separated `key:value` Jinja2 parameter overrides |
+| `--test-procedure` | string | workload default | Test procedure name to run |
+| `--include-tasks` | string | — | Comma-separated task names to run; all other tasks are skipped |
+| `--exclude-tasks` | string | — | Comma-separated task names to skip |
+| `--enable-assertions` | flag | off | Enable task-level assertions defined in the workload |
+
+### Cluster and pipeline
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--pipeline` | string | (dynamic) | Pipeline to use: `benchmark-only`, `docker`, `from-distribution`, or `from-sources`. Defaults to `benchmark-only` when no provisioning flags are given |
+| `--target-hosts` | string | — | Comma-separated `host:port` list |
+| `--distribution-version` | string | — | Solr version for provisioning pipelines |
+| `--cluster-config` | string | `defaults` | Cluster config preset for provisioning pipelines |
+
+### Distributed load generation
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--worker-ips` | string | `localhost` | Comma-separated IP addresses of worker coordinator machines |
+
+### Multiple-iteration aggregation
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--test-iterations` | integer | `1` | Number of times to repeat the workload |
+| `--aggregate` | boolean | `true` | Aggregate results from all iterations |
+| `--sleep-timer` | integer | `5` | Seconds to wait between iterations |
+| `--cancel-on-error` | boolean | `false` | Abort remaining iterations on first error |
+
+### Telemetry
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--telemetry` | string | — | Comma-separated telemetry device names |
+| `--telemetry-params` | string | — | Telemetry device parameters |
+
+### Result output
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--test-run-id` | string | auto-generated | Custom ID for this run; used with `compare` and `aggregate` |
+| `--user-tag` | string | — | A single `key:value` metadata pair attached to every metric record in this run (e.g., `intention:baseline`) |
+| `--results-format` | string | `markdown` | Summary table format: `markdown` or `csv` |
+| `--results-numbers-align` | string | `right` | Column alignment: `right`, `left`, `center`, or `decimal` |
+| `--results-file` | path | — | Write the summary table to this file |
+| `--show-in-results` | string | `available` | Values to include: `available`, `all-percentiles`, or `all` |
+| `--visualize` | flag | off | Generate an interactive HTML visualization after the run |
+| `--visualize-output-path` | path | — | Path to write the HTML visualization file (defaults to the test run directory) |
+
+### General
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--test-mode` | flag | off | Run with ≤1,000 documents for quick validation |
+| `--on-error` | string | `continue` | Error strategy: `continue` or `abort` |
+| `--client-options` | string | `timeout:60` | Comma-separated client options passed to the Solr client (short: `-c`) |
+| `--kill-running-processes` | flag | off | Kill other running `solr-benchmark` processes before starting (short: `-k`) |
+| `--preserve-install` | flag | off | Keep the Solr installation after the run (provisioned pipelines only) |
+
+### Provisioning
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--revision` | string | — | Source code revision for the `from-sources` pipeline |
+| `--runtime-jdk` | integer | — | Major JDK version to use for provisioned Solr nodes (e.g., `21`) |
+| `--solr-modules` | string | — | Comma-separated Solr modules to enable (e.g., `extraction`) |
+| `--plugin-params` | string | — | Comma-separated `key:value` pairs passed to all configured plugins |
+| `--cluster-config-params` | string | — | Comma-separated `key:value` variable overrides for the cluster config |
+| `--cluster-config-repository` | string | — | Git URL for a custom cluster-config repository |
+| `--cluster-config-revision` | string | — | Git revision of the cluster-config repository |
+| `--distribution-repository` | string | `release` | Repository to download Solr from |
+
+### Advanced load generation
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--grpc-target-hosts` | string | `localhost:9400` | gRPC endpoint(s) for worker coordinator communication |
+| `--enable-worker-coordinator-profiling` | flag | off | Profile the worker coordinator process |
+| `--latency-percentiles` | string | — | Comma-separated additional percentiles to report for latency (e.g., `50,90,99,99.9`) |
+| `--throughput-percentiles` | string | — | Comma-separated additional percentiles to report for throughput |
+
+### Load testing
+
+These flags enable automated load-ramp and redline testing to find a cluster's performance limits.
+
+| Flag | Type | Default | Description |
+|------|------|---------|-------------|
+| `--load-test-qps` | integer | — | Run a load test that ramps up to this target QPS value |
+| `--redline-test` | integer? | `0` (off) | Run a redline test to find the maximum sustainable throughput. Use as a bare flag (`--redline-test`, implicitly 1000 QPS) or with an explicit QPS target (e.g. `--redline-test 5000`) |
+| `--redline-scale-step` | integer | — | Number of clients to add per scale step |
+| `--redline-scaledown-percentage` | float | — | Percentage of clients to remove when the error threshold is exceeded |
+| `--redline-post-scaledown-sleep` | integer | — | Seconds to wait after a scale-down event before resuming |
+| `--redline-max-clients` | integer | — | Maximum number of concurrent clients during a redline test |
+| `--redline-max-cpu-usage` | float | — | CPU usage percentage at which to begin scaling back |
+| `--redline-cpu-window-seconds` | integer | `30` | Window in seconds over which average CPU load is measured |
+| `--redline-cpu-check-interval` | integer | `30` | Seconds between CPU usage checks |
+
+## list flags
+
+| Flag | Description |
+|------|-------------|
+| `--workload` | Workload name (used with `list workloads` to filter by workload) |
+| `--workload-path` | Local workload directory |
+| `--workload-repository` | Git URL for the workloads repository |
+| `--workload-revision` | Git revision of the workloads repository |
+| `--limit` | Maximum number of test-run results to show (default: `10`; applies to `list test-runs`) |
+
+## info flags
+
+| Flag | Description |
+|------|-------------|
+| `--workload` | Workload name |
+| `--workload-path` | Local workload directory |
+| `--workload-repository` | Git URL for the workloads repository |
+| `--workload-revision` | Git revision of the workloads repository |
+| `--workload-params` | Comma-separated `key:value` Jinja2 parameter overrides |
+| `--test-procedure` | Specific test procedure to describe |
+| `--include-tasks` | Comma-separated task names to display |
+| `--exclude-tasks` | Comma-separated task names to hide |
+
+## compare flags
+
+| Flag | Description |
+|------|-------------|
+| `--baseline` | Test run ID of the baseline run (see `list test-runs`) |
+| `--contender` | Test run ID of the contender run (see `list test-runs`) |
+| `--results-format` | Output format: `markdown` (default) or `csv` |
+| `--results-numbers-align` | Column alignment: `right` (default), `left`, `center`, or `decimal` |
+| `--results-file` | Write the comparison table to a file |
+| `--show-in-results` | Values to include: `available` (default), `all-percentiles`, or `all` |
+| `--percentiles` | Comma-separated list of percentiles to include in the comparison |
+
+## aggregate flags
+
+| Flag | Description |
+|------|-------------|
+| `--test-runs` | Comma-separated test run IDs to aggregate |
+| `--test-runs-id` | Custom ID for the aggregated result |
+| `--results-file` | Path to write the aggregated results JSON |
+| `--workload-repository` | Git URL for the workloads repository |
+
+## download flags
+
+Solr is pure Java — no OS- or architecture-specific variants exist.
+
+| Flag | Description |
+|------|-------------|
+| `--distribution-version` | Solr version to download (e.g., `9.10.1`) |
+| `--distribution-repository` | Source repository (default: `release`) |
+| `--cluster-config` | Cluster configuration preset to apply |
+| `--cluster-config-params` | Comma-separated `key:value` variable overrides for the cluster configuration |
+| `--cluster-config-path` | Local path to a cluster configuration directory |
+| `--cluster-config-repository` | Git URL for a cluster configuration repository |
+| `--cluster-config-revision` | Git revision of the cluster-config repository to use |
+
+## convert-workload flags
+
+| Flag | Description |
+|------|-------------|
+| `--workload-path` | Path to the source (OpenSearch Benchmark format) workload directory |
+| `--output-path` | Destination directory for the converted workload |
+| `--force` | Overwrite the output directory if it already exists |
diff --git a/docs/reference/commands/compare.md b/docs/reference/commands/compare.md
new file mode 100644
index 00000000..fbc35e4c
--- /dev/null
+++ b/docs/reference/commands/compare.md
@@ -0,0 +1,49 @@
+---
+title: compare
+parent: Command Reference
+grand_parent: Reference
+nav_order: 20
+---
+
+# compare
+
+Compares results from two benchmark runs.
+
+## Syntax
+
+```bash
+solr-benchmark compare --baseline BASELINE_ID --contender CONTENDER_ID
+```
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--baseline` | Test execution ID of the baseline run |
+| `--contender` | Test execution ID of the contender run |
+| `--percentiles` | Comma-separated list of latency percentiles to report (e.g., `50,90,99,99.9`) |
+| `--results-format` | Output format: `markdown` (default) or `csv` |
+| `--results-numbers-align` | Column alignment in the output table: `right` (default), `left`, `center`, or `decimal` |
+| `--results-file` | Write the comparison table to a file in addition to console output |
+| `--show-in-results` | Which values to include: `available` (default), `all-percentiles`, or `all` |
+
+## Example
+
+```bash
+solr-benchmark compare \
+ --baseline 20240101T120000Z \
+ --contender 20240115T120000Z
+```
+
+The output shows the delta between baseline and contender for each metric:
+
+```
+| Metric | Baseline | Contender | Diff |
+|----------------------|----------|-----------|--------|
+| bulk-index (50th ms) | 14.2 | 12.3 | -13.4% |
+| bulk-index (docs/s) | 4200 | 4500 | +7.1% |
+| search (50th ms) | 6.1 | 5.1 | -16.4% |
+| search (ops/s) | 105 | 120 | +14.3% |
+```
+
+Negative differences in latency and positive differences in throughput indicate improvement.
diff --git a/docs/reference/commands/create-workload.md b/docs/reference/commands/create-workload.md
new file mode 100644
index 00000000..75f38825
--- /dev/null
+++ b/docs/reference/commands/create-workload.md
@@ -0,0 +1,57 @@
+---
+title: create-workload
+parent: Command Reference
+grand_parent: Reference
+nav_order: 35
+---
+
+# create-workload
+
+Creates a new workload definition from data already indexed in a live Solr instance. Use this to capture a real dataset as a replayable workload corpus.
+
+## Syntax
+
+```bash
+solr-benchmark create-workload --workload NAME --indices LIST --target-hosts HOSTS [OPTIONS]
+```
+
+## Options
+
+| Option | Short | Required | Default | Description |
+|--------|-------|----------|---------|-------------|
+| `--workload` | `-w` | Yes | — | Name for the generated workload |
+| `--indices` | `-i` | Yes | — | Comma-separated list of Solr collection (index) names to include |
+| `--target-hosts` | `-t` | Yes | — | Comma-separated `host:port` pairs for the source Solr cluster |
+| `--client-options` | `-c` | No | `timeout:60` | Comma-separated client options (e.g., `timeout:60`) |
+| `--output-path` | | No | `./workloads` | Directory where the generated workload files will be written |
+| `--custom-queries` | | No | — | Path to a JSON file containing custom queries to include in the workload (overrides the default `match_all` query) |
+| `--number-of-docs` | | No | — | Per-index document count overrides in `index:count` format (e.g., `my_index:50000 other_index:10000`). The index name must also appear in `--indices` |
+| `--sample-frequency` | | No | — | Per-index sampling frequency in `index:n` format. Every *n*th document is extracted. The index name must also appear in `--indices` |
+
+## Examples
+
+Create a workload from two collections on a local Solr instance:
+
+```bash
+solr-benchmark create-workload \
+ --workload my_workload \
+ --indices products,reviews \
+ --target-hosts localhost:8983 \
+ --output-path /workloads/my_workload
+```
+
+Create a workload extracting a fixed number of documents per collection:
+
+```bash
+solr-benchmark create-workload \
+ --workload sales_workload \
+ --indices orders,customers \
+ --target-hosts solr-prod:8983 \
+ --number-of-docs orders:100000 customers:50000 \
+ --output-path /workloads/sales
+```
+
+## See also
+
+- [Working with Workloads](../../user-guide/working-with-workloads/index.html)
+- [generate-data](generate-data.html)
diff --git a/docs/reference/commands/download.md b/docs/reference/commands/download.md
new file mode 100644
index 00000000..f07a37d1
--- /dev/null
+++ b/docs/reference/commands/download.md
@@ -0,0 +1,51 @@
+---
+title: download
+parent: Command Reference
+grand_parent: Reference
+nav_order: 30
+---
+
+# download
+
+Downloads a Solr distribution to the local machine without running a benchmark. Use this to pre-fetch a distribution before running a `from-distribution` or `docker` pipeline benchmark.
+
+## Syntax
+
+```bash
+solr-benchmark download --distribution-version VERSION [OPTIONS]
+```
+
+Because Solr is pure Java, the distribution tarball is the same on every operating system and CPU architecture. There are no OS- or architecture-specific variants to select.
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--distribution-version` | Solr version to download (e.g., `9.10.1`, `10.0.0`) |
+| `--distribution-repository` | Source repository (default: `release`) |
+| `--cluster-config` | Cluster configuration preset to apply |
+| `--cluster-config-params` | Comma-separated `key:value` variable overrides for the cluster configuration |
+| `--cluster-config-path` | Local path to a cluster configuration directory |
+| `--cluster-config-repository` | Git URL for a cluster configuration repository |
+
+## Output
+
+On success, the command prints a JSON object with the path to the downloaded artifact:
+
+```json
+{
+ "solr": "/Users/yourname/.solr-benchmark/distributions/solr-9.10.1.tgz"
+}
+```
+
+## Examples
+
+```bash
+# Download Solr 9.10.1
+solr-benchmark download --distribution-version 9.10.1
+```
+
+## See also
+
+- [from-distribution pipeline](../../user-guide/concepts.html#pipelines)
+- [Cluster Config](../../cluster-config/)
diff --git a/docs/reference/commands/generate-data.md b/docs/reference/commands/generate-data.md
new file mode 100644
index 00000000..b8017e35
--- /dev/null
+++ b/docs/reference/commands/generate-data.md
@@ -0,0 +1,58 @@
+---
+title: generate-data
+parent: Command Reference
+grand_parent: Reference
+nav_order: 40
+---
+
+# generate-data
+
+Generates synthetic benchmark data from an existing index schema (JSON mappings) or a custom Python module. The generated corpus can be used in Apache Solr Benchmark workloads.
+
+## Syntax
+
+```bash
+solr-benchmark generate-data --index-mappings FILE --total-size N --index-name NAME [OPTIONS]
+solr-benchmark generate-data --custom-module FILE --total-size N --index-name NAME [OPTIONS]
+```
+
+`--index-mappings` and `--custom-module` are mutually exclusive. `--total-size` is required.
+
+## Options
+
+| Option | Short | Required | Default | Description |
+|--------|-------|----------|---------|-------------|
+| `--index-mappings` | `-i` | Yes (or `--custom-module`) | — | Path to a JSON file containing index mappings to use as the schema for generated documents |
+| `--custom-module` | `-m` | Yes (or `--index-mappings`) | — | Path to a custom Python module that defines document generation logic. The module must contain a `generate_synthetic_document()` function |
+| `--total-size` | `-s` | Yes | — | Target corpus size in GB |
+| `--index-name` | `-n` | Yes | — | Name for the generated corpus (used in the output file path) |
+| `--output-path` | `-p` | No | `./generated_corpora` | Directory where the generated corpus files will be written |
+| `--custom-config` | `-c` | No | — | Optional config file for overriding synthetic data generation settings or providing values used by a custom module |
+| `--test-document` | `-t` | No | off | Generate a single document and print it to the console for validation, without writing a full corpus |
+
+## Examples
+
+Generate 10 GB of synthetic data from an existing schema:
+
+```bash
+solr-benchmark generate-data \
+ --index-mappings /path/to/mappings.json \
+ --index-name my_index \
+ --total-size 10 \
+ --output-path /data/corpora
+```
+
+Preview a single generated document using a custom module:
+
+```bash
+solr-benchmark generate-data \
+ --custom-module /path/to/my_generator.py \
+ --index-name my_index \
+ --total-size 1 \
+ --test-document
+```
+
+## See also
+
+- [Corpora](../workloads/corpora.html)
+- [create-workload](create-workload.html)
diff --git a/docs/reference/commands/index.md b/docs/reference/commands/index.md
new file mode 100644
index 00000000..ca1b5128
--- /dev/null
+++ b/docs/reference/commands/index.md
@@ -0,0 +1,39 @@
+---
+title: Command Reference
+parent: Reference
+nav_order: 50
+has_children: true
+---
+
+# Command Reference
+
+Reference documentation for all `solr-benchmark` subcommands and their flags.
+
+| Command | Description |
+|---------|-------------|
+| [aggregate](aggregate.html) | Combine results from multiple benchmark runs |
+| [compare](compare.html) | Compare two benchmark runs side by side |
+| [convert-workload](../../converter/) | Convert an OpenSearch Benchmark workload to Solr format |
+| [create-workload](create-workload.html) | Create a workload definition from an existing Solr collection |
+| [download](download.html) | Download a Solr distribution without running a benchmark |
+| [generate-data](generate-data.html) | Generate synthetic data from an index schema or custom module |
+| [info](info.html) | Show detailed information about a workload |
+| [install](install.html) | Install a Solr node locally (for manual lifecycle control) |
+| [list](list.html) | List available workloads, telemetry, pipelines, or past runs |
+| [run](run.html) | Run a benchmark workload |
+| [start](start.html) | Start a locally installed Solr node |
+| [stop](stop.html) | Stop a locally installed Solr node |
+| [visualize](visualize.html) | Generate an HTML visualization for a completed run |
+
+## Daemon
+
+The `solr-benchmarkd` binary manages worker daemon processes for distributed load generation. See [solr-benchmarkd](benchmarkd.html) for the full reference.
+
+## Common options
+
+The following flags are accepted by every `solr-benchmark` subcommand.
+
+| Flag | Short | Description |
+|------|-------|-------------|
+| `--help` | `-h` | Display help text for the current command and exit |
+| `--offline` | — | Run without network access; disables workload repository fetching and any update checks |
diff --git a/docs/reference/commands/info.md b/docs/reference/commands/info.md
new file mode 100644
index 00000000..4b236add
--- /dev/null
+++ b/docs/reference/commands/info.md
@@ -0,0 +1,48 @@
+---
+title: info
+parent: Command Reference
+grand_parent: Reference
+nav_order: 70
+---
+
+# info
+
+Shows detailed information about a workload.
+
+## Syntax
+
+```bash
+solr-benchmark info --workload WORKLOAD [OPTIONS]
+```
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--workload` | Workload name (fetched from the workloads repository) |
+| `--workload-path` | Path to a local workload directory |
+| `--workload-repository` | Git URL for the workloads repository |
+| `--workload-revision` | Git revision of the workloads repository |
+| `--workload-params` | Comma-separated `key:value` Jinja2 parameter overrides |
+| `--test-procedure` | Show details for a specific test procedure |
+| `--include-tasks` | Comma-separated list of task names to display (others are hidden) |
+| `--exclude-tasks` | Comma-separated list of task names to hide from the output |
+
+## Examples
+
+```bash
+# Show information about a named workload
+solr-benchmark info --workload nyc_taxis
+
+# Show information about a local workload
+solr-benchmark info --workload-path /path/to/my-workload
+
+# Show details for a specific test procedure
+solr-benchmark info --workload nyc_taxis --test-procedure append-no-conflicts
+```
+
+The output includes:
+- Workload description
+- Available test procedures and their descriptions
+- Corpora names and document counts
+- Default parameters and their values
diff --git a/docs/reference/commands/install.md b/docs/reference/commands/install.md
new file mode 100644
index 00000000..32c278da
--- /dev/null
+++ b/docs/reference/commands/install.md
@@ -0,0 +1,81 @@
+---
+title: install
+parent: Command Reference
+grand_parent: Reference
+nav_order: 55
+---
+
+# install
+
+Installs a local Solr node for use with the [`start`](start.html) and [`stop`](stop.html) commands. This is a low-level provisioning command for when you want manual control over the Solr lifecycle rather than having `run` manage it automatically.
+
+## Syntax
+
+```bash
+solr-benchmark install [OPTIONS]
+```
+
+## Options
+
+### Distribution
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--distribution-version` | — | Solr version to install (e.g., `9.10.1`) |
+| `--distribution-repository` | `release` | Repository to download Solr from |
+| `--revision` | — | Source code revision for the `from-sources` pipeline |
+
+### Runtime
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--runtime-jdk` | — | Major JDK version to use (e.g., `21`) |
+| `--solr-modules` | — | Comma-separated Solr modules to enable (e.g., `extraction`) |
+| `--plugin-params` | — | Comma-separated `key:value` pairs passed to all configured plugins |
+
+### Cluster configuration
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--cluster-config` | `defaults` | Cluster config preset to apply |
+| `--cluster-config-params` | — | Comma-separated `key:value` variable overrides for the cluster config |
+| `--cluster-config-path` | — | Local path to a cluster configuration directory |
+| `--cluster-config-repository` | — | Git URL for a custom cluster-config repository |
+| `--cluster-config-revision` | — | Git revision of the cluster-config repository |
+
+### Node placement
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--network-host` | `127.0.0.1` | Host address for Solr to bind to |
+| `--http-port` | `38983` | HTTP port for the Solr node |
+| `--node-name` | `benchmark-node-0` | Name for the installed node |
+| `--master-nodes` | — | Comma-separated list of master node names (for multi-node setups) |
+| `--seed-hosts` | — | Comma-separated seed host addresses for cluster discovery |
+
+## Example
+
+Install Solr 9.10.1 locally on the default port:
+
+```bash
+solr-benchmark install \
+ --distribution-version 9.10.1 \
+ --cluster-config defaults
+```
+
+Install on a custom port with a specific JDK:
+
+```bash
+solr-benchmark install \
+ --distribution-version 9.10.1 \
+ --runtime-jdk 21 \
+ --http-port 18983 \
+ --node-name my-node
+```
+
+After installation, use the printed installation ID with the [`start`](start.html) command.
+
+## See also
+
+- [start](start.html)
+- [stop](stop.html)
diff --git a/docs/reference/commands/list.md b/docs/reference/commands/list.md
new file mode 100644
index 00000000..b5b91a9a
--- /dev/null
+++ b/docs/reference/commands/list.md
@@ -0,0 +1,70 @@
+---
+title: list
+parent: Command Reference
+grand_parent: Reference
+nav_order: 80
+---
+
+# list
+
+Lists available resources such as workloads and telemetry devices.
+
+## Syntax
+
+```bash
+solr-benchmark list [RESOURCE] [OPTIONS]
+```
+
+## Resources
+
+| Resource | Description |
+|----------|-------------|
+| `workloads` | List workloads from the configured workloads repository |
+| `telemetry` | List available telemetry device names |
+| `pipelines` | List available pipeline names |
+| `test-runs` | List past benchmark runs with their IDs, timestamps, and metadata |
+| `aggregated-results` | List past aggregated results |
+| `cluster-configs` | List available cluster configuration presets |
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--workload` | Workload name (used with `list workloads`) |
+| `--workload-path` | Path to a local workload directory |
+| `--workload-repository` | Git URL for the workloads repository |
+| `--workload-revision` | Git revision of the workloads repository |
+| `--limit` | Maximum number of entries to show (default: `10`; applies to `list test-runs`) |
+
+## Examples
+
+```bash
+# List available workloads
+solr-benchmark list workloads
+
+# List available telemetry devices
+solr-benchmark list telemetry
+
+# List available pipelines
+solr-benchmark list pipelines
+
+# List available cluster config presets
+solr-benchmark list cluster-configs
+
+# List recent test runs (shows IDs for use with compare and aggregate)
+solr-benchmark list test-runs
+
+# List the 20 most recent test runs
+solr-benchmark list test-runs --limit 20
+
+# List aggregated results
+solr-benchmark list aggregated-results
+```
+
+To list the test procedures available in a workload, use `solr-benchmark info`:
+
+```bash
+solr-benchmark info --workload nyc_taxis
+```
+
+The `test-runs` output includes the test run ID, timestamp, workload name, test procedure, pipeline, and any user tags. Use the ID with `solr-benchmark compare` to compare two runs.
diff --git a/docs/reference/commands/run.md b/docs/reference/commands/run.md
new file mode 100644
index 00000000..4e7bff5c
--- /dev/null
+++ b/docs/reference/commands/run.md
@@ -0,0 +1,164 @@
+---
+title: run
+parent: Command Reference
+grand_parent: Reference
+nav_order: 90
+---
+
+# run
+
+Runs a benchmark workload.
+
+## Syntax
+
+```bash
+solr-benchmark run [OPTIONS]
+```
+
+## Workload selection
+
+| Option | Description |
+|--------|-------------|
+| `--workload` | Named workload from the workloads repository |
+| `--workload-path` | Path to a local workload directory |
+| `--workload-repository` | Git URL for the workloads repository (default: `"default"`, resolved from `benchmark.ini`) |
+| `--workload-revision` | Git revision (branch, tag, or commit) of the workloads repository; optional, uses the repository's default branch if omitted |
+| `--workload-params` | Override workload Jinja2 parameters (comma-separated `key:value` pairs) |
+| `--test-procedure` | Test procedure to run (default: the workload's default test procedure) |
+| `--include-tasks` | Comma-separated list of task names to run; all other tasks are skipped |
+| `--exclude-tasks` | Comma-separated list of task names to skip |
+| `--enable-assertions` | Enable task-level assertions defined in the workload |
+
+## Cluster and pipeline
+
+| Option | Description |
+|--------|-------------|
+| `--pipeline` | Pipeline to use: `benchmark-only`, `docker`, `from-distribution`, `from-sources`. If omitted, the pipeline is selected automatically (see below) |
+| `--target-hosts` | Comma-separated list of Solr `host:port` targets |
+| `--distribution-version` | Solr version (e.g., `9.10.1`) for `docker`/`from-distribution` pipelines |
+| `--cluster-config` | Cluster configuration preset for `docker`/`from-distribution`/`from-sources` pipelines |
+
+### Pipeline auto-selection
+
+If `--pipeline` is omitted, Solr Benchmark selects a pipeline automatically:
+
+- `from-distribution` — when `--distribution-version` is specified
+- `benchmark-only` — otherwise (connects to an already-running cluster)
+
+## Distributed load generation
+
+| Option | Description |
+|--------|-------------|
+| `--worker-ips` | Comma-separated IP addresses of worker coordinator machines for distributed load generation (default: `localhost`) |
+
+## Multiple-iteration aggregation
+
+| Option | Default | Description |
+|--------|---------|-------------|
+| `--test-iterations` | `1` | Number of times to repeat the workload |
+| `--aggregate` | `true` | Aggregate results from all iterations |
+| `--sleep-timer` | `5` | Seconds to wait between iterations |
+| `--cancel-on-error` | `false` | Abort remaining iterations if any iteration fails |
+
+## Telemetry
+
+| Option | Description |
+|--------|-------------|
+| `--telemetry` | Comma-separated list of optional telemetry devices to enable (see [Telemetry reference](../telemetry.html)) |
+| `--telemetry-params` | Key-value parameters for telemetry devices |
+
+## Result output
+
+| Option | Description |
+|--------|-------------|
+| `--test-run-id` | Custom unique ID for this run (auto-generated if omitted); used with `compare` |
+| `--user-tag` | A single `key:value` metadata pair attached to every metric record in this run (e.g., `intention:baseline`) |
+| `--results-format` | Output format: `markdown` (default) or `csv` |
+| `--results-numbers-align` | Column alignment in the summary table: `right` (default), `left`, `center`, or `decimal` |
+| `--results-file` | Write the summary table to a file in addition to the default location |
+| `--show-in-results` | Which values to include in output: `available` (default), `all-percentiles`, or `all` |
+| `--visualize` | Generate an interactive HTML visualization after the run |
+| `--visualize-output-path` | Path to write the HTML visualization file |
+
+## General
+
+| Option | Description |
+|--------|-------------|
+| `--test-mode` | Run a shortened version of the workload (≤1,000 docs) for quick validation |
+| `--on-error` | Error handling: `continue` (default), `abort` |
+| `--client-options` / `-c` | Comma-separated client options (default: `timeout:60`) |
+| `--kill-running-processes` / `-k` | Kill other running `solr-benchmark` processes before starting |
+| `--preserve-install` | Keep the Solr installation after the run (provisioned pipelines only) |
+
+The `--quiet` flag is accepted by all subcommands; see [Command Flags](command-flags.html#global-flags).
+
+## Workload parameter overrides
+
+Workload files can contain Jinja2 template variables, for example:
+
+```json
+{
+ "bulk-size": {{ bulk_size | default(500) }},
+ "clients": {{ clients | default(4) }}
+}
+```
+
+Override these at run time with `--workload-params` as a comma-separated list of `key:value` pairs:
+
+```bash
+solr-benchmark run --workload-params "bulk_size:1000,clients:8" ...
+```
+
+Related parameter flags:
+
+| Flag | Purpose |
+|------|---------|
+| `--workload-params` | Jinja2 variable overrides for the workload |
+| `--cluster-config-params` | Variable overrides for the cluster configuration |
+| `--plugin-params` | Parameters passed to all configured plugins |
+| `--telemetry-params` | Parameters passed to telemetry devices |
+
+## Examples
+
+```bash
+# Benchmark an existing cluster
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --test-mode
+
+# Docker pipeline with Solr 9.10.1
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis
+
+# Custom workload with parameter overrides
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path /path/to/my-workload \
+ --workload-params "bulk_size:1000,clients:8"
+
+# With optional telemetry devices
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --telemetry shard-stats,cluster-environment-info
+
+# Run 3 iterations and aggregate results
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --test-iterations 3 \
+ --sleep-timer 10
+```
+
+## See also
+
+- [Pipelines overview](../../user-guide/concepts.html#pipelines)
+- [Cluster Config](../../cluster-config/)
+- [Command Flags](command-flags.html)
diff --git a/docs/reference/commands/start.md b/docs/reference/commands/start.md
new file mode 100644
index 00000000..7762432d
--- /dev/null
+++ b/docs/reference/commands/start.md
@@ -0,0 +1,53 @@
+---
+title: start
+parent: Command Reference
+grand_parent: Reference
+nav_order: 60
+---
+
+# start
+
+Starts a locally installed Solr node previously created by the [`install`](install.html) command.
+
+## Syntax
+
+```bash
+solr-benchmark start --installation-id ID --test-run-id ID [OPTIONS]
+```
+
+## Options
+
+| Option | Required | Description |
+|--------|----------|-------------|
+| `--installation-id` | Yes | ID of the installation to start (returned by the `install` command) |
+| `--test-run-id` | Yes | Test run ID to associate with this start event |
+| `--runtime-jdk` | No | Major JDK version to use (e.g., `21`) |
+| `--telemetry` | No | Comma-separated telemetry device names to activate |
+| `--telemetry-params` | No | Comma-separated `key:value` parameters for telemetry devices |
+
+## Example
+
+```bash
+# Install and capture the installation ID
+solr-benchmark install --distribution-version 9.10.1
+
+# Start the installed node (use the ID printed by install)
+solr-benchmark start \
+ --installation-id <installation-id> \
+ --test-run-id my-run-001
+
+# Run the benchmark against the manually started node
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:38983 \
+ --workload nyc_taxis \
+ --test-run-id my-run-001
+
+# Stop when done
+solr-benchmark stop --installation-id <installation-id>
+```
+
+## See also
+
+- [install](install.html)
+- [stop](stop.html)
diff --git a/docs/reference/commands/stop.md b/docs/reference/commands/stop.md
new file mode 100644
index 00000000..35b1acd1
--- /dev/null
+++ b/docs/reference/commands/stop.md
@@ -0,0 +1,42 @@
+---
+title: stop
+parent: Command Reference
+grand_parent: Reference
+nav_order: 85
+---
+
+# stop
+
+Stops a locally running Solr node that was started with the [`start`](start.html) command.
+
+## Syntax
+
+```bash
+solr-benchmark stop --installation-id ID [OPTIONS]
+```
+
+## Options
+
+| Option | Required | Default | Description |
+|--------|----------|---------|-------------|
+| `--installation-id` | Yes | — | ID of the installation to stop |
+| `--preserve-install` | No | off | Keep the installation files on disk after stopping. By default, the installation directory is deleted when the node stops |
+
+## Examples
+
+Stop a node and clean up:
+
+```bash
+solr-benchmark stop --installation-id <installation-id>
+```
+
+Stop a node but keep the installation files:
+
+```bash
+solr-benchmark stop --installation-id <installation-id> --preserve-install
+```
+
+## See also
+
+- [install](install.html)
+- [start](start.html)
diff --git a/docs/reference/commands/visualize.md b/docs/reference/commands/visualize.md
new file mode 100644
index 00000000..cd6ea8cd
--- /dev/null
+++ b/docs/reference/commands/visualize.md
@@ -0,0 +1,56 @@
+---
+title: visualize
+parent: Command Reference
+grand_parent: Reference
+nav_order: 140
+---
+
+# visualize
+
+Generates an interactive HTML visualization for a completed benchmark run. The output file can be opened in any web browser to explore metric trends, throughput charts, and latency distributions.
+
+## Syntax
+
+```bash
+solr-benchmark visualize --test-run-id ID [OPTIONS]
+```
+
+## Options
+
+| Option | Short | Required | Default | Description |
+|--------|-------|----------|---------|-------------|
+| `--test-run-id` | `-tid` | Yes | — | ID of the completed test run to visualize (see `solr-benchmark list test-runs`) |
+| `--output-path` | | No | Test run directory | Path where the HTML report should be saved. Defaults to the run's own directory alongside `test_run.json` |
+
+## Example
+
+```bash
+# List available runs to find the ID
+solr-benchmark list test-runs
+
+# Generate a visualization
+solr-benchmark visualize --test-run-id 20240115T120000Z
+
+# Save to a specific location
+solr-benchmark visualize \
+ --test-run-id 20240115T120000Z \
+ --output-path /reports/run-20240115.html
+```
+
+## Inline visualization during a run
+
+You can also generate a visualization automatically at the end of a benchmark run using the `--visualize` and `--visualize-output-path` flags on the [`run`](run.html) command:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --visualize \
+ --visualize-output-path /reports/latest.html
+```
+
+## See also
+
+- [run](run.html)
+- [list](list.html)
diff --git a/docs/reference/index.md b/docs/reference/index.md
new file mode 100644
index 00000000..91918698
--- /dev/null
+++ b/docs/reference/index.md
@@ -0,0 +1,14 @@
+---
+title: Reference
+nav_order: 25
+has_children: true
+---
+
+# Reference
+
+Complete reference documentation for Apache Solr Benchmark commands, workload format, telemetry devices, result output, and metrics.
+
+| Section | Description |
+|---------|-------------|
+| [Command Reference](commands/) | All `solr-benchmark` subcommands and their flags |
+| [Metrics Reference](metrics/) | Metric keys, filesystem store layout, and raw data inspection |
diff --git a/docs/reference/metrics/filesystem-metrics-store.md b/docs/reference/metrics/filesystem-metrics-store.md
new file mode 100644
index 00000000..4c89b15d
--- /dev/null
+++ b/docs/reference/metrics/filesystem-metrics-store.md
@@ -0,0 +1,98 @@
+---
+title: Filesystem Metrics Store
+parent: Metrics Reference
+grand_parent: Reference
+nav_order: 1
+---
+
+# Filesystem Metrics Store
+
+The filesystem metrics store is an opt-in extension of the default in-memory store.
+It keeps all metric records in RAM and additionally streams each metric document to a
+`metrics.jsonl` file on disk as the benchmark runs, making raw samples available for
+offline analysis even after the process exits.
+
+The default metrics store is **in-memory**. Enable the filesystem store explicitly when
+you need access to individual raw samples after the run.
+
+## Configuration
+
+The store type is controlled by the `datastore.type` key under the `[reporting]` section in
+`~/.solr-benchmark/benchmark.ini`:
+
+```ini
+[reporting]
+# Default: in-memory (no disk writes for raw samples)
+# Set to "filesystem" to also stream raw metric documents to metrics.jsonl
+datastore.type = filesystem
+```
+
+## File layout
+
+When the filesystem metrics store is active, the following structure is created under
+`~/.solr-benchmark/` after a completed benchmark run:
+
+```
+~/.solr-benchmark/
+└── benchmarks/
+ └── test-runs/
+ └── <run-id>/
+ ├── test_run.json # full run record with calculated results
+ └── metrics.jsonl # raw metric documents, one JSON object per line
+```
+
+`test_run.json` is written by the test-run store after every run regardless of which
+metrics store is active.
+
+### `test_run.json`
+
+Contains the full computed results (percentiles, error rates, throughput summaries)
+together with workload metadata and benchmark environment information.
+
+### `metrics.jsonl`
+
+Written incrementally during the run, one JSON object per line.
+Lines are written with line buffering, so no data is lost if the process is killed after
+the first measurement.
+
+Example line:
+
+```json
+{"test-run-id":"abc123","environment":"local","workload":"nyc_taxis","test_procedure":"append-no-conflicts","name":"service_time","value":42.7,"unit":"ms","task":"index","operation-type":"bulk","sample-type":"normal","absolute-time-ms":1709123456789,"relative-time-ms":1234,"meta":{"success":true}}
+```
+
+## Inspecting raw metrics
+
+### Using `jq`
+
+List all distinct metric names recorded in a run:
+
+```sh
+jq -r '.name' ~/.solr-benchmark/benchmarks/test-runs/<run-id>/metrics.jsonl | sort -u
+```
+
+Compute the median service time for a task:
+
+```sh
+jq 'select(.name=="service_time" and .task=="index") | .value' \
+ ~/.solr-benchmark/benchmarks/test-runs/<run-id>/metrics.jsonl \
+ | sort -n | awk '{a[NR]=$0} END{print a[int(NR/2)]}'
+```
+
+### Using Python
+
+```python
+import json
+
+with open("~/.solr-benchmark/benchmarks/test-runs/<run-id>/metrics.jsonl") as f:
+ docs = [json.loads(line) for line in f]
+
+service_times = [d["value"] for d in docs if d["name"] == "service_time"]
+print(f"Samples: {len(service_times)}, avg: {sum(service_times)/len(service_times):.2f} ms")
+```
+
+### Pretty-printing a single line
+
+```sh
+head -1 ~/.solr-benchmark/benchmarks/test-runs/<run-id>/metrics.jsonl | python3 -m json.tool
+```
diff --git a/docs/reference/metrics/index.md b/docs/reference/metrics/index.md
new file mode 100644
index 00000000..5a16736d
--- /dev/null
+++ b/docs/reference/metrics/index.md
@@ -0,0 +1,64 @@
+---
+title: Metrics Reference
+parent: Reference
+nav_order: 60
+has_children: true
+---
+
+# Metrics
+
+Apache Solr Benchmark stores all metrics collected during a benchmark run so that they can be
+analyzed and compared across runs. This page describes the available storage options.
+
+## Storing metrics
+
+Metrics can be stored in two ways depending on your analysis requirements.
+
+### In memory (default)
+
+The default configuration keeps all metric records in RAM for the duration of the run.
+Results are computed from this in-memory state and written to `test_run.json` when the run
+completes. The raw individual samples are not persisted beyond the process lifetime, which
+keeps disk usage minimal and avoids per-sample write overhead.
+
+No configuration is required to use in-memory storage — it is the default. You can also
+set it explicitly in `~/.solr-benchmark/benchmark.ini`:
+
+```ini
+[reporting]
+datastore.type = in-memory
+```
+
+### Filesystem (opt-in)
+
+The filesystem metrics store keeps all metric records in RAM (exactly like the in-memory
+store) **and** also streams every raw metric document to a `metrics.jsonl` file on disk as
+it arrives. This makes individual samples available for offline analysis even after the
+benchmark process exits, at the cost of additional disk I/O during the run.
+
+To opt in, add the following to `~/.solr-benchmark/benchmark.ini`:
+
+```ini
+[reporting]
+datastore.type = filesystem
+```
+
+Files are written to:
+
+```
+~/.solr-benchmark/
+└── benchmarks/
+ └── test-runs/
+ └── <run-id>/
+ ├── test_run.json # computed results (percentiles, error rates, …)
+ └── metrics.jsonl # raw metric documents, one JSON object per line
+```
+
+See [Filesystem Metrics Store](filesystem-metrics-store.html) for full configuration and
+file layout details, including `jq` and Python examples for inspecting raw samples.
+
+## Next steps
+
+- [Filesystem Metrics Store](filesystem-metrics-store.html) — store configuration and file layout
+- [Metric Records](metric-records.html) — structure of individual metric documents
+- [Metric Keys](metrics-reference.html) — catalog of every metric key Solr Benchmark can record
diff --git a/docs/reference/metrics/metric-records.md b/docs/reference/metrics/metric-records.md
new file mode 100644
index 00000000..e6046aac
--- /dev/null
+++ b/docs/reference/metrics/metric-records.md
@@ -0,0 +1,82 @@
+---
+title: Metric Records
+parent: Metrics Reference
+grand_parent: Reference
+nav_order: 2
+---
+
+# Metric Records
+
+Apache Solr Benchmark stores metrics as individual JSON documents. When the
+[filesystem metrics store](filesystem-metrics-store.html) is active (the default),
+each document is appended as a single line to
+`~/.solr-benchmark/benchmarks/test-runs/<run-id>/metrics.jsonl`.
+
+The following example shows a single `service_time` record produced during a bulk-index
+operation on the `nyc_taxis` workload:
+
+```json
+{
+ "@timestamp": 1691702842821,
+ "relative-time-ms": 65.9,
+ "test-run-id": "8c43ee4c-cb34-494b-81b2-181be244f832",
+ "test-run-timestamp": "20230810T212711Z",
+ "environment": "local",
+ "workload": "nyc_taxis",
+ "test_procedure": "append-no-conflicts",
+ "cluster-config-instance": "external",
+ "name": "service_time",
+ "value": 42.3,
+ "unit": "ms",
+ "sample-type": "normal",
+ "meta": {
+ "collection": "nyc_taxis",
+ "took": 13,
+ "success": true,
+ "success-count": 125,
+ "error-count": 0
+ },
+ "task": "bulk-index",
+ "operation": "bulk-index",
+ "operation-type": "bulk"
+}
+```
+
+## Field reference
+
+| Field | Description |
+|-------|-------------|
+| `@timestamp` | The timestamp of when the sample was taken, in milliseconds since the Unix epoch. For request-related measurements this marks the moment Solr Benchmark issued the request. |
+| `relative-time-ms` | The relative time since the start of the benchmark, in milliseconds. This is useful for aligning time-series data across multiple test runs because it is always measured from the same benchmark-internal zero point. |
+| `test-run-id` | A UUID that uniquely identifies this invocation of the workload. Every metric record from a single run shares the same value. |
+| `test-run-timestamp` | The timestamp of when the workload was invoked, always expressed in UTC (for example `20230810T212711Z`). |
+| `environment` | The name of the benchmark environment, as set in the configuration. Different benchmark environments can be distinguished using this field when analyzing metrics from multiple setups. |
+| `workload` | The name of the workload that produced this metric (for example `nyc_taxis` or `geonames`). |
+| `test_procedure` | The name of the test procedure (challenge) that was executed within the workload. |
+| `cluster-config-instance` | The name of the cluster configuration instance used during this run (for example `external`, `docker`, `defaults`). |
+| `name` | The metric key that identifies what was measured. See [Metric Keys](metrics-reference.html) for the complete list. |
+| `value` | The measured value. |
+| `unit` | The unit of the measurement, such as `ms`, `bytes`, `ops/s`, or an empty string for dimensionless counts. |
+| `sample-type` | Either `warmup` or `normal`. Only `normal` samples are included in the reported percentiles and summaries. Warmup samples are collected for priming the benchmark but excluded from results. |
+| `task` | The name of the workload task that produced this sample (corresponds to an operation defined in the workload). |
+| `operation` | The operation name within the task (often the same as `task`). |
+| `operation-type` | The operation type as registered by the runner, for example `bulk`, `search`, `commit`, or `telemetry`. |
+| `meta` | Supplementary context recorded alongside the measurement. Contents vary by operation type — see below. |
+
+## The `meta` object
+
+The `meta` object carries additional context that is specific to the type of operation that
+produced the record. Common fields include:
+
+| Field | Description |
+|-------|-------------|
+| `success` | `true` if the operation completed without an error; `false` if Solr returned an error or the request timed out. |
+| `success-count` | For bulk operations, the number of documents successfully indexed in this batch. |
+| `error-count` | For bulk operations, the number of documents that failed to index in this batch. |
+| `collection` | The Solr collection name targeted by the operation. |
+| `took` | The `QTime` value reported by Solr, in milliseconds. |
+| `hits` | For search operations, the total number of matching documents reported by Solr. |
+| `tag_*` | Any key–value pairs supplied via `--user-tag` are stored with a `tag_` prefix, for example `tag_intention: baseline`. |
+
+Telemetry records (collected by background polling devices) carry additional node-level
+context in `meta`, such as the Solr node name.
diff --git a/docs/reference/metrics/metrics-reference.md b/docs/reference/metrics/metrics-reference.md
new file mode 100644
index 00000000..7ca14911
--- /dev/null
+++ b/docs/reference/metrics/metrics-reference.md
@@ -0,0 +1,309 @@
+---
+title: Metric Keys
+parent: Metrics Reference
+grand_parent: Reference
+nav_order: 3
+---
+
+# Metric Keys
+
+This page lists every metric key that Apache Solr Benchmark can record. Metric keys appear
+in the `name` field of [metric records](metric-records.html).
+
+Metrics fall into two broad groups:
+
+- **Task-level metrics** — recorded once per operation sample, tied to a specific workload task
+- **Telemetry metrics** — recorded by background polling devices throughout the run, independent of any task
+
+---
+
+## Task-level metrics
+
+### `latency`
+
+The time period between submitting a request and receiving the complete response, in
+milliseconds. Latency includes any time the operation spent waiting in Solr Benchmark's
+internal scheduling queue before being sent to the Solr cluster.
+
+When the workload is not rate-limited (no target throughput), latency and service time are
+equal. When a target throughput is configured and demand exceeds capacity, latency grows
+while service time stays constant.
+
+### `service_time`
+
+The time period between sending a request to Solr and receiving the corresponding response,
+in milliseconds. Unlike latency, service time excludes any time the operation spent waiting
+in the client-side scheduling queue.
+
+### `processing_time`
+
+The time period from when Solr Benchmark starts processing the operation (including
+client-side serialization, parameter setup, and response parsing) to when it receives the
+complete response from Solr, in milliseconds. Processing time includes all client-side
+overhead and is always greater than or equal to service time.
+
+### `throughput`
+
+The number of operations that Solr Benchmark completed within a unit of time, expressed
+as operations per second (`ops/s`). For bulk-indexing tasks the unit may instead be
+`docs/s` or `MB/s` depending on workload configuration.
+
+---
+
+## Disk I/O metrics
+
+### `disk_io_write_bytes`
+
+The number of bytes written to disk during the benchmark. On Linux, only bytes written by
+the Solr Benchmark process are counted. On macOS, the total is system-wide and may include
+writes from other processes.
+
+### `disk_io_read_bytes`
+
+The number of bytes read from disk during the benchmark.
+
+---
+
+## JVM metrics (SolrJvmStats)
+
+Collected every 5 seconds from the Solr `/admin/metrics?group=jvm` endpoint. Both Solr 9.x
+(JSON) and Solr 10.x (Prometheus) response formats are handled automatically.
+
+### `jvm_heap_used_bytes`
+
+The amount of JVM heap memory currently in use, in bytes.
+
+### `jvm_heap_max_bytes`
+
+The maximum JVM heap size configured for the Solr process, in bytes.
+
+### `jvm_gc_count`
+
+The cumulative total number of garbage-collection events across all collectors (young and
+old generation combined).
+
+### `jvm_gc_time_ms`
+
+The cumulative total time spent in garbage collection across all collectors, in milliseconds.
+
+### `jvm_gc_young_count`
+
+The cumulative number of young-generation (minor) garbage-collection events. The specific
+collector name (G1 Young Generation, ParNew, and so on) varies with the JVM configuration.
+
+### `jvm_gc_young_time_ms`
+
+The cumulative time spent in young-generation garbage collection, in milliseconds.
+
+### `jvm_gc_old_count`
+
+The cumulative number of old-generation (major) garbage-collection events.
+
+### `jvm_gc_old_time_ms`
+
+The cumulative time spent in old-generation garbage collection, in milliseconds.
+
+### `jvm_thread_count`
+
+The current number of live JVM threads in the Solr process.
+
+### `jvm_thread_peak_count`
+
+The peak number of live JVM threads since the Solr process started.
+
+### `jvm_buffer_pool_direct_bytes`
+
+The amount of memory used by the JVM direct byte-buffer pool, in bytes. Solr uses direct
+buffers extensively for network I/O and memory-mapped file access.
+
+### `jvm_buffer_pool_mapped_bytes`
+
+The amount of memory used by the JVM memory-mapped buffer pool, in bytes.
+
+---
+
+## Node metrics (SolrNodeStats)
+
+Collected from `/api/node/system` and `/admin/metrics` endpoints. Both Solr 9.x (JSON) and
+Solr 10.x (Prometheus) response formats are handled automatically.
+
+### `cpu_usage_percent`
+
+The CPU usage of the Solr process (or, where `processCpuLoad` is not available, the
+system-wide CPU load), expressed as a percentage from 0 to 100.
+
+### `os_memory_free_bytes`
+
+The amount of free physical memory available to the operating system, in bytes, as reported
+by the JVM OS management bean.
+
+### `node_file_descriptors_open`
+
+The current number of open file descriptors held by the Solr process.
+
+### `node_file_descriptors_max`
+
+The maximum number of file descriptors the Solr process is allowed to open (the system
+limit).
+
+### `node_http_requests_total`
+
+The cumulative total number of HTTP requests handled by Solr's embedded Jetty server since
+startup.
+
+### `query_handler_requests_total`
+
+The cumulative total number of requests handled by the `/select` query handler.
+
+### `query_handler_errors_total`
+
+The cumulative total number of errors returned by the `/select` query handler.
+
+### `query_handler_avg_latency_ms`
+
+The rolling mean (exponentially weighted moving average) of request latency for the
+`/select` query handler, in milliseconds. This reflects Solr's internal view of query
+latency, independent of network round-trip time.
+
+---
+
+## Collection metrics (SolrCollectionStats)
+
+Collected every 30 seconds per collection. Data is fetched from the collection
+`core-properties` API and the Luke request handler.
+
+### `num_docs`
+
+The total number of live (non-deleted) documents in the collection, summed across all
+shards.
+
+### `index_size_bytes`
+
+The on-disk size of the collection index (heap usage of the index, as reported by Solr),
+in bytes. Summed across all shards.
+
+### `segment_count`
+
+The total number of Lucene segments open in the collection's searcher, summed across all
+shards. A high segment count relative to the document count typically indicates that a
+merge is needed.
+
+### `num_deleted_docs`
+
+The total number of deleted (but not yet merged away) documents in the collection. These
+documents consume disk space until a merge removes them.
+
+---
+
+## Query statistics (SolrQueryStats)
+
+Collected from the `/admin/metrics` endpoint. Telemetry device `SolrQueryStats` must be
+enabled via `--telemetry solr-query-stats`.
+
+### `query_latency_p50_ms`
+
+The 50th-percentile (median) rolling query latency for the `/select` handler, in
+milliseconds, as reported by Solr's internal timer.
+
+### `query_latency_p99_ms`
+
+The 99th-percentile rolling query latency, in milliseconds.
+
+### `query_latency_p999_ms`
+
+The 99.9th-percentile rolling query latency, in milliseconds.
+
+### `query_requests_total`
+
+Alias of [`query_handler_requests_total`](#query_handler_requests_total) — cumulative total
+requests to the `/select` handler.
+
+### `query_errors_total`
+
+Alias of [`query_handler_errors_total`](#query_handler_errors_total) — cumulative total
+errors from the `/select` handler.
+
+### `query_cache_hit_ratio`
+
+The hit ratio (0.0–1.0) of the Solr filter cache (`filterCache`). A value of 1.0 means
+all filter queries were served from cache; 0.0 means all required a full disk read.
+
+---
+
+## Indexing statistics (SolrIndexingStats)
+
+Collected from the `/admin/metrics` endpoint. Telemetry device `SolrIndexingStats` must be
+enabled via `--telemetry solr-indexing-stats`.
+
+### `indexing_requests_total`
+
+The cumulative total number of requests handled by the `/update` handler (documents sent
+to Solr for indexing).
+
+### `indexing_errors_total`
+
+The cumulative total number of errors returned by the `/update` handler.
+
+### `indexing_avg_time_ms`
+
+The rolling mean request time for the `/update` handler, in milliseconds, as reported by
+Solr's internal timer.
+
+### `index_merge_major_running`
+
+The current number of major Lucene segment merges in progress. Major merges combine many
+segments and are typically I/O intensive.
+
+### `index_merge_minor_running`
+
+The current number of minor Lucene segment merges in progress.
+
+---
+
+## Cache statistics (SolrCacheStats)
+
+Collected from the `/admin/metrics` endpoint per cache. Telemetry device `SolrCacheStats`
+must be enabled via `--telemetry solr-cache-stats`. Statistics are reported separately for
+three caches: `queryResultCache`, `filterCache`, and `documentCache`. Each record carries a
+`cache` field in its `meta` object identifying which cache it belongs to.
+
+### `cache_hits_total`
+
+Cumulative total number of cache lookups that found a cached entry.
+
+### `cache_inserts_total`
+
+Cumulative total number of entries inserted into the cache (roughly equal to the number of
+cache misses).
+
+### `cache_evictions_total`
+
+Cumulative total number of entries evicted from the cache to make room for new entries.
+High eviction rates suggest the cache is undersized relative to the working set.
+
+### `cache_memory_bytes`
+
+The amount of RAM currently used by the cache, in bytes.
+
+### `cache_hit_ratio`
+
+The ratio of cache hits to total lookups (0.0–1.0). Equivalent to
+`cache_hits_total / (cache_hits_total + cache_inserts_total)`.
+
+---
+
+## Derived metrics
+
+The following values are not stored as raw records but are computed from raw samples by
+Solr Benchmark when generating the results report.
+
+### `error_rate`
+
+The fraction of operations that resulted in an error (Solr returned a 4xx/5xx response, or
+the request timed out). Computed as:
+
+```
+error_rate = error_count / (error_count + success_count)
+```
+
+A value of `0.0` means all operations succeeded.
diff --git a/docs/reference/summary-report.md b/docs/reference/summary-report.md
new file mode 100644
index 00000000..2b0dfb01
--- /dev/null
+++ b/docs/reference/summary-report.md
@@ -0,0 +1,85 @@
+---
+title: Summary Report Format
+parent: Reference
+nav_order: 40
+---
+
+# Summary Report Format
+
+Apache Solr Benchmark saves results in JSON and CSV format to `~/.solr-benchmark/benchmarks/test-runs/<run-id>/`.
+
+## JSON format
+
+The `results.json` file contains an array of metric objects, one per recorded metric per task:
+
+```json
+[
+ {
+ "task": "bulk-index",
+ "operation": "bulk-index",
+ "throughput": {
+ "mean": 4500.0,
+ "unit": "docs/s"
+ },
+ "latency": {
+ "50": 12.3,
+ "90": 18.7,
+ "99": 25.1,
+ "99.9": 32.4,
+ "unit": "ms"
+ },
+ "error_rate": 0.0,
+ "duration": 120.5
+ },
+ {
+ "task": "search",
+ "operation": "search",
+ "throughput": {
+ "mean": 120.0,
+ "unit": "ops/s"
+ },
+ "latency": {
+ "50": 5.1,
+ "90": 9.3,
+ "99": 15.2,
+ "99.9": 22.8,
+ "unit": "ms"
+ },
+ "error_rate": 0.0,
+ "duration": 20.1
+ }
+]
+```
+
+## CSV format
+
+The `results.csv` file contains the same data in tabular form, with one row per metric per task. Column names match the JSON field names.
+
+## Run metadata
+
+Each run also produces a `test_run.json` file alongside the results:
+
+```json
+{
+ "test_run_id": "20240115T120000Z",
+ "workload": "nyc_taxis",
+ "test_procedure": "append-no-conflicts",
+ "pipeline": "benchmark-only",
+ "target_hosts": ["localhost:8983"],
+ "start_time": "2024-01-15T12:00:00Z",
+ "end_time": "2024-01-15T12:05:00Z",
+ "solr_version": "9.10.1"
+}
+```
+
+## Metric definitions
+
+| Field | Description |
+|-------|-------------|
+| `task` | Name of the schedule step |
+| `operation` | Operation type (e.g., `bulk-index`, `search`) |
+| `throughput.mean` | Mean throughput over the measurement period |
+| `throughput.unit` | `docs/s` for indexing, `ops/s` for queries |
+| `latency.50` … `latency.99.9` | Latency percentiles in milliseconds |
+| `error_rate` | Fraction of operations that returned an error (0.0 to 1.0) |
+| `duration` | Total task duration in seconds |
diff --git a/docs/reference/telemetry.md b/docs/reference/telemetry.md
new file mode 100644
index 00000000..bca3f386
--- /dev/null
+++ b/docs/reference/telemetry.md
@@ -0,0 +1,291 @@
+---
+title: Telemetry Devices
+parent: Reference
+nav_order: 45
+---
+
+# Telemetry Devices
+
+Apache Solr Benchmark includes telemetry devices for collecting server-side metrics. Six devices are **always enabled** and collect metrics automatically without any flags. One additional device (`disk-io`) runs automatically on provisioned pipelines. Eight further devices can be enabled on demand with `--telemetry`.
+
+Both Solr 9.x (JSON format) and Solr 10.x (Prometheus text format) are supported. Format detection is automatic via the HTTP `Content-Type` header.
+
+---
+
+## solr-jvm-stats
+
+Collects JVM statistics from each Solr node via the Solr metrics API.
+
+**Always active.** No `--telemetry` flag required.
+
+**Metrics collected:**
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `jvm_heap_used_bytes` | bytes | JVM heap memory currently used |
+| `jvm_heap_max_bytes` | bytes | Maximum JVM heap size |
+| `jvm_gc_count` | count | Total GC collections across all collectors |
+| `jvm_gc_time_ms` | ms | Total GC wall time across all collectors |
+| `jvm_gc_young_count` | count | Young-generation GC collection count |
+| `jvm_gc_young_time_ms` | ms | Young-generation GC wall time |
+| `jvm_gc_old_count` | count | Old-generation GC collection count |
+| `jvm_gc_old_time_ms` | ms | Old-generation GC wall time |
+| `jvm_thread_count` | count | Current JVM thread count |
+| `jvm_thread_peak_count` | count | Peak JVM thread count since startup |
+| `jvm_buffer_pool_direct_bytes` | bytes | Direct byte buffer pool memory used |
+| `jvm_buffer_pool_mapped_bytes` | bytes | Memory-mapped buffer pool memory used |
+
+---
+
+## solr-node-stats
+
+Collects Solr node-level and OS statistics.
+
+**Always active.** No `--telemetry` flag required.
+
+**Metrics collected:**
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `cpu_usage_percent` | % | Process CPU load (0–100) |
+| `os_memory_free_bytes` | bytes | Free physical OS memory |
+| `node_file_descriptors_open` | count | Currently open file descriptors |
+| `node_file_descriptors_max` | count | Maximum allowed file descriptors |
+| `node_http_requests_total` | count | Total HTTP requests processed by Jetty |
+| `query_handler_requests_total` | count | Total `/select` query handler requests |
+| `query_handler_errors_total` | count | Total `/select` query handler errors |
+| `query_handler_avg_latency_ms` | ms | Rolling average `/select` request latency |
+
+---
+
+## solr-collection-stats
+
+Collects per-collection document count, segment count, and deleted doc count.
+
+**Always active.** No `--telemetry` flag required.
+
+**Metrics collected** (all tagged with `collection` metadata):
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `num_docs` | docs | Current document count |
+| `num_deleted_docs` | docs | Number of deleted (soft-deleted) documents |
+| `segment_count` | count | Number of Lucene segments |
+| `index_size_bytes` | bytes | Total index size on disk |
+
+**Notes:** Collection stats are polled every 30 seconds by default. Override with `--telemetry-params collection-stats-sample-interval:60`. Uses both the Collections API and the Luke request handler (`/admin/luke`) for full statistics.
+
+---
+
+## solr-query-stats
+
+Collects query latency percentiles and filter cache hit ratio.
+
+**Always active.** No `--telemetry` flag required.
+
+**Metrics collected:**
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `query_latency_p50_ms` | ms | 50th percentile `/select` request latency |
+| `query_latency_p99_ms` | ms | 99th percentile `/select` request latency |
+| `query_latency_p999_ms` | ms | 99.9th percentile `/select` request latency |
+| `query_requests_total` | count | Total `/select` handler request count |
+| `query_errors_total` | count | Total `/select` handler error count |
+| `query_cache_hit_ratio` | ratio | Filter cache hit ratio (0.0–1.0) |
+
+---
+
+## solr-indexing-stats
+
+Collects update handler and merge metrics.
+
+**Always active.** No `--telemetry` flag required.
+
+**Metrics collected:**
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `indexing_requests_total` | count | Total `/update` handler requests |
+| `indexing_errors_total` | count | Total `/update` handler errors |
+| `indexing_avg_time_ms` | ms | Rolling average `/update` request time |
+| `index_merge_major_running` | count | Currently running major merges |
+| `index_merge_minor_running` | count | Currently running minor merges |
+
+---
+
+## solr-cache-stats
+
+Collects hit/miss/eviction and memory statistics for the three primary Solr caches.
+
+**Always active.** No `--telemetry` flag required.
+
+**Metrics collected** (all tagged with `cache` metadata: `queryResultCache`, `filterCache`, `documentCache`):
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `cache_hits_total` | count | Cache hits since Solr start |
+| `cache_inserts_total` | count | Cache inserts since Solr start |
+| `cache_evictions_total` | count | Cache evictions since Solr start |
+| `cache_memory_bytes` | bytes | RAM used by this cache |
+| `cache_hit_ratio` | ratio | Hit ratio (0.0–1.0) |
+
+---
+
+## Always-on provisioned-pipeline device
+
+The following device activates automatically when using the `docker` or `from-distribution` pipeline. It cannot be disabled and does not need to be listed in `--telemetry`.
+
+### disk-io
+
+Measures disk I/O consumed by the Solr process during the benchmark run.
+
+**Pipeline:** `docker` or `from-distribution` (always active; not available on `benchmark-only`)
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `disk_io_read_bytes` | bytes | Bytes read by the Solr process |
+| `disk_io_write_bytes` | bytes | Bytes written by the Solr process |
+
+---
+
+## Optional devices
+
+Optional devices must be explicitly requested with `--telemetry <name>`. REST-based optional
+devices work with all pipelines. JVM/process devices inject flags into `SOLR_OPTS` before Solr
+starts and require the `docker` or `from-distribution` pipeline — they are silently skipped on
+`benchmark-only`.
+
+---
+
+### segment-stats
+
+Captures per-collection segment statistics via the Solr Luke request handler.
+
+**Pipeline:** All pipelines
+**Enable:** `--telemetry segment-stats`
+
+Collected on benchmark stop. Results are written to `segment_stats.log` in the benchmark log directory.
+
+| Metric (in log file) | Description |
+|----------------------|-------------|
+| `numDocs` | Current document count |
+| `maxDoc` | Maximum doc ID (includes deleted docs) |
+| `deletedDocs` | Number of soft-deleted documents |
+| `segmentCount` | Number of Lucene segments |
+| `sizeInBytes` | Total index size on disk |
+
+---
+
+### shard-stats
+
+Polls CLUSTERSTATUS and Core STATUS for each shard leader and records per-shard metrics. Skipped silently on standalone (non-SolrCloud) Solr.
+
+**Pipeline:** All pipelines
+**Enable:** `--telemetry shard-stats`
+
+| Metric | Unit | Description |
+|--------|------|-------------|
+| `shard_{name}_num_docs` | count | Document count for the named shard |
+| `shard_{name}_size_bytes` | bytes | Index size for the named shard |
+
+The default poll interval is 60 seconds. Override with `--telemetry-params shard-stats-sample-interval:30`.
+
+---
+
+### cluster-environment-info
+
+Records Solr version, JVM version, and hardware info as run metadata. Called once at benchmark start.
+
+**Pipeline:** All pipelines
+**Enable:** `--telemetry cluster-environment-info`
+
+| Metadata key | Description |
+|--------------|-------------|
+| `distribution_version` | Solr version string (e.g. `9.7.0`) |
+| `jvm_version` | JVM version (e.g. `21.0.1`) |
+| `jvm_vendor` | JVM vendor name (e.g. `OpenJDK 21`) |
+| `cpu_logical_cores` | Number of logical CPU cores on the node |
+| `cluster_node_count` | Number of live SolrCloud nodes |
+
+---
+
+### jfr
+
+Enables Java Flight Recorder. Injects `-XX:StartFlightRecording=...` into `SOLR_OPTS`.
+
+**Pipeline:** `docker` or `from-distribution` only
+**Enable:** `--telemetry jfr`
+
+The flight recording is written to `profile.jfr` in the benchmark log directory. Requires OpenJDK 11 or later. To use a custom recording template, pass `--telemetry-params recording-template:/path/to/template.jfc`.
+
+---
+
+### gc
+
+Enables GC logging (Java 9+ `-Xlog:` format). Injects the GC log flags into `SOLR_OPTS`.
+
+**Pipeline:** `docker` or `from-distribution` only
+**Enable:** `--telemetry gc`
+
+GC output is written to `gc.log` in the benchmark log directory. The default log configuration is `gc*=info,safepoint=info,age*=trace`. Override with `--telemetry-params gc-log-config:gc*=debug`.
+
+---
+
+### jit
+
+Enables JIT compiler logging. Injects `-XX:+LogCompilation` and related flags into `SOLR_OPTS`.
+
+**Pipeline:** `docker` or `from-distribution` only
+**Enable:** `--telemetry jit`
+
+JIT output is written to `jit.log` in the benchmark log directory.
+
+---
+
+### heapdump
+
+Captures a heap dump (`jmap -dump`) from the Solr JVM when the benchmark finishes.
+
+**Pipeline:** `docker` or `from-distribution` only
+**Enable:** `--telemetry heapdump`
+
+The heap dump is written to `heap_at_exit_{pid}.hprof` in the benchmark log directory. On Docker pipelines, `docker exec` is used automatically.
+
+---
+
+## Using multiple devices
+
+The six always-on devices collect metrics automatically — no flags required:
+
+```bash
+# Always-on devices activate without any --telemetry flag:
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis
+```
+
+Add optional REST devices on any pipeline:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --telemetry segment-stats,shard-stats,cluster-environment-info
+```
+
+Add JVM profiling devices on provisioned pipelines:
+
+```bash
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --telemetry gc,jfr
+```
+
+## Telemetry output location
+
+Telemetry metrics are written to the metrics store alongside all other benchmark metrics. When using the filesystem store, they are recorded in `metrics.jsonl` in the test run directory (`~/.solr-benchmark/benchmarks/test-runs/<run-id>/`). See [Filesystem Metrics Store](metrics/filesystem-metrics-store.html) for the file format.
diff --git a/docs/reference/workloads/collections.md b/docs/reference/workloads/collections.md
new file mode 100644
index 00000000..22754933
--- /dev/null
+++ b/docs/reference/workloads/collections.md
@@ -0,0 +1,62 @@
+---
+title: collections
+parent: Workload Reference
+grand_parent: Reference
+nav_order: 65
+---
+
+# collections
+
+The `"collections"` array in `workload.json` defines the Solr collections to create before the benchmark starts.
+
+## Syntax
+
+```json
+{
+ "collections": [
+ {
+ "name": "<collection-name>",
+ "configset-path": "<path>",
+ "num-shards": 1,
+ "replication-factor": 1,
+ "tlog-replicas": 0,
+ "pull-replicas": 0
+ }
+ ]
+}
+```
+
+## Fields
+
+| Field | Type | Required | Default | Description |
+|-------|------|----------|---------|-------------|
+| `name` | string | Yes | — | The collection name. Must be a valid Solr collection name. |
+| `configset-path` | string | No | — | Path relative to the workload directory pointing to a configset directory. If provided, the configset is uploaded to Solr/ZooKeeper before the collection is created. If omitted, the configset named by `configset` (or the collection name) must already exist on the server. |
+| `num-shards` | integer | No | `1` | Number of shards for the collection. |
+| `replication-factor` | integer | No | `1` | Number of NRT (near-real-time) replicas per shard. NRT replicas participate in leader elections. |
+| `tlog-replicas` | integer | No | `0` | Number of TLOG replicas per shard. TLOG replicas buffer updates in a transaction log. |
+| `pull-replicas` | integer | No | `0` | Number of Pull replicas per shard. Pull replicas are read-only and receive index segments from the leader. |
+
+## Example
+
+```json
+{
+ "collections": [
+ {
+ "name": "nyc_taxis",
+ "configset-path": "configsets/nyc_taxis",
+ "num-shards": 2,
+ "replication-factor": 1,
+ "tlog-replicas": 1,
+ "pull-replicas": 0
+ }
+ ]
+}
+```
+
+## Notes
+
+- When `configset-path` is provided, the directory must contain at minimum `schema.xml` and `solrconfig.xml`.
+- For SolrCloud, the configset is uploaded to ZooKeeper before the collection is created.
+- If the collection already exists when the benchmark starts, it is deleted and recreated so that benchmarks are repeatable.
+- See the [Apache Solr Reference Guide: Collections API](https://solr.apache.org/guide/solr/latest/deployment-guide/collections-api.html) for background.
diff --git a/docs/reference/workloads/corpora.md b/docs/reference/workloads/corpora.md
new file mode 100644
index 00000000..3f05b37b
--- /dev/null
+++ b/docs/reference/workloads/corpora.md
@@ -0,0 +1,72 @@
+---
+title: corpora
+parent: Workload Reference
+grand_parent: Reference
+nav_order: 70
+---
+
+# corpora
+
+The `"corpora"` array defines the datasets to index. Each corpus references one or more document files.
+
+## Syntax
+
+```json
+{
+ "corpora": [
+ {
+ "name": "<corpus-name>",
+ "documents": [
+ {
+ "source-file": "<file>",
+ "document-count": <n>,
+ "compressed-bytes": <bytes>,
+ "uncompressed-bytes": <bytes>,
+ "target-collection": "<collection-name>"
+ }
+ ]
+ }
+ ]
+}
+```
+
+## Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | string | Yes | Corpus name, referenced from `bulk-index` operations. |
+| `documents` | array | Yes | List of document file descriptors. |
+| `source-file` | string | Yes | Path (relative to workload dir) to the compressed NDJSON data file. |
+| `document-count` | integer | Yes | Number of documents in the file (used for progress display and `--test-mode` limits). |
+| `compressed-bytes` | integer | No | Compressed file size in bytes (for download progress display). |
+| `uncompressed-bytes` | integer | No | Uncompressed file size in bytes. |
+| `target-collection` | string | No | Target collection name. Defaults to the workload's primary collection. |
+
+## Data file format
+
+Documents must be in gzip-compressed NDJSON (Newline-Delimited JSON) format. Each line is one JSON document. Each document should include an `id` field matching the unique key field in your Solr schema:
+
+```json
+{"id": "1", "title": "My document", "timestamp": "2024-01-01T00:00:00Z"}
+{"id": "2", "title": "Another document", "timestamp": "2024-01-02T00:00:00Z"}
+```
+
+## Example
+
+```json
+{
+ "corpora": [
+ {
+ "name": "nyc_taxis",
+ "documents": [
+ {
+ "source-file": "files/data.json.gz",
+ "document-count": 165346692,
+ "compressed-bytes": 4917851637,
+ "uncompressed-bytes": 74818096036
+ }
+ ]
+ }
+ ]
+}
+```
diff --git a/docs/reference/workloads/index.md b/docs/reference/workloads/index.md
new file mode 100644
index 00000000..0746a0cf
--- /dev/null
+++ b/docs/reference/workloads/index.md
@@ -0,0 +1,10 @@
+---
+title: Workload Reference
+parent: Reference
+nav_order: 60
+has_children: true
+---
+
+# Workload Reference
+
+Complete reference for all `workload.json` keys and structures.
diff --git a/docs/reference/workloads/operations.md b/docs/reference/workloads/operations.md
new file mode 100644
index 00000000..3a757b68
--- /dev/null
+++ b/docs/reference/workloads/operations.md
@@ -0,0 +1,274 @@
+---
+title: operations
+parent: Workload Reference
+grand_parent: Reference
+nav_order: 100
+---
+
+# operations
+
+Operations define the actions performed during a test procedure. They are referenced from test procedure schedules.
+
+## Syntax
+
+Operations can be defined inline in a schedule or in a top-level `"operations"` section:
+
+```json
+{
+ "operations": [
+ {
+ "name": "my-search",
+ "operation-type": "search",
+ "body": {
+ "query": "*:*",
+ "rows": 10
+ }
+ }
+ ]
+}
+```
+
+## Built-in operation types
+
+### bulk-index
+
+```json
+{
+ "operation-type": "bulk-index",
+ "bulk-size": 500,
+ "collection": "my_collection"
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `bulk-size` | `500` | Number of documents per batch |
+| `collection` | (first collection in workload) | Target collection |
+| `corpora` | (all corpora) | Corpus name to index from |
+| `commit` | `false` | If `true`, issue a hard commit to Solr after each batch |
+
+### search
+
+The `search` operation supports two styles.
+
+**JSON body style** — passes a full Solr JSON Request API object:
+
+```json
+{
+ "operation-type": "search",
+ "body": {
+ "query": "*:*",
+ "rows": 10,
+ "fl": "id,title"
+ },
+ "collection": "my_collection"
+}
+```
+
+**Classic params style** — individual Solr query parameters as top-level fields:
+
+```json
+{
+ "operation-type": "search",
+ "q": "city:New York",
+ "fl": "id,name",
+ "rows": 10,
+ "fq": "type:restaurant",
+ "sort": "score desc",
+ "request-params": {
+ "defType": "edismax"
+ },
+ "collection": "my_collection"
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `body` | (none) | Full Solr JSON query body ([JSON Request API](https://solr.apache.org/guide/solr/latest/query-guide/json-request-api.html) format). Takes precedence over classic params if both are present |
+| `q` | `*:*` | Query string (classic params style) |
+| `fl` | (none) | Field list to return |
+| `rows` | (none) | Number of documents to return |
+| `fq` | (none) | Filter query |
+| `sort` | (none) | Sort order |
+| `request-params` | `{}` | Additional Solr query parameters appended to the request |
+| `collection` | (first collection in workload) | Target collection |
+
+### commit
+
+```json
+{ "operation-type": "commit" }
+```
+
+Issues a hard commit to Solr. Also registered as `refresh`.
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `collection` | (first collection in workload) | Target collection |
+| `soft-commit` | `false` | If `true`, issues a soft commit instead of a hard commit |
+
+### optimize
+
+```json
+{ "operation-type": "optimize", "max-segments": 1 }
+```
+
+Issues a force-merge (optimize) to reduce the segment count to `max-segments` (default: 1).
+
+### wait-for-merges
+
+```json
+{ "operation-type": "wait-for-merges" }
+```
+
+Polls the Solr node metrics API until no active merge operations remain across any core, or the timeout is reached.
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `collection` | (first collection in workload) | Target collection |
+| `retry-wait-period` | `2.0` | Seconds between polling attempts |
+| `max-wait-seconds` | `3600` | Maximum seconds to wait before giving up |
+
+### paginated-search
+
+```json
+{
+ "operation-type": "paginated-search",
+ "q": "my query",
+ "rows": 100,
+ "sort": "id asc"
+}
+```
+
+Executes a cursor-paginated Solr search using Solr's `cursorMark` deep pagination API. Fetches all result pages and returns the total document count. Also registered as `scroll-search`.
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `collection` | (first collection in workload) | Target collection |
+| `q` | `*:*` | Query string |
+| `rows` | `100` | Page size (documents per request) |
+| `sort` | `id asc` | Sort order — must include a uniqueKey field for cursor pagination to work |
+| `fl` | (none) | Field list to return |
+| `fq` | (none) | Filter query |
+| `request-params` | `{}` | Additional Solr query parameters |
+
+### create-collection
+
+```json
+{
+ "operation-type": "create-collection",
+ "collection": "my_collection",
+ "configset-path": "configsets/my_schema",
+ "num-shards": 1,
+ "replication-factor": 1
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `collection` | (required) | Collection name |
+| `configset` | (collection name) | Configset name to use; defaults to the collection name |
+| `configset-path` | (none) | Path to local configset directory (relative to workload dir) |
+| `num-shards` | `1` | Number of shards |
+| `replication-factor` | `1` | Number of NRT replicas per shard |
+| `tlog-replicas` | `0` | Number of TLOG replicas per shard |
+| `pull-replicas` | `0` | Number of pull replicas per shard |
+| `delete-configset-on-error` | `true` | Delete uploaded configset if collection creation fails |
+
+### delete-collection
+
+```json
+{
+ "operation-type": "delete-collection",
+ "collection": "my_collection"
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `collection` | (required) | Collection name to delete |
+| `configset` | (collection name) | Configset name to delete alongside the collection |
+| `delete-configset` | `true` | If `true`, also deletes the associated configset |
+| `ignore-missing` | `true` | If `true`, silently succeeds if the collection does not exist |
+
+### raw-request
+
+```json
+{
+ "operation-type": "raw-request",
+ "path": "/api/collections/my_collection/config",
+ "method": "POST",
+ "body": {
+ "set-property": {
+ "updateHandler.autoSoftCommit.maxTime": "5000"
+ }
+ }
+}
+```
+
+Executes an arbitrary HTTP request against the Solr Admin API (`/api/...` V2 endpoints).
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `path` | (required) | API path, e.g. `/api/collections/my_coll/config` |
+| `method` | `GET` | HTTP method: `GET`, `POST`, `DELETE` |
+| `body` | (none) | Request body (JSON object) |
+| `headers` | `{}` | Additional HTTP headers |
+
+### sleep
+
+```json
+{ "operation-type": "sleep", "duration": 5 }
+```
+
+Pauses the schedule for the specified number of seconds. Useful between tasks that need a settling period (for example, after a commit before running queries).
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `duration` | (required) | Seconds to sleep |
+
+### composite
+
+```json
+{
+ "operation-type": "composite",
+ "operations": [
+ { "operation-type": "bulk-index", "bulk-size": 500 },
+ { "operation-type": "commit" }
+ ]
+}
+```
+
+Groups multiple operations into a single logical unit. Operations within the composite execute sequentially; the composite completes when all child operations finish. Useful for measuring end-to-end latency of a multi-step sequence.
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `operations` | (required) | Array of operation definitions to execute in sequence |
+
+### retry wrapper
+
+Any operation can be wrapped with retry logic by adding retry parameters directly to the operation definition:
+
+```json
+{
+ "operation-type": "bulk-index",
+ "bulk-size": 500,
+ "retries": 3,
+ "retry-wait-period": 0.5,
+ "retry-on-timeout": true,
+ "retry-on-error": true
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `retries` | `0` | Number of times to retry on failure |
+| `retry-until-success` | `false` | If `true`, retry indefinitely until the operation succeeds |
+| `retry-wait-period` | `0.5` | Seconds to wait between retry attempts |
+| `retry-on-timeout` | `true` | Retry when a timeout error occurs |
+| `retry-on-error` | `false` | Retry when any other error occurs |
+
+## Backup operations
+
+The following backup-related operation types are registered but not yet implemented in this release:
+`create-backup`, `restore-backup`, `create-backup-repository`, `delete-backup-repository`, `wait-for-backup-create`.
+Use `raw-request` to call the [Solr Backup V2 API](https://solr.apache.org/guide/solr/latest/configuration-guide/backups.html) directly.
diff --git a/docs/reference/workloads/test-procedures.md b/docs/reference/workloads/test-procedures.md
new file mode 100644
index 00000000..64e72db4
--- /dev/null
+++ b/docs/reference/workloads/test-procedures.md
@@ -0,0 +1,75 @@
+---
+title: test-procedures
+parent: Workload Reference
+grand_parent: Reference
+nav_order: 110
+---
+
+# test-procedures
+
+Test procedures define named benchmark scenarios within a workload. A workload typically defines one or more test procedures, each specifying a schedule of operations.
+
+## Syntax
+
+```json
+{
+ "test-procedures": [
+ {
+ "name": "append-no-conflicts",
+ "description": "Index all documents then run searches",
+ "default": true,
+ "schedule": [
+ {
+ "operation": "bulk-index",
+ "warmup-time-period": 120,
+ "clients": 8
+ },
+ { "operation": "commit" },
+ {
+ "operation": "search",
+ "clients": 1,
+ "iterations": 200,
+ "target-throughput": 10
+ }
+ ]
+ }
+ ]
+}
+```
+
+## Fields
+
+| Field | Type | Required | Description |
+|-------|------|----------|-------------|
+| `name` | string | Yes | Test procedure name, referenced with `--test-procedure` at run time |
+| `description` | string | No | Human-readable description |
+| `default` | boolean | No | If `true`, this test procedure is used when `--test-procedure` is not specified |
+| `schedule` | array | Yes | Sequence of schedule items (operations with execution parameters) |
+
+## Schedule item fields
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `operation` | string or object | Operation name (string reference) or inline operation definition (object) |
+| `clients` | integer | Number of parallel clients |
+| `iterations` | integer | Number of times to run the operation |
+| `warmup-iterations` | integer | Iterations to discard before recording metrics |
+| `warmup-time-period` | integer | Seconds to warm up before recording metrics |
+| `target-throughput` | number | Target operations per second (throttle if faster) |
+| `time-period` | integer | Fixed duration in seconds (alternative to `iterations`) |
+
+## Selecting a test procedure at run time
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload my-workload \
+ --test-procedure append-no-conflicts
+```
+
+## Listing available test procedures
+
+```bash
+solr-benchmark info --workload my-workload
+```
diff --git a/docs/user-guide/concepts.md b/docs/user-guide/concepts.md
new file mode 100644
index 00000000..9265e243
--- /dev/null
+++ b/docs/user-guide/concepts.md
@@ -0,0 +1,120 @@
+---
+title: Concepts
+parent: User Guide
+nav_order: 3
+---
+
+# Concepts
+
+## Workloads
+
+A *workload* is the central concept in Apache Solr Benchmark. It defines:
+
+- The **data** to load (corpora — compressed NDJSON files)
+- The **collections** to create and configure
+- The **operations** to run (bulk indexing, search queries, commits, etc.)
+- The **test procedures** that sequence those operations
+
+Workloads are defined in a `workload.json` file. Pre-built workloads for Apache Solr are at [https://github.com/janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads).
+
+## Test Procedures
+
+A *test procedure* is a named configuration within a workload that specifies a particular benchmark scenario. A workload can have multiple test procedures; you select one with `--test-procedure` when running the benchmark.
+
+## Pipelines
+
+A *pipeline* is a sequence of high-level phases that a benchmark run executes:
+
+| Pipeline | Description |
+|----------|-------------|
+| `benchmark-only` | Run against an existing Solr cluster; no provisioning |
+| `docker` | Start a Solr cluster via Docker, then benchmark, then tear down |
+| `from-distribution` | Download and install Solr, benchmark, tear down |
+| `from-sources` | Build Solr from source, install, benchmark, tear down |
+
+## Collections
+
+A *collection* is the Solr equivalent of an OpenSearch index — a logical grouping of documents distributed across shards. Collections are defined in the workload's `"collections"` array and are created before benchmarking begins.
+
+## Configsets
+
+A *configset* is a named set of Solr configuration files (primarily `schema.xml` and `solrconfig.xml`) stored in ZooKeeper. Every collection references a configset. Supply a custom configset in your workload's `configset-path`. See the [Apache Solr Reference Guide](https://solr.apache.org/guide/solr/latest/configuration-guide/configsets.html) for more information.
+
+## Operations
+
+*Operations* are the individual benchmarking actions. Built-in operations include:
+
+| Operation | Description |
+|-----------|-------------|
+| `bulk-index` | Index a batch of documents from a corpus |
+| `search` | Execute a Solr query |
+| `commit` | Issue a hard (or soft) commit to Solr |
+| `optimize` | Issue an optimize (force-merge) command |
+| `wait-for-merges` | Wait until all background merge operations finish |
+| `paginated-search` | Cursor-paginated search using `cursorMark` |
+| `create-collection` | Create a Solr collection |
+| `delete-collection` | Delete a Solr collection |
+| `raw-request` | Execute an arbitrary Solr Admin API request |
+
+## Schedules
+
+A *schedule* controls how an operation executes: number of iterations, target throughput (ops/s), warmup iterations, and parallel client count.
+
+## Corpora
+
+*Corpora* are the datasets used by workloads. Each corpus references one or more data files (gzip-compressed NDJSON). Apache Solr Benchmark downloads corpora from the workload repository or a configured data URL.
+
+## Facets
+
+*Facets* are Solr's aggregation mechanism — the Solr equivalent of OpenSearch aggregations. When using the [Converter Tool](../converter/), OpenSearch aggregation expressions are translated into Solr facet syntax.
+
+---
+
+## Metrics
+
+At the end of each benchmark run, Apache Solr Benchmark prints a summary table and saves it to disk. The table covers these metrics for every task in the challenge:
+
+| Metric | Description |
+|--------|-------------|
+| Throughput | Operations completed per second |
+| Service time | Round-trip time from client request to client receipt of response |
+| Latency | Service time plus any queue waiting time (differs from service time only when `target-throughput` is set) |
+| Error rate | Fraction of operations that returned an error |
+
+### How Apache Solr Benchmark defines service time and latency
+
+These terms are often used interchangeably in the industry but have distinct meanings in Apache Solr Benchmark:
+
+| Metric | Common definition | Apache Solr Benchmark definition |
+|--------|------------------|----------------------------------|
+| **Service time** | Server processing time, excluding network | Time from when the HTTP client sends the request to when it receives the full response — *including* network latency, load balancer overhead, and serialization/deserialization |
+| **Latency** | Service time plus network latency | Service time *plus* any time the request spent waiting in a local queue before being dispatched — only non-zero when `target-throughput` is configured |
+
+### Processing time
+
+Processing time measures the overhead that Apache Solr Benchmark adds during a request — for example, setting up the request context or dispatching to the client library. It is distinct from and excluded from service time measurements. This value is useful for understanding the benchmarking tool's own footprint.
+
+### Service time
+
+Service time is measured from the moment the HTTP client sends the request until the moment it receives the complete response. It includes:
+
+- Network round-trip time
+- Load balancer overhead (if any)
+- Server processing time
+- Serialization and deserialization on both ends
+
+### Latency
+
+Latency is service time plus any time the request spent waiting in a local queue *before* being sent. A queue only builds up when you set `target-throughput` on a task and the cluster cannot keep up with the requested rate. In that case, subsequent requests must wait for an earlier request to complete, adding queue time to the total latency.
+
+When no `target-throughput` is set — or when the cluster can handle every request as fast as they arrive — latency equals service time.
+
+### Throughput
+
+Throughput is the rate at which Apache Solr Benchmark *issues* requests, assuming that responses are returned instantaneously. It is not a measure of how many requests completed; it is a measure of how quickly requests were dispatched.
+
+### The two benchmark modes
+
+**Pure throughput mode** (`target-throughput` not set): Requests are issued as fast as possible — each client sends one request, waits for the response, then sends the next. Latency equals service time.
+
+**Throughput-throttled mode** (`target-throughput` set): Requests are issued at a target rate (in ops/s). If you set a rate higher than the cluster can sustain, requests pile up in the local queue and latency grows. Set `target-throughput` to a value you know is achievable; see [Target throughput](optimizing-benchmarks/target-throughput.html) for practical guidance.
diff --git a/docs/user-guide/index.md b/docs/user-guide/index.md
new file mode 100644
index 00000000..5a62f594
--- /dev/null
+++ b/docs/user-guide/index.md
@@ -0,0 +1,18 @@
+---
+title: User Guide
+nav_order: 5
+has_children: true
+---
+
+# User Guide
+
+The User Guide covers the core concepts and workflows for using Apache Solr Benchmark: from installation and configuration, through understanding the workload format, running benchmarks, and interpreting results.
+
+| Section | Description |
+|---------|-------------|
+| [Concepts](concepts.html) | Core terminology: workloads, challenges, pipelines, collections, metrics |
+| [Install and Configure](install-and-configure/) | Prerequisites, installation, and configuration file |
+| [Understanding Workloads](understanding-workloads/) | Workload anatomy, available workloads, built-in operations |
+| [Working with Workloads](working-with-workloads/) | Running, fine-tuning, creating, and contributing workloads |
+| [Understanding Results](understanding-results/) | Summary reports, metrics definitions, telemetry |
+| [Optimizing Benchmarks](optimizing-benchmarks/) | Target throughput, distributed load, best practices, query randomization |
diff --git a/docs/user-guide/install-and-configure/configuring.md b/docs/user-guide/install-and-configure/configuring.md
new file mode 100644
index 00000000..2ad2124f
--- /dev/null
+++ b/docs/user-guide/install-and-configure/configuring.md
@@ -0,0 +1,148 @@
+---
+title: Configuring
+parent: Install and Configure
+grand_parent: User Guide
+nav_order: 7
+---
+
+# Configuring Apache Solr Benchmark
+
+Apache Solr Benchmark stores its configuration in `~/.solr-benchmark/benchmark.ini`, which is
+automatically created the first time Solr Benchmark runs.
+
+The file is divided into the following sections, which you can customize based on the needs of
+your environment.
+
+## meta
+
+This section contains meta information about the configuration file.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `config.version` | Integer | The version of the configuration file format. This value is managed by Solr Benchmark and should not be changed manually. |
+
+## system
+
+This section contains global information for the current benchmark environment. These settings
+should be identical on all machines where Solr Benchmark is installed.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `env.name` | String | The name of the benchmark environment, used as metadata in result documents. Only alphanumeric characters are allowed. Default is `local`. |
+| `available.cores` | Integer | The number of available CPU cores. Solr Benchmark aims to create one asyncio event loop per core and distributes clients evenly. Defaults to the number of logical CPU cores on the host. |
+| `async.debug` | Boolean | Enables debug mode on Solr Benchmark's asyncio event loop. Default is `false`. |
+| `passenv` | String | A comma-separated list of environment variable names that should be forwarded during the benchmark run. |
+
+## node
+
+This section contains node-specific information that can be customized for your environment.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `root.dir` | String | The directory that stores all Solr Benchmark data. Solr Benchmark assumes full control over this directory and all its subdirectories. Default is `~/.solr-benchmark`. |
+
+## benchmarks
+
+This section configures the Solr Benchmark data directory.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `local.dataset.cache` | String | The directory in which benchmark datasets are stored. Depending on the workloads you run, this directory may contain hundreds of GB of data. Default is `~/.solr-benchmark/benchmarks/data`. |
+
+## reporting
+
+This section defines how benchmark metrics are stored during a run.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `datastore.type` | String | `in-memory` (default) keeps all metrics in RAM for the duration of the run. `filesystem` keeps metrics in RAM and also streams every raw sample to a `metrics.jsonl` file on disk. See [Metrics](../../../reference/metrics/) for details. |
+| `sample.queue.size` | Integer | The number of metric samples that can be held in Solr Benchmark's in-memory queue. Default is `2^20`. |
+| `metrics.request.downsample.factor` | Integer | Saves only every Nth sample. Useful when running benchmarks with many clients to avoid storing excessive data. Default is `1` (all samples saved). |
+| `output.processingtime` | Boolean | If `true`, shows an additional processing time metric in the console report. Default is `false`. |
+
+### Example: filesystem metrics store
+
+```ini
+[reporting]
+datastore.type = filesystem
+```
+
+When `datastore.type` is set to `filesystem`, raw metric documents are written to
+`~/.solr-benchmark/benchmarks/test-runs/<run-id>/metrics.jsonl` incrementally during the run.
+See [Filesystem Metrics Store](../../../reference/metrics/filesystem-metrics-store.html) for
+file layout details and examples for inspecting raw samples.
+
+## workloads
+
+This section defines how workloads are retrieved. Keys follow the format
+`<repository-name>.url`, selectable at run time with `--workload-repository=<repository-name>`.
+
+```ini
+[workloads]
+default.url = https://github.com/janhoy/solr-benchmark-workloads
+```
+
+## defaults
+
+This section defines the default values for certain Solr Benchmark CLI parameters.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `preserve_benchmark_candidate` | Boolean | Determines whether Solr installations are preserved or removed after a benchmark run. To preserve an installation for a single run, use `--preserve-install` on the command line. Default is `false`. |
+
+## distributions
+
+This section configures how Solr distribution archives are managed when using
+`--pipeline=from-distribution`.
+
+| Parameter | Type | Description |
+| :---- | :---- | :---- |
+| `release.cache` | Boolean | Determines whether downloaded Solr release archives are cached locally for reuse. Default is `true`. |
+
+## Proxy configurations
+
+If your host accesses the internet through an HTTP proxy, you can configure Solr Benchmark to
+use it for downloading workloads and distribution archives.
+
+1. Add your proxy URL to your shell profile:
+
+ ```bash
+ export http_proxy=http://proxy.example.org:4444/
+ ```
+
+2. Source your shell profile and verify the proxy URL is set:
+
+ ```bash
+ source ~/.bash_profile ; echo $http_proxy
+ ```
+
+3. Configure Git to use the proxy:
+
+ ```bash
+ git config --global http.proxy $http_proxy
+ ```
+
+4. Verify the proxy is working by cloning the workloads repository:
+
+ ```bash
+ git clone https://github.com/janhoy/solr-benchmark-workloads.git
+ ```
+
+5. Confirm Solr Benchmark picks up the proxy setting by checking the log at
+ `~/.solr-benchmark/logs/benchmark.log`. At startup you should see:
+
+ ```
+ Connecting via proxy URL [http://proxy.example.org:4444/] to the Internet (picked up from the environment variable [http_proxy]).
+ ```
+
+## Logging
+
+Logs from Solr Benchmark can be configured in `~/.solr-benchmark/logging.json`. For details on
+the log file format, see the Python documentation:
+
+- [Python Logging Cookbook](https://docs.python.org/3/howto/logging-cookbook.html) — general tips.
+- [Logging configuration schema](https://docs.python.org/3/library/logging.config.html#logging-config-dictschema) — file format reference.
+- [Logging handlers](https://docs.python.org/3/library/logging.handlers.html) — customizing where log output is written.
+
+By default, Solr Benchmark writes all log output to `~/.solr-benchmark/logs/benchmark.log`.
+Log level is controlled via `~/.solr-benchmark/logging.json`, not via a CLI flag.
diff --git a/docs/user-guide/install-and-configure/index.md b/docs/user-guide/install-and-configure/index.md
new file mode 100644
index 00000000..eae83375
--- /dev/null
+++ b/docs/user-guide/install-and-configure/index.md
@@ -0,0 +1,10 @@
+---
+title: Install and Configure
+parent: User Guide
+nav_order: 5
+has_children: true
+---
+
+# Install and Configure
+
+This section covers installing Apache Solr Benchmark and configuring it for your environment.
diff --git a/docs/user-guide/install-and-configure/installing.md b/docs/user-guide/install-and-configure/installing.md
new file mode 100644
index 00000000..de47aca9
--- /dev/null
+++ b/docs/user-guide/install-and-configure/installing.md
@@ -0,0 +1,181 @@
+---
+title: Installing
+parent: Install and Configure
+grand_parent: User Guide
+nav_order: 5
+---
+
+# Installing Apache Solr Benchmark
+
+You can install Apache Solr Benchmark directly on a host running Linux or macOS. This page
+provides general hardware considerations and step-by-step installation instructions.
+
+## Choosing appropriate hardware
+
+When selecting a host, consider which workloads you want to run. To see a list of available
+benchmark workloads, visit the
+[solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads) repository on
+GitHub. Make sure that the Solr Benchmark host has enough free storage space to store the
+compressed data corpus and the fully decompressed data once benchmarking begins.
+
+Use the following table to estimate the minimum free space required (compressed + uncompressed):
+
+| Workload name | Document count | Compressed size | Uncompressed size |
+| :----: | :----: | :----: | :----: |
+| eventdata | 20,000,000 | 756.0 MB | 15.3 GB |
+| geonames | 11,396,503 | 252.9 MB | 3.3 GB |
+| geopoint | 60,844,404 | 482.1 MB | 2.3 GB |
+| http_logs | 247,249,096 | 1.2 GB | 31.1 GB |
+| noaa | 33,659,481 | 949.4 MB | 9.0 GB |
+| nyc_taxis | 165,346,692 | 4.5 GB | 74.3 GB |
+| pmc | 574,199 | 5.5 GB | 21.7 GB |
+| so | 36,062,278 | 8.9 GB | 33.1 GB |
+
+Your Solr Benchmark host should use solid-state drives (SSDs) for storage. Spinning-disk hard
+drives introduce performance bottlenecks that make benchmark results unreliable.
+{: .tip}
+
+## Prerequisites
+
+Before installing Solr Benchmark, ensure the following software is available on your host:
+
+- **Python 3.10 or later** — required for all pipelines.
+- **pip** — Python package manager.
+- **Git 2.3 or later** — required to fetch workloads from a remote repository.
+- **Docker** — required for the `--pipeline=docker` pipeline, which starts a Solr cluster
+ automatically before the run.
+- **JDK 21** — required for the `--pipeline=from-distribution` pipeline, which downloads and
+ installs a Solr release locally.
+- **pbzip2** *(optional)* — parallel bzip2 decompressor for faster decompression of `.bz2`
+ corpora. Install via `apt install pbzip2` or `brew install pbzip2`. If absent, Solr Benchmark
+ falls back to Python's standard `bz2` library automatically (slower but fully functional).
+
+### Checking software dependencies
+
+Use [pyenv](https://github.com/pyenv/pyenv) to manage multiple versions of Python on your host.
+This is especially useful if your system Python is older than 3.10.
+{: .tip}
+
+- Check that Python 3.10 or later is installed:
+
+ ```bash
+ python3 --version
+ ```
+
+- Check that `pip` is installed and functional:
+
+ ```bash
+ pip --version
+ ```
+
+- Check that Git 2.3 or later is installed:
+
+ ```bash
+ git --version
+ ```
+
+## Installing on Linux and macOS
+
+Apache Solr Benchmark is not yet published on PyPI. Install it directly from the source
+repository.
+{: .note}
+
+Clone the repository and install in editable mode:
+
+```bash
+git clone https://github.com/janhoy/solr-benchmark.git
+cd solr-benchmark
+pip install -e .
+```
+
+After the installation completes, verify it is working:
+
+```bash
+solr-benchmark --version
+```
+
+### Virtual environment (recommended)
+
+Install Solr Benchmark inside a virtual environment to avoid dependency conflicts with other
+Python packages on your system:
+
+```bash
+git clone https://github.com/janhoy/solr-benchmark.git
+cd solr-benchmark
+python3 -m venv .venv
+source .venv/bin/activate
+pip install -e .
+```
+
+### Developer install
+
+To also install development and test dependencies:
+
+```bash
+pip install -e ".[develop]"
+```
+
+### Upgrading
+
+To pick up the latest changes, pull from the repository and reinstall:
+
+```bash
+cd solr-benchmark
+git pull
+pip install -e .
+```
+
+## Starting a Solr cluster for benchmarking
+
+Solr Benchmark can start and stop a Solr cluster for you as part of a benchmark run using two
+built-in pipelines:
+
+- `--pipeline=docker` — pulls the official `solr` Docker image and starts a single-node Solr
+ cluster before the run. No JDK is required.
+- `--pipeline=from-distribution` — downloads a Solr release archive, installs it locally, and
+ starts a cluster. JDK 21 must be available on the host.
+
+If you already have a running Solr cluster, use `--pipeline=benchmark-only` and point Solr
+Benchmark at it with `--target-hosts`.
+
+See the [run command reference](../../../reference/commands/run.html) for the full list of
+pipeline options and flags.
+
+## Directory structure
+
+After running Solr Benchmark for the first time, all related files are stored under
+`~/.solr-benchmark/`:
+
+```
+~/.solr-benchmark/
+├── benchmark.ini
+├── benchmarks/
+│ ├── data/
+│ │ └── nyc_taxis/
+│ ├── distributions/ # populated by --pipeline=from-distribution
+│ │ └── solr-9.10.1.tgz
+│ ├── test-runs/
+│ │ └── <run-id>/
+│ │ └── test_run.json
+│ └── workloads/
+│ └── default/
+│ └── nyc_taxis/
+├── logging.json
+└── logs/
+ └── benchmark.log
+```
+
+- **`benchmark.ini`** — main configuration file. See [Configuring](configuring.html).
+- **`benchmarks/data/`** — downloaded and decompressed workload data corpora.
+- **`benchmarks/distributions/`** — cached Solr release archives (only present when using the
+ `from-distribution` pipeline).
+- **`benchmarks/test-runs/`** — one subdirectory per run, each containing `test_run.json` with
+ the computed results for that run.
+- **`benchmarks/workloads/`** — cached workload definitions fetched from the workload repository.
+- **`logging.json`** — logging configuration. See [Logging](configuring.html#logging).
+- **`logs/`** — benchmark run logs, useful for diagnosing errors.
+
+## Next steps
+
+- [Configuring](configuring.html) — customize `benchmark.ini` for your environment.
+- [Running workloads](../working-with-workloads/running-workloads.html) — run your first full benchmark.
diff --git a/docs/user-guide/optimizing-benchmarks/best-practices.md b/docs/user-guide/optimizing-benchmarks/best-practices.md
new file mode 100644
index 00000000..a190edc4
--- /dev/null
+++ b/docs/user-guide/optimizing-benchmarks/best-practices.md
@@ -0,0 +1,108 @@
+---
+title: Performance Testing Best Practices
+parent: Optimizing Benchmarks
+grand_parent: User Guide
+nav_order: 20
+---
+
+# Performance Testing Best Practices
+
+Following consistent methodology produces benchmark results you can trust and compare over time. This page summarizes the most important practices for running Apache Solr Benchmark.
+
+## Use a dedicated, stable environment
+
+Benchmark results are only meaningful when the environment is consistent across runs. Avoid:
+
+- Running other workloads on the benchmark host or the Solr nodes during the test
+- Sharing the Solr cluster with production traffic
+- Running benchmarks on laptops or developer machines (thermal throttling, background processes)
+
+Use dedicated hardware or fixed-size cloud instances, and keep the OS, JVM, and Solr version pinned.
+
+## Always warm up before measuring
+
+Cold runs produce misleading results because:
+- The JVM has not JIT-compiled the hot code paths
+- Solr's filter cache and query result cache are empty
+- OS page cache may not contain index data
+
+Use `warmup-iterations` or `warmup-time-period` in your test procedure to discard the first N operations from measurement:
+
+```json
+{
+ "operation": "search",
+ "clients": 4,
+ "warmup-iterations": 500,
+ "iterations": 2000
+}
+```
+
+A warm-up of 100–500 iterations (or 30–60 seconds) is typical for query tasks. For indexing, allow at least one full merge cycle to complete before measuring.
+
+## Run multiple iterations and aggregate
+
+A single benchmark run can be influenced by transient noise (GC pause, OS scheduling, network blip). Run the workload three or more times and use `solr-benchmark aggregate` to combine the results:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --test-iterations 3 \
+ --aggregate true \
+ --sleep-timer 60
+```
+
+The aggregated output includes the mean relative standard deviation (RSD). An RSD below 5% indicates good result stability.
+
+## Tag your runs
+
+Use `--user-tag` to attach metadata to each run. This makes it easy to filter results when comparing configurations:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --user-tag "heap:4g,gc:g1gc,version:9.10.1"
+```
+
+## Use --test-mode before committing to a full run
+
+Before running a multi-hour benchmark, validate your workload with `--test-mode` (≤1,000 documents, abbreviated schedule). This catches configuration errors quickly without wasting time.
+
+## Let the cluster reach steady state between runs
+
+After indexing large datasets, Solr may continue background merging for minutes or even hours. Wait for merge activity to subside (check the Solr admin UI or the `solr-indexing-stats` telemetry output) before starting the query-focused portion of your benchmark.
+
+## Capture telemetry alongside results
+
+Enable at least `solr-jvm-stats` and `solr-node-stats` on every run:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --telemetry solr-jvm-stats,solr-node-stats
+```
+
+JVM GC pauses are a common explanation for latency spikes. Having telemetry data lets you correlate benchmark metrics with system-level events.
+
+## Compare, don't just measure
+
+A single number is hard to interpret. Use `solr-benchmark compare` to evaluate whether a configuration change improved or regressed performance:
+
+```bash
+solr-benchmark compare \
+ --baseline 20260101_120000_abc12345 \
+ --contender 20260220_143052_a34ff090
+```
+
+A negative diff in latency and a positive diff in throughput indicate improvement.
+
+## Document your setup
+
+For every benchmark, record:
+- Solr version and configuration (number of shards, replicas, heap size)
+- Hardware specs (CPU, RAM, disk type)
+- Workload name, challenge, and key parameters
+- Any non-default JVM flags or OS tuning
+
+Use `--user-tag` to capture the key facts inside the result file itself.
diff --git a/docs/user-guide/optimizing-benchmarks/index.md b/docs/user-guide/optimizing-benchmarks/index.md
new file mode 100644
index 00000000..35f8e9c5
--- /dev/null
+++ b/docs/user-guide/optimizing-benchmarks/index.md
@@ -0,0 +1,17 @@
+---
+title: Optimizing Benchmarks
+parent: User Guide
+nav_order: 55
+has_children: true
+---
+
+# Optimizing Benchmarks
+
+This section provides guidance for getting the most meaningful and reliable results from Apache Solr Benchmark.
+
+| Topic | Description |
+|-------|-------------|
+| [Target throughput](target-throughput.html) | Understand how `target-throughput` affects latency measurements and how to set it correctly |
+| [Running distributed load](running-distributed-load.html) | Generate load from multiple machines to saturate large Solr clusters |
+| [Performance testing best practices](best-practices.html) | Methodology for repeatable, meaningful benchmark results |
+| [Randomizing queries](randomizing-queries.html) | Vary query parameters across iterations to simulate real-world cache behavior |
diff --git a/docs/user-guide/optimizing-benchmarks/randomizing-queries.md b/docs/user-guide/optimizing-benchmarks/randomizing-queries.md
new file mode 100644
index 00000000..559d4ec6
--- /dev/null
+++ b/docs/user-guide/optimizing-benchmarks/randomizing-queries.md
@@ -0,0 +1,112 @@
+---
+title: Randomizing Queries
+parent: Optimizing Benchmarks
+grand_parent: User Guide
+nav_order: 40
+---
+
+# Randomizing Queries
+
+By default, a workload runs the same fixed queries on every iteration. This produces unrealistically optimistic latency numbers because Solr's filter cache and query result cache will be warm after the first pass — subsequent identical queries hit the cache and complete much faster than they would in production.
+
+Randomizing queries generates varied parameter values across iterations, so each run exercises a realistic mix of cache hits and misses.
+
+## How it works
+
+Apache Solr Benchmark uses a **Zipf probability distribution** to model realistic cache behavior:
+
+1. At benchmark startup, N value pairs are generated and stored in an indexed list.
+2. For each operation, the benchmark probabilistically decides whether to reuse a stored pair (cache hit scenario) or generate a new random pair (cache miss scenario).
+3. The **repeat frequency** (`rf`, 0.0–1.0) controls the maximum fraction of queries that reuse stored values.
+
+With the default settings (`rf=0.3`, `N=5000`), 30% of queries reuse stored value pairs (likely cache hits) and 70% generate fresh random values (likely cache misses).
+
+## Implementing randomized queries in a workload
+
+Randomization requires a `workload.py` file in your workload directory. This file registers functions that generate random parameter values.
+
+### Example: randomizing range query parameters
+
+```python
+import random
+
+def random_fare_range(max_value):
+ gte_cents = random.randrange(0, max_value * 100)
+ lte_cents = random.randrange(gte_cents, max_value * 100)
+ return {
+ "gte": gte_cents / 100,
+ "lte": lte_cents / 100,
+ }
+
+def fare_range_value_source():
+ return random_fare_range(120.00)
+
+def register(registry):
+ registry.register_standard_value_source(
+ "range", # query type
+ "fare_amount", # field name
+ fare_range_value_source,
+ )
+```
+
+The `register` function is called once at startup. The `register_standard_value_source` call tells the benchmark: "when running a `range` query on the `fare_amount` field, use this function to generate parameter values."
+
+### Example: randomizing non-range queries
+
+For queries that are not range queries, use `register_query_randomization_info`:
+
+```python
+def register(registry):
+ registry.register_query_randomization_info(
+ "bbox", # operation name in the workload
+ "geo_bounding_box", # Solr query type
+ [["top_left"], ["bottom_right"]], # parameter variants
+ [], # optional parameters
+ )
+```
+
+## CLI flags
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--randomization-enabled` | `false` | Activate query randomization |
+| `--randomization-repeat-frequency` | `0.3` | Fraction of queries that reuse stored value pairs (0.0–1.0) |
+| `--randomization-n` | `5000` | Number of value pairs to generate at startup |
+| `--randomization-alpha` | `1.0` | Zipf distribution alpha (≥ 0); higher values skew selection toward lower-indexed pairs |
+
+## Enabling randomization at runtime
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --randomization-enabled true \
+ --randomization-repeat-frequency 0.2 \
+ --randomization-n 10000
+```
+
+## Choosing the right repeat frequency
+
+| rf value | Interpretation |
+|----------|---------------|
+| `0.0` | Every query is unique — maximum cache miss rate |
+| `0.3` | 30% reuse (default) — models typical mixed workloads |
+| `1.0` | All queries reuse stored pairs — maximum cache hit rate |
+
+Set `rf` to match your production cache hit ratio if you know it. If you don't know it, the default of `0.3` is a reasonable starting point.
+
+## The Zipf distribution
+
+The probability of selecting value pair *i* from the stored list follows the Zipf distribution: *P(i) ∝ 1/i^α*. This means:
+
+- The first stored pair is selected most frequently
+- Frequency drops off sharply for higher-indexed pairs
+- `alpha=1.0` (default) gives the standard Zipf distribution
+- Higher `alpha` increases the skew (more of the probability mass on the first few pairs)
+- `alpha=0.0` makes all stored pairs equally likely
+
+## See also
+
+- [Fine-tuning workloads](../working-with-workloads/finetune-workloads.html)
+- [Creating custom workloads](../working-with-workloads/creating-custom-workloads.html)
diff --git a/docs/user-guide/optimizing-benchmarks/running-distributed-load.md b/docs/user-guide/optimizing-benchmarks/running-distributed-load.md
new file mode 100644
index 00000000..31b520ae
--- /dev/null
+++ b/docs/user-guide/optimizing-benchmarks/running-distributed-load.md
@@ -0,0 +1,105 @@
+---
+title: Running Distributed Load
+parent: Optimizing Benchmarks
+grand_parent: User Guide
+nav_order: 30
+---
+
+# Running Distributed Load
+
+By default, Apache Solr Benchmark generates all load from the machine where you run the `solr-benchmark` command. For large clusters or high-throughput benchmarks, a single machine may become the bottleneck before Solr does. Distributed load generation lets you spread the workload across multiple machines.
+
+## Architecture
+
+In distributed mode, one machine acts as the **coordinator** and one or more additional machines act as **workers**. The coordinator:
+
+- Parses and distributes the workload
+- Sends task assignments to each worker
+- Collects and aggregates metrics from all workers
+
+Each worker:
+- Executes its assigned portion of the load against the Solr cluster
+- Sends metrics back to the coordinator
+
+The coordinator and workers communicate over the network. All machines must be able to reach each other and the Solr cluster.
+
+## Prerequisites
+
+- The same version of Apache Solr Benchmark must be installed on all coordinator and worker machines
+- All machines must have network access to the Solr cluster
+- Workers must have the workload data files available at the same path as the coordinator (or accessible via a shared filesystem)
+
+## Configuration
+
+### Start the benchmark daemon on each worker
+
+On each worker machine, start the benchmark daemon:
+
+```bash
+solr-benchmarkd start --node-ip WORKER_IP --coordinator-ip COORDINATOR_IP
+```
+
+Replace `WORKER_IP` with the IP address of that worker machine and `COORDINATOR_IP` with the IP address of the machine that will run `solr-benchmark run`.
+
+### Run the benchmark from the coordinator
+
+On the coordinator machine, pass the worker IPs via `--worker-ips`:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts solr1:8983,solr2:8983,solr3:8983 \
+ --worker-ips 192.168.1.10,192.168.1.11,192.168.1.12
+```
+
+The coordinator automatically divides the corpus and task schedule across the specified workers.
+
+## How load is divided
+
+The corpus is partitioned by line ranges in the NDJSON data files. Each worker receives a non-overlapping slice of documents to index. For query tasks, each worker runs the full query schedule with the specified `clients` count, so the effective query rate is `clients × number_of_workers`.
+
+## Example: 3-worker setup
+
+```bash
+# On worker-1 (192.168.1.10):
+solr-benchmarkd start --node-ip 192.168.1.10 --coordinator-ip 192.168.1.1
+
+# On worker-2 (192.168.1.11):
+solr-benchmarkd start --node-ip 192.168.1.11 --coordinator-ip 192.168.1.1
+
+# On worker-3 (192.168.1.12):
+solr-benchmarkd start --node-ip 192.168.1.12 --coordinator-ip 192.168.1.1
+
+# On the coordinator (192.168.1.1):
+solr-benchmark run \
+ --workload nyc_taxis \
+ --pipeline benchmark-only \
+ --target-hosts solr1:8983,solr2:8983 \
+ --worker-ips 192.168.1.10,192.168.1.11,192.168.1.12 \
+ --user-tag "workers:3"
+```
+
+## Stopping workers
+
+When the benchmark is complete, stop each worker daemon:
+
+```bash
+solr-benchmarkd stop --node-ip 192.168.1.10 --coordinator-ip 192.168.1.1
+```
+
+Check daemon status on any worker:
+
+```bash
+solr-benchmarkd status
+```
+
+See the [solr-benchmarkd reference](../../reference/commands/benchmarkd.html) for full daemon documentation.
+
+## When to use distributed load
+
+Consider distributed load generation when:
+
+- A single machine cannot sustain the target query rate (CPU or network saturated on the load generator, not on Solr)
+- You want to simulate many independent clients connecting from different source IPs
+- Your indexing throughput is limited by the speed of reading and sending documents from the coordinator
diff --git a/docs/user-guide/optimizing-benchmarks/target-throughput.md b/docs/user-guide/optimizing-benchmarks/target-throughput.md
new file mode 100644
index 00000000..844f5a07
--- /dev/null
+++ b/docs/user-guide/optimizing-benchmarks/target-throughput.md
@@ -0,0 +1,79 @@
+---
+title: Target Throughput
+parent: Optimizing Benchmarks
+grand_parent: User Guide
+nav_order: 10
+---
+
+# Target Throughput
+
+`target-throughput` is one of the most important — and most misunderstood — parameters in Apache Solr Benchmark. Getting it right is key to measuring meaningful latency.
+
+## What target-throughput means
+
+`target-throughput` sets the rate at which Apache Solr Benchmark issues requests, measured in operations per second (ops/s). It does **not** guarantee that Solr will complete requests at that rate — it only controls how fast the benchmark sends them.
+
+## Two benchmark modes
+
+### Pure throughput mode (no target-throughput)
+
+When `target-throughput` is not set (or set to `0`), each benchmark client sends one request, waits for the response, then immediately sends the next. The benchmark drives load as fast as the cluster can handle.
+
+In this mode **latency equals service time** — there is no queue wait because every request is dispatched as soon as the previous one completes.
+
+Use this mode to find the maximum throughput your Solr cluster can sustain.
+
+### Throughput-throttled mode (target-throughput set)
+
+When `target-throughput` is set, the benchmark issues requests at a fixed rate regardless of how fast responses arrive. If a request takes longer than the inter-request interval, the next request must wait in a local queue. That wait time is added to the latency measurement.
+
+**Latency = service time + queue wait time**
+
+Use this mode to measure how your cluster behaves under a *specific, realistic load* — for example, the query rate your production system typically receives.
+
+## Why setting target-throughput too high is dangerous
+
+If you set `target-throughput` higher than what Solr can actually sustain, the request queue grows without bound. Each successive request waits longer than the previous one, and measured latency climbs continuously — but this inflation is an artifact of the benchmark configuration, not a property of the cluster.
+
+**Example:** Suppose individual search requests take 200 ms (5 ops/s max throughput). If you set `target-throughput: 10`, the benchmark tries to send a request every 100 ms. After the first request, a second request is already queued. It waits 100 ms before sending, adding 100 ms to its latency. By the tenth request, there is 900 ms of accumulated queue wait. This grows without limit as the run continues.
+
+## How to set target-throughput correctly
+
+1. Run the workload once **without** `target-throughput` to establish the maximum sustainable throughput.
+2. Set `target-throughput` to a value **below** the maximum — for example, 70–80% of the observed peak.
+3. Re-run and verify that latency stays stable throughout the run (not climbing over time). A stable latency indicates the queue is not growing.
+
+### Checking for queue buildup
+
+If latency climbs steadily from the start of the run to the end, your `target-throughput` is too high. Reduce it until latency stabilizes.
+
+## Setting target-throughput in a workload
+
+Set `target-throughput` in the schedule section of your workload's test procedure:
+
+```json
+{
+ "schedule": [
+ {
+ "operation": "search",
+ "clients": 4,
+ "target-throughput": 20
+ }
+ ]
+}
+```
+
+This example issues 20 search requests per second across 4 clients (5 requests per client per second).
+
+You can also override it at runtime without editing the workload file:
+
+```bash
+solr-benchmark run \
+ --workload nyc_taxis \
+ --workload-params "search_target_throughput:20"
+```
+
+## See also
+
+- [Concepts — Metrics](../concepts.html#metrics) — definitions of latency, service time, and throughput
+- [Fine-tuning workloads](../working-with-workloads/finetune-workloads.html) — other schedule parameters
diff --git a/docs/user-guide/understanding-results/index.md b/docs/user-guide/understanding-results/index.md
new file mode 100644
index 00000000..d4997b60
--- /dev/null
+++ b/docs/user-guide/understanding-results/index.md
@@ -0,0 +1,10 @@
+---
+title: Understanding Results
+parent: User Guide
+nav_order: 20
+has_children: true
+---
+
+# Understanding Results
+
+How to read the benchmark output, interpret summary reports, and enable telemetry to collect JVM and Solr node metrics during a run.
diff --git a/docs/user-guide/understanding-results/summary-reports.md b/docs/user-guide/understanding-results/summary-reports.md
new file mode 100644
index 00000000..e613f525
--- /dev/null
+++ b/docs/user-guide/understanding-results/summary-reports.md
@@ -0,0 +1,136 @@
+---
+title: Summary Reports
+parent: Understanding Results
+grand_parent: User Guide
+nav_order: 22
+---
+
+# Understanding the Summary Report
+
+At the end of each test run, Apache Solr Benchmark prints a summary report to the console and saves the results to disk. The report shows metrics such as throughput, latency, service time, and error rate for each task in the challenge.
+
+## Console output
+
+The summary table is printed after all tasks complete. The following is a real example from a `nyc_taxis` workload run in `--test-mode` against Apache Solr 9.10.1:
+
+```
+------------------------------------------------------
+ _______ __ _____
+ / ____(_)___ ____ _/ / / ___/_________ ________
+ / /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
+ / __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
+/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
+------------------------------------------------------
+
+| Metric | Task | Value | Unit |
+|------------------------------:|--------------------------:|--------:|-------:|
+| Min Throughput | delete-collection | 10.8 | ops/s |
+| Mean Throughput | delete-collection | 10.8 | ops/s |
+| Median Throughput | delete-collection | 10.8 | ops/s |
+| Max Throughput | delete-collection | 10.8 | ops/s |
+| 100th percentile latency | delete-collection | 88.6857 | ms |
+| 100th percentile service time | delete-collection | 88.6857 | ms |
+| error rate | delete-collection | 0 | % |
+| Min Throughput | create-collection | 1.16 | ops/s |
+| Mean Throughput | create-collection | 1.16 | ops/s |
+| Median Throughput | create-collection | 1.16 | ops/s |
+| Max Throughput | create-collection | 1.16 | ops/s |
+| 100th percentile latency | create-collection | 865.011 | ms |
+| 100th percentile service time | create-collection | 865.011 | ms |
+| error rate | create-collection | 0 | % |
+| Min Throughput | check-cluster-health | 84.98 | ops/s |
+| Mean Throughput | check-cluster-health | 84.98 | ops/s |
+| Median Throughput | check-cluster-health | 84.98 | ops/s |
+| Max Throughput | check-cluster-health | 84.98 | ops/s |
+| 100th percentile latency | check-cluster-health | 9.81933 | ms |
+| 100th percentile service time | check-cluster-health | 9.81933 | ms |
+| error rate | check-cluster-health | 0 | % |
+| Min Throughput | index | 3860.9 | docs/s |
+| Mean Throughput | index | 3860.9 | docs/s |
+| Median Throughput | index | 3860.9 | docs/s |
+| Max Throughput | index | 3860.9 | docs/s |
+| 50th percentile latency | index | 177.464 | ms |
+| 100th percentile latency | index | 204.121 | ms |
+| 50th percentile service time | index | 177.464 | ms |
+| 100th percentile service time | index | 204.121 | ms |
+| error rate | index | 0 | % |
+```
+
+## Metrics explained
+
+### Throughput
+
+Throughput measures the rate at which operations were completed during the task. Apache Solr Benchmark reports four values:
+
+| Metric | Description |
+|--------|-------------|
+| Min Throughput | The lowest throughput observed across all measurement samples |
+| Mean Throughput | The arithmetic mean throughput across all measurement samples |
+| Median Throughput | The 50th-percentile throughput |
+| Max Throughput | The highest throughput observed |
+
+The unit depends on the task type:
+- **`docs/s`** — documents per second, for indexing tasks
+- **`ops/s`** — operations per second, for all other tasks (collection management, queries, etc.)
+
+In the example above, the `index` task achieved ~3,861 docs/s. The `check-cluster-health` task ran at ~85 ops/s, and `create-collection` ran at 1.16 ops/s — which makes sense, since collection creation in SolrCloud involves ZooKeeper coordination and takes nearly a second.
+
+### Latency
+
+**Latency** is the total elapsed time from when the client submitted the request to when it received the complete response. This includes any time the request spent waiting in a queue before it was dispatched to the server, plus the server processing time.
+
+Latency percentiles show the distribution of response times across all operations in the task:
+
+- **50th percentile latency** — half of all requests completed within this time
+- **100th percentile latency** — the slowest request observed (the worst case)
+
+Higher percentiles (99th, 99.9th) appear in full runs; in `--test-mode` only the 50th and 100th are typically shown due to the small operation count.
+
+### Service time
+
+**Service time** is the time the server spent actively processing the request, excluding any client-side queuing wait. It is always ≤ latency.
+
+When Apache Solr Benchmark runs operations at maximum throughput (no `target-throughput` set), or when there is only one operation in flight at a time, latency and service time will be equal — as seen in the example above. The difference becomes visible under load when a schedule produces more requests than the cluster can immediately process.
+
+### Error rate
+
+The error rate is the fraction of operations that returned an error response, expressed as a percentage. An error rate of `0` means every operation succeeded. Any non-zero error rate should be investigated before treating the throughput and latency figures as valid.
+
+## Result files
+
+Results are saved to `~/.solr-benchmark/benchmarks/test-runs/` after each run:
+
+```
+~/.solr-benchmark/benchmarks/test-runs/
+└── 20260220_143052_a34ff090/
+ ├── test_run.json ← complete metrics record (JSON)
+ ├── results.csv ← key metrics in CSV format
+ └── summary.txt ← the same table printed to the console
+```
+
+The directory name combines the run timestamp and a short unique ID. The `test_run.json` file contains all per-task metrics, workload metadata, cluster configuration, and environment information. See [Summary Report Format](../../reference/summary-report.html) for the full schema.
+
+### Controlling result output
+
+The following flags on the `run` command control how results are stored and displayed:
+
+| Flag | Description |
+|------|-------------|
+| `--results-file` | Write the summary table to a specific file path in addition to the default location |
+| `--results-format` | Output format: `markdown` (default) or `csv` |
+| `--show-in-results` | Which values to include: `available` (default), `all-percentiles`, or `all` |
+| `--user-tag` | Attach key:value metadata to the run, e.g. `--user-tag "baseline:true,heap:4g"` — useful for filtering results when comparing runs |
+
+## Comparing runs
+
+Use `solr-benchmark compare` to diff two runs by their run IDs:
+
+```bash
+solr-benchmark compare \
+ --baseline 20260101_120000_abc12345 \
+ --contender 20260220_143052_a34ff090
+```
+
+The output shows the delta for each metric, making it easy to spot regressions or improvements between configurations or Solr versions.
+
+See [compare](../../reference/commands/compare.html) for full documentation.
diff --git a/docs/user-guide/understanding-results/telemetry.md b/docs/user-guide/understanding-results/telemetry.md
new file mode 100644
index 00000000..88e5892a
--- /dev/null
+++ b/docs/user-guide/understanding-results/telemetry.md
@@ -0,0 +1,94 @@
+---
+title: Enabling Telemetry
+parent: Understanding Results
+grand_parent: User Guide
+nav_order: 30
+---
+
+# Enabling Telemetry
+
+Apache Solr Benchmark can collect server-side metrics from your Solr cluster during a benchmark
+run using *telemetry devices*. Devices are split into two groups:
+
+- **Always-enabled devices** — active on every run, no flag needed.
+- **Optional devices** — enabled on demand with `--telemetry <name>`.
+
+## Always-enabled devices
+
+These six devices are active on every benchmark run:
+
+| Device | Description |
+|--------|-------------|
+| `solr-jvm-stats` | JVM heap, GC pause times, GC counts, threads, buffer pools |
+| `solr-node-stats` | CPU, OS memory, file descriptors, HTTP requests, query handler metrics |
+| `solr-collection-stats` | Per-collection document counts, deleted docs, segment counts, index size |
+| `solr-query-stats` | Query latency percentiles (p50/p99/p99.9), request counts, cache hit ratios |
+| `solr-indexing-stats` | Indexing throughput, error counts, merge activity |
+| `solr-cache-stats` | Per-cache hit/miss/eviction counts and memory usage |
+
+## Optional telemetry devices
+
+Enable optional devices by passing `--telemetry` with a comma-separated list of device names:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --telemetry segment-stats,shard-stats,cluster-environment-info
+```
+
+### Optional devices — all pipelines
+
+REST-based devices that work with any pipeline, including `benchmark-only`:
+
+| Device | Description |
+|--------|-------------|
+| `segment-stats` | Per-collection segment count, doc count, deleted doc count, and index size via the Luke API; written to `segment_stats.log` on benchmark stop |
+| `shard-stats` | Per-shard document count and index size polled from CLUSTERSTATUS + Core STATUS; SolrCloud only — skipped silently on standalone Solr |
+| `cluster-environment-info` | Records Solr version, JVM version, JVM vendor, and CPU core count as run metadata once at benchmark start |
+
+### Optional devices — provisioned pipelines only
+
+JVM and process devices that inject flags into `SOLR_OPTS` before Solr starts. Require the
+`docker` or `from-distribution` pipeline. They are silently skipped on `benchmark-only`.
+
+| Device | Description |
+|--------|-------------|
+| `jfr` | Java Flight Recorder — writes `profile.jfr` to the log directory (requires OpenJDK 11+) |
+| `gc` | GC logging (`-Xlog:` format) — writes `gc.log` to the log directory |
+| `jit` | JIT compiler logging — writes `jit.log` to the log directory |
+| `heapdump` | Heap dump (`jmap -dump`) on benchmark stop — writes `heap_at_exit_{pid}.hprof` |
+| `disk-io` | Disk I/O bytes read/written by the Solr process (always active on provisioned pipelines) |
+
+**Note:** JVM devices (jfr, gc, jit, heapdump, disk-io) silently skip when pipeline is
+`benchmark-only` — no warning is shown in the results.
+{: .note}
+
+## When to use optional devices
+
+- **`segment-stats`** — Diagnose segment fragmentation. Run after bulk indexing to see whether
+ force-merging improved segment count and deleted doc ratio.
+- **`shard-stats`** — Verify document distribution across shards in a SolrCloud cluster. Useful
+ for detecting hot shards or uneven distribution.
+- **`cluster-environment-info`** — Audit metadata. Adds Solr version and JVM info to every run
+ record so you can compare results across Solr versions.
+- **`jfr`, `gc`, `jit`** — Profiling during provisioned benchmark runs. Use these when
+ diagnosing JVM performance issues (GC pressure, JIT regressions, etc.).
+- **`heapdump`** — Capture a heap snapshot on benchmark completion. Use for offline memory
+ analysis with tools like Eclipse MAT or VisualVM.
+
+## Telemetry output
+
+Telemetry metrics are included in the `results.json` file alongside the workload operation
+metrics. They are also printed as additional rows in the console summary table.
+
+See [Telemetry Devices](../../reference/telemetry.html) for full device documentation, metric
+names, and configuration options.
+
+## Solr version compatibility
+
+Both Solr 9.x and Solr 10.x expose metrics at `/solr/admin/metrics`. The response format
+differs: Solr 9.x returns custom JSON, Solr 10.x returns Prometheus text format. Apache Solr
+Benchmark auto-detects the format via the HTTP `Content-Type` header at runtime. No
+configuration is required.
diff --git a/docs/user-guide/understanding-workloads/anatomy-of-a-workload.md b/docs/user-guide/understanding-workloads/anatomy-of-a-workload.md
new file mode 100644
index 00000000..3fdb0630
--- /dev/null
+++ b/docs/user-guide/understanding-workloads/anatomy-of-a-workload.md
@@ -0,0 +1,419 @@
+---
+title: Anatomy of a Workload
+parent: Understanding Workloads
+grand_parent: User Guide
+nav_order: 15
+---
+
+# Anatomy of a workload
+
+A workload is a directory that describes a complete benchmark scenario: which Solr collection
+to create, what data to index, and which operations to run. All workload files are processed
+as [Jinja2](https://jinja.palletsprojects.com/) templates before being parsed as JSON, which
+allows workload authors to parametrize any value and let users override it at run time.
+
+A workload contains the following files and directories:
+
+- [workload.json](#workloadjson) — the main descriptor: collections, corpora, and the default schedule.
+- [configsets/](#configsets) — Solr configset directories, each containing `schema.xml` and `solrconfig.xml`.
+- [files/](#filestxt) — compressed NDJSON data corpora.
+- [operations/](#operations-and-test-procedures) — named operation definitions referenced from schedules.
+- [test_procedures/](#operations-and-test-procedures) — named test procedures.
+
+## Workload directory structure
+
+```
+my-workload/
+├── workload.json # Main descriptor
+├── operations/
+│ └── default.json # Named operation definitions
+├── test_procedures/
+│ └── default.json # Named test procedures
+├── files/
+│ └── data.json.gz # Corpus data (gzip-compressed NDJSON)
+└── configsets/
+ └── my-schema/
+ ├── schema.xml
+ └── solrconfig.xml
+```
+
+## workload.json
+
+The following example shows all the essential elements of a `workload.json` file:
+
+```json
+{
+ "description": "NYC taxi ride benchmark for Apache Solr",
+ "collections": [
+ {
+ "name": "nyc_taxis",
+ "configset-path": "configsets/nyc_taxis",
+ "shards": 1,
+ "nrt_replicas": 1
+ }
+ ],
+ "corpora": [
+ {
+ "name": "nyc_taxis",
+ "documents": [
+ {
+ "source-file": "files/data.json.gz",
+ "document-count": 165346692,
+ "compressed-bytes": 4917851637,
+ "uncompressed-bytes": 74818096036
+ }
+ ]
+ }
+ ],
+ "schedule": [
+ {
+ "operation": {
+ "operation-type": "create-collection",
+ "collection": "nyc_taxis"
+ }
+ },
+ {
+ "operation": {
+ "operation-type": "bulk-index",
+ "bulk-size": 5000
+ },
+ "warmup-time-period": 120,
+ "clients": 8
+ },
+ {
+ "operation": {
+ "operation-type": "commit",
+ "collection": "nyc_taxis"
+ }
+ },
+ {
+ "operation": {
+ "name": "match-all",
+ "operation-type": "search",
+ "param-source": "solr-search-source",
+ "collection": "nyc_taxis",
+ "body": {
+ "query": "*:*",
+ "rows": 10
+ }
+ },
+ "iterations": 1000,
+ "target-throughput": 100
+ }
+ ]
+}
+```
+
+A workload always includes the following elements:
+
+- **`collections`** — defines the Solr collection or collections to create before benchmarking.
+- **`corpora`** — defines the document datasets to index.
+- **`schedule`** — defines the operations and the order in which they run. You can also define
+ operations separately using the `operations` key and group them into named test procedures
+ using `test-procedures`.
+
+### `collections`
+
+The `collections` element replaces the `indices` concept from OpenSearch Benchmark. Each entry
+describes a Solr collection and the configset to use when creating it.
+
+| Field | Type | Description |
+| :---- | :---- | :---- |
+| `name` | string | The name of the Solr collection. |
+| `configset-path` | string | Path to a configset directory, relative to the workload root. The directory must contain at least `schema.xml` and `solrconfig.xml`. |
+| `shards` | integer | Number of shards. Default: `1`. |
+| `nrt_replicas` | integer | Number of NRT (near-real-time) replicas per shard. Default: `1`. |
+| `tlog_replicas` | integer | Number of TLOG replicas per shard. Default: `0`. |
+| `pull_replicas` | integer | Number of pull replicas per shard. Default: `0`. |
+
+### `corpora`
+
+The `corpora` element lists the datasets that Solr Benchmark downloads and indexes. Each corpus
+entry names the dataset and lists one or more document files.
+
+| Field | Type | Description |
+| :---- | :---- | :---- |
+| `name` | string | The name of the data corpus, used to match against a collection when indexing. |
+| `source-file` | string | The relative path to the data file inside the workload directory. Must be a gzip-compressed NDJSON file (one JSON document per line). |
+| `document-count` | integer | The number of documents in the source file. Solr Benchmark uses this to divide the corpus evenly among indexing clients. |
+| `uncompressed-bytes` | integer | The decompressed size in bytes. Used to estimate required disk space. |
+| `compressed-bytes` | integer | The compressed size in bytes. Used to estimate download time. |
+
+### `schedule`
+
+The `schedule` element lists the operations that run in order during the benchmark. The
+following walkthrough describes how the example schedule above executes:
+
+1. **`create-collection`** creates the `nyc_taxis` collection using the configset at
+ `configsets/nyc_taxis`. The collection is empty after this step.
+
+2. **`bulk-index`** indexes documents from the corpus into the collection.
+ - The `clients` field (set to `8`) specifies how many concurrent indexing clients Solr
+ Benchmark runs. Each client receives an equal share of the corpus.
+ - The `warmup-time-period` field (set to `120`) tells Solr Benchmark to index for 120 seconds
+ before starting to record metrics. Warmup traffic heats up JVM JIT compilation and caches
+ so that measurements are not skewed by cold-start effects.
+ - The `bulk-size` field (set to `5000`) controls how many documents are sent per HTTP request.
+
+3. **`commit`** issues a hard commit so that all indexed documents become visible to queries.
+
+4. **`search`** runs the `match-all` query repeatedly against the collection.
+ - The `iterations` field (set to `1000`) controls how many times each client executes the
+ query. To generate precise percentile figures in the summary report, run at least 1,000
+ iterations.
+ - The `target-throughput` field (set to `100`) defines the number of query requests per second
+ across all clients combined. Solr Benchmark throttles requests to stay at this target, which
+ keeps service-time measurements independent of scheduling overhead. See
+ [Target throughput](../optimizing-benchmarks/target-throughput.html) for details.
+
+### `operations` (optional)
+
+Named operations can be defined in a top-level `"operations"` array and referenced by name
+inside `schedule` entries. For complex workloads, operations are typically moved to a separate
+`operations/default.json` file and included via a Jinja2 `{% raw %}{% include %}{% endraw %}` statement. This keeps
+`workload.json` readable while allowing many operations to be defined and reused.
+
+### `test-procedures` (optional)
+
+Multiple named test procedures can be defined in a `test-procedures` array and
+selected at run time with `--test-procedure=<name>`. For details see
+[Choosing a workload](choosing-a-workload.html).
+
+---
+
+## Configsets
+
+Instead of an `index.json` mapping file (as used by OpenSearch Benchmark), Solr workloads
+provide a **configset** — a directory that Solr Benchmark uploads to the Solr cluster before
+creating a collection.
+
+A minimal configset directory contains:
+
+```
+configsets/
+└── my-schema/
+ ├── schema.xml
+ └── solrconfig.xml
+```
+
+### Minimal `schema.xml`
+
+```xml
+<?xml version="1.0" encoding="UTF-8" ?>
+<schema name="my-schema" version="1.6">
+
+ <field name="id" type="string" indexed="true" stored="true" required="true" multiValued="false"/>
+ <field name="_version_" type="plong" indexed="true" stored="false" docValues="true"/>
+ <field name="title" type="text_general" indexed="true" stored="true"/>
+
+ <uniqueKey>id</uniqueKey>
+
+ <fieldType name="string" class="solr.StrField" sortMissingLast="true"/>
+ <fieldType name="plong" class="solr.LongPointField" docValues="true"/>
+ <fieldType name="text_general" class="solr.TextField" positionIncrementGap="100">
+ <analyzer><tokenizer class="solr.StandardTokenizerFactory"/></analyzer>
+ </fieldType>
+
+</schema>
+```
+
+The `id` field is required by Solr as the unique key. The `_version_` field is required for
+optimistic concurrency control in SolrCloud and must have `indexed="true"` and `docValues="true"`.
+{: .important}
+
+### Minimal `solrconfig.xml`
+
+```xml
+<?xml version="1.0" encoding="UTF-8" ?>
+<config>
+ <luceneMatchVersion>9.0.0</luceneMatchVersion>
+
+ <requestHandler name="/select" class="solr.SearchHandler">
+ <lst name="defaults">
+ <str name="echoParams">explicit</str>
+ <int name="rows">10</int>
+ </lst>
+ </requestHandler>
+
+ <requestHandler name="/update" class="solr.UpdateRequestHandler"/>
+
+</config>
+```
+
+---
+
+## files.txt
+
+When a workload's corpus files are hosted on a remote server, the `files.txt` file lists the
+files that belong to the corpus, one per line. Solr Benchmark downloads each listed file from
+the configured `base_url` before the benchmark starts.
+
+```
+data.json.gz
+```
+
+For local workloads (where files already exist on disk), `files.txt` is optional.
+
+---
+
+## operations/ and test_procedures/
+
+To keep `workload.json` readable for large workloads, operations and test procedures are
+typically split into separate directories.
+
+### `operations/default.json`
+
+Defines the full set of named operations that test procedures can reference. The following
+example shows a realistic set of Solr Benchmark operations from an `nyc_taxis`-style workload:
+
+```json
+[
+ {
+ "name": "index",
+ "operation-type": "bulk-index",
+ "bulk-size": {% raw %}{{ bulk_size | default(5000) }}{% endraw %}
+ },
+ {
+ "name": "commit",
+ "operation-type": "commit",
+ "collection": "nyc_taxis"
+ },
+ {
+ "name": "match-all",
+ "operation-type": "search",
+ "param-source": "solr-search-source",
+ "collection": "nyc_taxis",
+ "body": {
+ "query": "*:*",
+ "rows": 10
+ }
+ },
+ {
+ "name": "range",
+ "operation-type": "search",
+ "param-source": "solr-search-source",
+ "collection": "nyc_taxis",
+ "body": {
+ "query": "total_amount:[5 TO 15}",
+ "rows": 10
+ }
+ },
+ {
+ "name": "asc-sort-passenger-count",
+ "operation-type": "search",
+ "param-source": "solr-search-source",
+ "collection": "nyc_taxis",
+ "body": {
+ "query": "*:*",
+ "sort": "passenger_count asc",
+ "rows": 10
+ }
+ },
+ {
+ "name": "passenger-count-agg",
+ "operation-type": "search",
+ "param-source": "solr-search-source",
+ "collection": "nyc_taxis",
+ "body": {
+ "query": "*:*",
+ "rows": 0,
+ "facet": {
+ "passengers": {
+ "type": "terms",
+ "field": "passenger_count",
+ "limit": 10
+ }
+ }
+ }
+ }
+]
+```
+
+### `test_procedures/default.json`
+
+Defines the order in which operations run. A test procedure is a named sequence of operations
+with its own schedule. The following example shows a default test procedure for the `nyc_taxis` workload:
+
+```json
+[
+ {
+ "name": "append-no-conflicts",
+ "description": "Index all documents, then run a set of search queries.",
+ "schedule": [
+ {
+ "operation": "delete-collection"
+ },
+ {
+ "operation": "create-collection"
+ },
+ {
+ "operation": "index",
+ "warmup-time-period": {% raw %}{{ warmup_time_period | default(240) }}{% endraw %},
+ "clients": {% raw %}{{ bulk_indexing_clients | default(8) }}{% endraw %}
+ },
+ {
+ "operation": "commit"
+ },
+ {
+ "operation": "match-all",
+ "warmup-iterations": 50,
+ "iterations": 500,
+ "target-throughput": {% raw %}{{ target_throughput | default(20) }}{% endraw %}
+ },
+ {
+ "operation": "range",
+ "warmup-iterations": 50,
+ "iterations": 200,
+ "target-throughput": {% raw %}{{ target_throughput | default(10) }}{% endraw %}
+ },
+ {
+ "operation": "passenger-count-agg",
+ "warmup-iterations": 50,
+ "iterations": 200,
+ "target-throughput": {% raw %}{{ target_throughput | default(5) }}{% endraw %}
+ }
+ ]
+ }
+]
+```
+
+---
+
+## Jinja2 templating
+
+All workload files are rendered as [Jinja2](https://jinja.palletsprojects.com/) templates
+before being parsed as JSON. This lets workload authors expose tunable parameters with default
+values:
+
+```json
+{
+ "operation-type": "bulk-index",
+ "bulk-size": {% raw %}{{ bulk_size | default(5000) }}{% endraw %}
+}
+```
+
+Override any parameter at run time with the `--workload-params` flag:
+
+```bash
+solr-benchmark run \
+ --workload=nyc_taxis \
+ --pipeline=benchmark-only \
+ --workload-params="bulk_size:10000,bulk_indexing_clients:4"
+```
+
+Multiple parameters are separated by commas. Parameter values can be integers, floats,
+booleans, or strings.
+
+The `default()` Jinja2 filter sets the value used when no override is provided. To make a
+parameter mandatory (no default), omit the filter — Solr Benchmark raises a clear error if
+the parameter is missing.
+{: .tip}
+
+---
+
+## Next steps
+
+- [Choosing a workload](choosing-a-workload.html) — browse the available workloads and select
+ one that matches your use case.
+- [Creating custom workloads](../working-with-workloads/creating-custom-workloads.html) — write
+ your own workload from scratch.
diff --git a/docs/user-guide/understanding-workloads/choosing-a-workload.md b/docs/user-guide/understanding-workloads/choosing-a-workload.md
new file mode 100644
index 00000000..792417fb
--- /dev/null
+++ b/docs/user-guide/understanding-workloads/choosing-a-workload.md
@@ -0,0 +1,82 @@
+---
+title: Choosing a Workload
+parent: Understanding Workloads
+grand_parent: User Guide
+nav_order: 12
+---
+
+# Choosing a Workload
+
+## Overview
+
+The [solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads) repository offers pre-built workloads for performance testing Apache Solr clusters. Selecting a workload that mirrors your cluster's actual use cases streamlines the benchmarking process and reduces custom development overhead.
+
+A practical example: a rideshare company can leverage the `nyc_taxis` workload instead of building a proprietary benchmark, because the taxi trip dataset closely resembles operational geospatial and time-series data.
+
+{: .note }
+> The Solr workload library is still growing. Only a subset of the original OpenSearch Benchmark workloads have been converted to Solr format so far. If a workload you need is not yet available, you can [convert an existing OpenSearch Benchmark workload](../../converter/) or [create a custom workload](../working-with-workloads/creating-custom-workloads.html).
+
+## Selection criteria
+
+When evaluating workloads, examine:
+
+- **Cluster scale**: small clusters (1–10 nodes) suit development; medium clusters (11–50 nodes) approximate production environments.
+- **Data compatibility**: review the example documents and the collection schema in the workload to compare field types with your actual data.
+- **Query patterns**: inspect the operations defined in the workload to verify it exercises your typical query types (term queries, range queries, facets, etc.).
+
+## Available workloads
+
+### nyc_taxis
+
+The `nyc_taxis` workload benchmarks typical search and analytics scenarios using ride data from yellow taxis in New York City in 2015. It evaluates:
+
+- Range and term queries
+- Geo-distance queries
+- Date-range queries
+- Faceted aggregations (histogram and date histogram)
+
+The dataset contains around 165 million documents and is suitable for small to medium clusters. A `--test-mode` run uses a small document subset and completes in minutes.
+
+**Example run:**
+
+```bash
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --test-mode
+```
+
+### geonames
+
+The `geonames` workload benchmarks search and geospatial scenarios using geographic place-name data from the [GeoNames](https://www.geonames.org/) database. It evaluates:
+
+- Full-text name search queries
+- Geo-distance queries
+- Faceted aggregations by country code and feature classification
+
+The dataset contains around 11.4 million documents and is suitable for small clusters. A `--test-mode` run uses a small document subset and completes in minutes.
+
+**Test procedures:**
+
+| Procedure | Description |
+|-----------|-------------|
+| `append-no-conflicts` (default) | Indexes the full corpus, then runs search and faceting queries |
+| `append-no-conflicts-index-only` | Indexing only, without query execution |
+
+**Example run:**
+
+```bash
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload geonames \
+ --test-mode
+```
+
+## Custom workloads
+
+For specialized requirements, see:
+
+- [Creating Custom Workloads](../working-with-workloads/creating-custom-workloads.html) — build a workload from scratch for your own data
+- [Converter Tool](../../converter/) — convert an existing OpenSearch Benchmark workload to Solr format
diff --git a/docs/user-guide/understanding-workloads/common-operations.md b/docs/user-guide/understanding-workloads/common-operations.md
new file mode 100644
index 00000000..8dc1dce5
--- /dev/null
+++ b/docs/user-guide/understanding-workloads/common-operations.md
@@ -0,0 +1,184 @@
+---
+title: Common Operations
+parent: Understanding Workloads
+grand_parent: User Guide
+nav_order: 16
+---
+
+# Common Operations
+
+Apache Solr Benchmark provides the following built-in operation types.
+
+## bulk-index
+
+Indexes documents from a corpus into a Solr collection in batches.
+
+```json
+{
+ "name": "bulk-index",
+ "operation-type": "bulk-index",
+ "bulk-size": 500,
+ "collection": "nyc_taxis"
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `bulk-size` | `500` | Number of documents per batch |
+| `collection` | (first collection in workload) | Target collection |
+
+## search
+
+Executes a Solr query and measures latency and throughput.
+
+```json
+{
+ "name": "match-all",
+ "operation-type": "search",
+ "body": {
+ "query": "*:*",
+ "rows": 10,
+ "fl": "id"
+ }
+}
+```
+
+For Solr JSON DSL queries using structured query syntax:
+
+```json
+{
+ "name": "range-query",
+ "operation-type": "search",
+ "body": {
+ "query": {
+ "range": "pickup_datetime:[2015-01-01T00:00:00Z TO 2015-02-01T00:00:00Z]"
+ }
+ }
+}
+```
+
+The `body` is passed directly as the Solr JSON query body. Use standard [Solr JSON Request API](https://solr.apache.org/guide/solr/latest/query-guide/json-request-api.html) syntax.
+
+## commit
+
+Issues a hard commit to flush all pending documents.
+
+```json
+{
+ "name": "commit",
+ "operation-type": "commit"
+}
+```
+
+To issue a soft commit instead:
+
+```json
+{
+ "name": "soft-commit",
+ "operation-type": "commit",
+ "soft-commit": true
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `soft-commit` | `false` | If `true`, issues a soft commit instead of a hard commit |
+
+## optimize
+
+Issues an optimize (force-merge) command to reduce the number of index segments.
+
+```json
+{
+ "name": "optimize",
+ "operation-type": "optimize",
+ "max-segments": 1
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `max-segments` | `1` | Target segment count after optimization |
+
+## wait-for-merges
+
+Polls the Solr node metrics API until all background merge operations have completed.
+
+```json
+{
+ "name": "wait-for-merges",
+ "operation-type": "wait-for-merges"
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `retry-wait-period` | `2.0` | Seconds between polling attempts |
+| `max-wait-seconds` | `3600` | Maximum seconds to wait |
+
+## paginated-search
+
+Executes a cursor-paginated Solr search using `cursorMark` deep pagination. Fetches all result pages and returns the total document count.
+
+```json
+{
+ "name": "paginated-search",
+ "operation-type": "paginated-search",
+ "q": "*:*",
+ "rows": 100,
+ "sort": "id asc"
+}
+```
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `q` | `*:*` | Query string |
+| `rows` | `100` | Page size (documents per request) |
+| `sort` | `id asc` | Sort order — must include a uniqueKey field |
+| `fl` | (none) | Field list |
+| `fq` | (none) | Filter query |
+
+## create-collection
+
+Creates a Solr collection.
+
+```json
+{
+ "name": "create-collection",
+ "operation-type": "create-collection",
+ "collection": "my_collection",
+ "configset-path": "configsets/my_schema",
+ "num-shards": 1,
+ "replication-factor": 1
+}
+```
+
+## delete-collection
+
+Deletes a Solr collection.
+
+```json
+{
+ "name": "delete-collection",
+ "operation-type": "delete-collection",
+ "collection": "my_collection"
+}
+```
+
+## raw-request
+
+Executes a raw HTTP request against the Solr Admin API. Useful for custom operations not covered by built-in types.
+
+```json
+{
+ "name": "my-custom-op",
+ "operation-type": "raw-request",
+ "path": "/api/collections/my_collection/config",
+ "method": "POST",
+ "body": {
+ "set-property": {
+ "updateHandler.autoSoftCommit.maxTime": "5000"
+ }
+ }
+}
+```
diff --git a/docs/user-guide/understanding-workloads/index.md b/docs/user-guide/understanding-workloads/index.md
new file mode 100644
index 00000000..70e4157f
--- /dev/null
+++ b/docs/user-guide/understanding-workloads/index.md
@@ -0,0 +1,10 @@
+---
+title: Understanding Workloads
+parent: User Guide
+nav_order: 10
+has_children: true
+---
+
+# Understanding Workloads
+
+A workload is the blueprint for a benchmark: it defines what data to load, what collections to create, and what operations to run.
diff --git a/docs/user-guide/working-with-workloads/contributing-workloads.md b/docs/user-guide/working-with-workloads/contributing-workloads.md
new file mode 100644
index 00000000..ce792525
--- /dev/null
+++ b/docs/user-guide/working-with-workloads/contributing-workloads.md
@@ -0,0 +1,18 @@
+---
+title: Sharing Custom Workloads
+parent: Working with Workloads
+grand_parent: User Guide
+nav_order: 13
+---
+
+# Sharing Custom Workloads
+
+You can share a custom workload with other Apache Solr Benchmark users by contributing it to the
+[solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads) repository on
+GitHub — the default workload source for Apache Solr Benchmark. Workloads in this repository
+are available to all users without needing `--workload-path`.
+
+For the full contribution guidelines — required files, README contents, data and licensing
+requirements, testing requirements, the pull request and review process, and the backporting
+policy — see the repository's
+**[CONTRIBUTING.md](https://github.com/janhoy/solr-benchmark-workloads/blob/main/CONTRIBUTING.md)**.
diff --git a/docs/user-guide/working-with-workloads/creating-custom-workloads.md b/docs/user-guide/working-with-workloads/creating-custom-workloads.md
new file mode 100644
index 00000000..525d22c6
--- /dev/null
+++ b/docs/user-guide/working-with-workloads/creating-custom-workloads.md
@@ -0,0 +1,287 @@
+---
+title: Creating Custom Workloads
+parent: Working with Workloads
+grand_parent: User Guide
+nav_order: 10
+---
+
+# Creating Custom Workloads
+
+To create a workload tailored for your Solr data and queries, create a directory with the
+structure described in [Anatomy of a Workload](../understanding-workloads/anatomy-of-a-workload.html).
+
+## Minimal workload example
+
+```
+my-benchmark/
+├── workload.json
+├── configsets/
+│ └── my_schema/
+│ ├── schema.xml
+│ └── solrconfig.xml
+└── files/
+ └── my_data.json.gz
+```
+
+**workload.json:**
+
+```json
+{
+ "description": "My custom Solr benchmark",
+ "collections": [
+ {
+ "name": "my_collection",
+ "configset-path": "configsets/my_schema",
+ "shards": 1,
+ "nrt_replicas": 1
+ }
+ ],
+ "corpora": [
+ {
+ "name": "my_data",
+ "documents": [
+ {
+ "source-file": "files/my_data.json.gz",
+ "document-count": 100000,
+ "compressed-bytes": 50000000,
+ "uncompressed-bytes": 200000000
+ }
+ ]
+ }
+ ],
+ "schedule": [
+ {
+ "operation": {
+ "operation-type": "bulk-index",
+ "bulk-size": 500
+ },
+ "warmup-time-period": 60,
+ "clients": 4
+ },
+ { "operation": "commit" },
+ {
+ "operation": {
+ "name": "my-search",
+ "operation-type": "search",
+ "body": { "query": "*:*", "rows": 10 }
+ },
+ "clients": 1,
+ "iterations": 100
+ }
+ ]
+}
+```
+
+---
+
+## Invoking your custom workload
+
+Point Solr Benchmark at the workload directory with `--workload-path`:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path /path/to/my-benchmark
+```
+
+Use `--test-mode` to do a quick validation pass before running the full workload:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path /path/to/my-benchmark \
+ --test-mode
+```
+
+---
+
+## Preparing corpus data
+
+Corpus data must be in gzip-compressed NDJSON (Newline-Delimited JSON) format, where each
+line is a JSON document to index. Documents should include an `id` field matching your
+Solr schema's unique key field.
+
+```json
+{"id": "1", "title": "My document", "timestamp": "2024-01-01T00:00:00Z"}
+{"id": "2", "title": "Another document", "timestamp": "2024-01-02T00:00:00Z"}
+```
+
+Compress the file:
+
+```bash
+gzip my_data.json
+```
+
+---
+
+## Test mode support
+
+`--test-mode` reads only the first 1,000 documents instead of the full corpus. To support
+it, create a companion `-1k` file alongside the main corpus:
+
+```bash
+# Create a 1,000-document companion file
+zcat files/my_data.json.gz | head -n 1000 | gzip > files/my_data-1k.json.gz
+```
+
+Solr Benchmark automatically uses the `-1k` variant when `--test-mode` is passed — no
+changes to `workload.json` are required. The `document-count` in the corpus definition
+should reflect the **full** dataset; Solr Benchmark adjusts internally for test mode.
+
+Test mode is not a substitute for a full run. Always verify results against the complete
+corpus before drawing conclusions.
+{: .note}
+
+---
+
+## Defining a configset
+
+A configset directory must contain at minimum:
+- `schema.xml` — field definitions and types
+- `solrconfig.xml` — request handler and cache configuration
+
+See the [Apache Solr Reference Guide: Configsets](https://solr.apache.org/guide/solr/latest/configuration-guide/configsets.html) for full documentation.
+
+---
+
+## Using Jinja2 parameters
+
+Workload files are Jinja2 templates. Add parameters to allow runtime overrides:
+
+{% raw %}
+```json
+{
+ "operation-type": "bulk-index",
+ "bulk-size": {{ bulk_size | default(500) }},
+ "clients": {{ index_clients | default(4) }}
+}
+```
+{% endraw %}
+
+Then override at runtime:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path my-benchmark \
+ --workload-params "bulk_size:1000,index_clients:8"
+```
+
+---
+
+## Adding multiple test procedures
+
+Replace the top-level `schedule` with a `test-procedures` array to define named test
+procedures that can be selected at run time with `--test-procedure`:
+
+```json
+{
+ "description": "My custom benchmark",
+ "collections": [ ... ],
+ "corpora": [ ... ],
+ "test-procedures": [
+ {
+ "name": "index-only",
+ "description": "Index all documents without running any queries.",
+ "default": false,
+ "schedule": [
+ { "operation": { "operation-type": "bulk-index", "bulk-size": 5000 }, "clients": 8 },
+ { "operation": "commit" }
+ ]
+ },
+ {
+ "name": "index-and-query",
+ "description": "Index all documents then run a set of search queries.",
+ "default": true,
+ "schedule": [
+ { "operation": { "operation-type": "bulk-index", "bulk-size": 5000 }, "clients": 8 },
+ { "operation": "commit" },
+ {
+ "operation": {
+ "name": "match-all",
+ "operation-type": "search",
+ "body": { "query": "*:*", "rows": 10 }
+ },
+ "iterations": 1000,
+ "target-throughput": 50
+ }
+ ]
+ }
+ ]
+}
+```
+
+Run a specific test procedure:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path my-benchmark \
+ --test-procedure index-only
+```
+
+The test procedure marked `"default": true` is used when `--test-procedure` is not
+specified.
+
+---
+
+## Separating operations and test procedures
+
+For large workloads, split operations and test procedures into separate files and reference
+them from `workload.json` using the `benchmark.collect()` Jinja2 macro:
+
+{% raw %}
+```json
+{
+ "description": "My custom benchmark",
+ "collections": [ ... ],
+ "corpora": [ ... ],
+ "operations": [
+ {{ benchmark.collect(parts="operations/*.json") }}
+ ],
+ "test-procedures": [
+ {{ benchmark.collect(parts="test_procedures/*.json") }}
+ ]
+}
+```
+{% endraw %}
+
+The macro reads every `.json` file in the specified directory and inlines its contents.
+Directory layout:
+
+```
+my-benchmark/
+├── workload.json
+├── operations/
+│ ├── indexing.json # bulk-index, commit operations
+│ └── search.json # match-all, range, aggregation operations
+└── test_procedures/
+ └── default.json # append-no-conflicts test procedure
+```
+
+This pattern keeps `workload.json` readable regardless of how many operations the workload
+defines.
+
+---
+
+## Splitting operations into separate files (simple include)
+
+For simpler workloads where you only need one operations file and one test-procedures file,
+use a Jinja2 {% raw %}`{% include %}`{% endraw %} statement:
+
+{% raw %}
+```json
+{
+ "operations": [
+ {% include "operations/default.json" %}
+ ],
+ "test-procedures": [
+ {% include "test_procedures/default.json" %}
+ ]
+}
+```
+{% endraw %}
diff --git a/docs/user-guide/working-with-workloads/finetune-workloads.md b/docs/user-guide/working-with-workloads/finetune-workloads.md
new file mode 100644
index 00000000..f1c36109
--- /dev/null
+++ b/docs/user-guide/working-with-workloads/finetune-workloads.md
@@ -0,0 +1,205 @@
+---
+title: Fine-tuning Workloads
+parent: Working with Workloads
+grand_parent: User Guide
+nav_order: 12
+---
+
+# Fine-tuning Workloads
+
+## Overriding parameters at runtime
+
+Workloads can expose Jinja2 parameters that you override at runtime with `--workload-params`.
+
+In `workload.json` (or an included operations file):
+
+{% raw %}
+```json
+{
+ "operation-type": "bulk-index",
+ "bulk-size": {{ bulk_size | default(500) }}
+}
+```
+{% endraw %}
+
+Override at runtime:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload my-workload \
+ --workload-params "bulk_size:1000"
+```
+
+Multiple parameters are separated by commas. Parameter values can be integers, floats,
+booleans (`true`/`false`), or quoted strings:
+
+```bash
+# Integer and float
+--workload-params "bulk_size:5000,target_throughput:25.5"
+
+# Boolean
+--workload-params "include_aggs:true,enable_warmup:false"
+
+# String (use quotes around the whole value list if it contains spaces)
+--workload-params "collection_name:my_collection,query_type:dismax"
+```
+
+The `default()` Jinja2 filter sets the value used when no override is provided. Omitting
+`default()` makes the parameter mandatory — Solr Benchmark raises an error if it is not
+supplied at run time.
+{: .tip}
+
+---
+
+## Controlling throughput
+
+Use `target-throughput` to cap the rate of an operation. The value is **operations per
+second across all clients combined**:
+
+```json
+{
+ "operation": "search",
+ "target-throughput": 100,
+ "clients": 4
+}
+```
+
+In the example above, all four clients together issue 100 search requests per second (25
+per client). If the operation is slower than the target, Solr Benchmark runs it as fast as
+possible without throttling.
+
+Setting `target-throughput` keeps service-time measurements independent of scheduling
+overhead, which produces more reproducible and meaningful latency percentiles. Without a
+throughput cap, clients run at full speed and the measured latency includes queueing effects
+inside Solr.
+
+See [Target throughput](../optimizing-benchmarks/target-throughput.html) for a detailed
+explanation of how Solr Benchmark implements throughput control.
+{: .note}
+
+---
+
+## Controlling warmup
+
+Use `warmup-time-period` (seconds) or `warmup-iterations` to discard initial measurements
+so that JVM JIT compilation and Solr cache warm-up do not skew results:
+
+```json
+{
+ "operation": "search",
+ "warmup-time-period": 60,
+ "iterations": 500
+}
+```
+
+```json
+{
+ "operation": "bulk-index",
+ "warmup-iterations": 100,
+ "clients": 8
+}
+```
+
+Warmup traffic uses the same clients and parameters as the measured traffic — only the
+metric recording is suppressed.
+
+---
+
+## Controlling concurrency
+
+Use `clients` to set the number of parallel clients per operation:
+
+```json
+{
+ "operation": "bulk-index",
+ "clients": 8
+}
+```
+
+Each client receives an equal partition of the corpus. Increasing `clients` can improve
+throughput on systems with many CPU cores, but results in higher resource usage on both the
+Solr Benchmark host and the Solr cluster.
+
+---
+
+## Controlling duration
+
+Use `time-period` (seconds) to run an operation for a fixed duration instead of a fixed
+number of iterations:
+
+```json
+{
+ "operation": "search",
+ "time-period": 120,
+ "clients": 4
+}
+```
+
+Use `iterations` when you need a precise number of samples for percentile calculations.
+At least 1,000 iterations are recommended for stable 99th-percentile figures.
+
+---
+
+## Selecting a task subset
+
+Use `--include-tasks` or `--exclude-tasks` to run only part of a schedule during a run:
+
+```bash
+# Run only the indexing task
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --include-tasks bulk-index,commit
+
+# Skip slow aggregation tasks
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --exclude-tasks passenger-count-agg
+```
+
+This is particularly useful when iterating on a single operation type — for example, tuning
+search query performance without waiting for indexing to complete on every run.
+
+---
+
+## Selecting a subset of documents
+
+Use `number-of-docs` and `offset` in the corpus definition to benchmark on a subset of the
+data without downloading the full corpus:
+
+```json
+{
+ "source-file": "files/data.json.gz",
+ "document-count": 165346692,
+ "number-of-docs": 1000000,
+ "offset": 0
+}
+```
+
+---
+
+## Replicating production characteristics
+
+To produce benchmark results that reflect real-world Solr performance, tune the workload
+parameters to match the load pattern of your production system:
+
+- **Indexing throughput**: Adjust `clients` and `bulk-size` to match your production
+ indexing rate. More clients increase parallelism; larger bulk size reduces per-request
+ overhead.
+- **Search rate**: Use `target-throughput` and `clients` to reproduce the number of
+ concurrent search requests your cluster handles in production.
+- **Read/write mix**: If production traffic is a mix of indexing and querying, consider
+ using a `time-period`-based schedule with separate indexing and search tasks running
+ back-to-back, rather than a strictly sequential index-then-query approach.
+- **Data volume**: Run against a data corpus that matches the size and distribution of your
+ production data. Results from a 1-million-document corpus may not accurately predict
+ behavior at 100 million documents, where segment merges and cache behavior differ
+ significantly.
+- **Warmup**: Always include a warmup period (`warmup-time-period` or `warmup-iterations`)
+ before recording measurements. Cold JVM and empty Solr caches can inflate latency
+ numbers by 2–10× compared to steady-state.
diff --git a/docs/user-guide/working-with-workloads/index.md b/docs/user-guide/working-with-workloads/index.md
new file mode 100644
index 00000000..7af43af2
--- /dev/null
+++ b/docs/user-guide/working-with-workloads/index.md
@@ -0,0 +1,17 @@
+---
+title: Working with Workloads
+parent: User Guide
+nav_order: 15
+has_children: true
+---
+
+# Working with Workloads
+
+This section covers how to run, create, and tune workloads for Apache Solr Benchmark.
+
+| Page | Description |
+| :--- | :--- |
+| [Running a Workload](running-workloads.html) | How to invoke `solr-benchmark run` with all common flags, pipelines, and options. |
+| [Creating Custom Workloads](creating-custom-workloads.html) | Build a workload from scratch for your own Solr data and query patterns. |
+| [Fine-tuning Workloads](finetune-workloads.html) | Override parameters and tune throughput, warmup, and concurrency for accurate results. |
+| [Sharing Custom Workloads](contributing-workloads.html) | Guidelines for contributing a workload to the shared workloads repository. |
diff --git a/docs/user-guide/working-with-workloads/running-workloads.md b/docs/user-guide/working-with-workloads/running-workloads.md
new file mode 100644
index 00000000..b9757f51
--- /dev/null
+++ b/docs/user-guide/working-with-workloads/running-workloads.md
@@ -0,0 +1,216 @@
+---
+title: Running a Workload
+parent: Working with Workloads
+grand_parent: User Guide
+nav_order: 9
+---
+
+# Running a Workload
+
+## Listing available workloads
+
+To see which named workloads are available in the configured workload repository:
+
+```bash
+solr-benchmark list workloads
+```
+
+This prints the name and description of each workload that Solr Benchmark can fetch and run
+by name. Workloads are fetched from the repository configured under `[workloads]` in
+`~/.solr-benchmark/benchmark.ini` (default:
+[https://github.com/janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads)).
+
+---
+
+## Basic syntax
+
+```bash
+solr-benchmark run [--pipeline PIPELINE] [--target-hosts HOSTS] \
+ [--workload WORKLOAD | --workload-path PATH] [OPTIONS]
+```
+
+---
+
+## Using a named workload
+
+Named workloads are fetched from
+[https://github.com/janhoy/solr-benchmark-workloads](https://github.com/janhoy/solr-benchmark-workloads):
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis
+```
+
+---
+
+## Using a local workload path
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload-path /path/to/my-workload
+```
+
+---
+
+## Selecting a test procedure
+
+A workload may define multiple test procedures. Use `--test-procedure` to select one:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --test-procedure bulk-only
+```
+
+If `--test-procedure` is omitted, Solr Benchmark runs the default test procedure defined
+in `workload.json`.
+
+---
+
+## Running a subset of tasks
+
+Use `--include-tasks` or `--exclude-tasks` to run only part of a workload's schedule. Both
+flags accept a comma-separated list of task names.
+
+Run only the indexing and commit tasks, skipping all search operations:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --include-tasks bulk-index,commit
+```
+
+Run the full workload but skip aggregation-heavy tasks:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --exclude-tasks passenger-count-agg,date-histogram
+```
+
+Task names correspond to the `name` field on each operation in the workload's schedule.
+{: .note}
+
+---
+
+## Test mode
+
+Pass `--test-mode` to run a shortened version of the workload (at most 1,000 documents) for
+quick validation:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --test-mode
+```
+
+When `--test-mode` is active, Solr Benchmark uses a companion `-1k` corpus file (for
+example, `data-1k.json.gz`) instead of the full dataset. See
+[Creating Custom Workloads](creating-custom-workloads.html#test-mode-support) for instructions
+on creating the companion file for your own workload.
+
+---
+
+## Targeting a multi-node cluster
+
+Separate multiple hosts with commas:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts node1:8983,node2:8983,node3:8983 \
+ --workload nyc_taxis
+```
+
+---
+
+## Using the Docker pipeline
+
+The `docker` pipeline pulls the official `solr` Docker image, starts a single-node cluster,
+runs the workload, and stops the cluster when finished. No JDK is required.
+
+```bash
+solr-benchmark run \
+ --pipeline docker \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --test-mode
+```
+
+---
+
+## Using the from-distribution pipeline
+
+The `from-distribution` pipeline downloads a Solr release archive, installs it locally,
+and starts a cluster. JDK 21 must be available on the host.
+
+```bash
+solr-benchmark run \
+ --pipeline from-distribution \
+ --distribution-version 9.10.1 \
+ --workload nyc_taxis \
+ --cluster-config 4gheap
+```
+
+---
+
+## Customizing workload parameters
+
+Override workload Jinja2 parameters at runtime with `--workload-params`:
+
+```bash
+solr-benchmark run \
+ --pipeline benchmark-only \
+ --target-hosts localhost:8983 \
+ --workload nyc_taxis \
+ --workload-params "bulk_size:1000,search_clients:4"
+```
+
+Multiple parameters are separated by commas. See
+[Fine-tuning Workloads](finetune-workloads.html) for a full list of tuning options.
+
+---
+
+## Error handling
+
+By default, the run continues if individual operations fail. To abort on the first error:
+
+```bash
+solr-benchmark run --on-error abort ...
+```
+
+---
+
+## Verifying results
+
+After a successful run, check the console summary for error rates and throughput. To verify
+that documents were actually indexed, open the Solr Admin UI collection overview:
+
+```
+http://localhost:8983/solr/#/~collections
+```
+
+Select the collection and confirm that the document count matches the expected number for
+your workload. For `nyc_taxis` in `--test-mode`, you should see approximately 1,000
+documents.
+
+The full computed results for every run are stored in
+`~/.solr-benchmark/benchmarks/test-runs/<run-id>/test_run.json`. To view the results for
+the most recent run:
+
+```bash
+ls -t ~/.solr-benchmark/benchmarks/test-runs/ | head -1 | \
+ xargs -I{} cat ~/.solr-benchmark/benchmarks/test-runs/{}/test_run.json | python3 -m json.tool
+```
diff --git a/docs/user-guides/create-pipeline.md b/docs/user-guides/create-pipeline.md
deleted file mode 100644
index 0e50a976..00000000
--- a/docs/user-guides/create-pipeline.md
+++ /dev/null
@@ -1,129 +0,0 @@
----
-layout: default
-title: Create and manage pipelines
-parent: OSB Use Cases
-nav_order: 20
----
-
-///// EXAMPLE TEMPLATE
-
-Pipelines allow you to perform X task. Enter some additional introductory information about pipelines here.
-
-NOTE: This page is an example of how pipelines can be used in the context of OpenSearch Benchmark. It uses content from a different user guide as a template. Use the following example from the Observability plugin to form of the basis of an OpenSearch Benchmark User Guide.
-
-# Notebooks
-
-An OpenSearch Dashboards notebook is an interface that lets you easily combine code snippets, live visualizations, and narrative text in a single notebook interface.
-
-Notebooks let you interactively explore data by running different visualizations that you can share with team members to collaborate on a project.
-
-A notebook is a document composed of two elements: code blocks (Markdown/SQL/PPL) and visualizations. Choose multiple timelines to compare and contrast visualizations.
-
-You can also generate [reports]({{site.url}}{{site.baseurl}}/dashboards/reporting/) directly from your notebooks.
-
-Common use cases include creating postmortem reports, designing runbooks, building live infrastructure reports, and writing documentation.
-
-Tenants in OpenSearch Dashboards are spaces for saving notebooks and other OpenSearch Dashboards objects. For more information, see [OpenSearch Dashboards multi-tenancy]({{site.url}}{{site.baseurl}}/security-plugin/access-control/multi-tenancy/).
-{: .note }
-
-
-## Get started with notebooks
-
-To get started, choose **Notebooks** within OpenSearch Dashboards.
-
-
-### Step 1: Create a notebook
-
-A notebook is an interface for creating reports.
-
-1. Choose **Create notebook** and enter a descriptive name.
-1. Choose **Create**.
-
-Choose **Actions** to rename, duplicate, or delete a notebook.
-
-
-
-### Step 2: Add a paragraph
-
-Paragraphs combine code blocks and visualizations for describing data.
-
-#### Add a code block
-
-Code blocks support markdown, SQL, and PPL languages.
-
-Specify the input language on the first line using `%[language type]` syntax.
-For example, type `%md` for markdown, `%sql` for SQL, and `%ppl` for PPL.
-
-##### Sample markdown block
-
-```
-%md
-Add in text formatted in markdown.
-```
-
-
-
-##### Sample SQL block
-
-```sql
-%sql
-Select * from opensearch_dashboards_sample_data_flights limit 20;
-```
-
-
-
-##### Sample PPL block
-
-```
-%ppl
-source=opensearch_dashboards_sample_data_logs | head 20
-```
-
-
-
-
-#### Add a visualization
-
-1. To add a visualization, choose **Add paragraph** and select **Visualization**.
-1. In **Title**, select your visualization and choose a date range. You can choose multiple timelines to compare and contrast visualizations.
-1. To run and save a paragraph, choose **Run**.
-
-
-
-## Paragraph actions
-
-You can perform the following actions on paragraphs:
-
-- Add a new paragraph to the top of a report.
-- Add a new paragraph to the bottom of a report.
-- Run all the paragraphs at the same time.
-- Clear the outputs of all paragraphs.
-- Delete all the paragraphs.
-
-
-
-## Sample notebooks
-
-We prepared the following sample notebooks that showcase a variety of use cases:
-
-- Using SQL to query the OpenSearch Dashboards sample flight data.
-- Using PPL to query the OpenSearch Dashboards sample web logs data.
-- Using PPL and visualizations to perform sample root cause event analysis on the OpenSearch Dashboards sample web logs data.
-
-To add a sample notebook, choose **Actions** and select **Add sample notebooks**.
-
-
-
-## Create a report
-
-You can use notebooks to create PNG and PDF reports:
-
-1. From the top menu bar, choose **Reporting actions**.
-1. You can choose to **Download PDF** or **Download PNG**.
-
- Reports generate asynchronously in the background and might take a few minutes, depending on the size of the report. A notification appears when your report is ready to download.
-
-1. To create a schedule-based report, choose **Create report definition**. For steps to create a report definition, see [Create reports using a definition]({{site.url}}{{site.baseurl}}/dashboards/reporting#create-reports-using-a-definition).
-1. To see all your reports, choose **View all reports**.
-
-
diff --git a/docs/user-guides/index.md b/docs/user-guides/index.md
deleted file mode 100644
index e69de29b..00000000
diff --git a/it/README.md b/it/README.md
new file mode 100644
index 00000000..cc2f0772
--- /dev/null
+++ b/it/README.md
@@ -0,0 +1,13 @@
+# Integration Tests
+
+> **Note**: The integration tests in this directory were originally written for OpenSearch Benchmark and test OpenSearch-specific pipeline steps (provisioning, starting, and querying OpenSearch clusters). They are **not yet adapted** for Apache Solr Benchmark and will not pass against a Solr cluster.
+>
+> A Solr-native integration test suite is planned as a future user story. Until then, use the Solr unit tests in `tests/unit/solr/` to validate Solr-specific functionality.
+
+## Running the existing tests
+
+These tests require a running OpenSearch cluster and are not part of the default CI for this branch.
+
+```bash
+make it # Requires Java, Docker; ~30 min
+```
diff --git a/opensearch_benchmark.png b/opensearch_benchmark.png
deleted file mode 100644
index 29fa3c04..00000000
Binary files a/opensearch_benchmark.png and /dev/null differ