You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@datafusion.apache.org by gi...@apache.org on 2026/10/15 15:31:26 UTC
(datafusion-ballista) branch asf-site updated: Publish built docs triggered by 4f9655f5595b90c589dfdbdf78a02ae1c0a98d08
This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/datafusion-ballista.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 6fe1678d4 Publish built docs triggered by 4f9655f5595b90c589dfdbdf78a02ae1c0a98d08
6fe1678d4 is described below
commit 6fe1678d463f7c6e76635012707deafc6bc6f2cd
Author: github-actions[bot] <gi...@users.noreply.github.com>
AuthorDate: Thu Oct 15 15:31:19 2026 +0000
Publish built docs triggered by 4f9655f5595b90c589dfdbdf78a02ae1c0a98d08
---
_sources/upgrading/56.0.0.md.txt | 90 ++++++++
_sources/upgrading/index.rst.txt | 1 +
.../user-guide/deployment/cargo-install.md.txt | 7 +
_sources/user-guide/deployment/docker.md.txt | 4 +
_sources/user-guide/deployment/kubernetes.md.txt | 239 +++++++++++++++++++++
_sources/user-guide/scheduler.md.txt | 75 ++++---
changelog/0.10.0.html | 1 +
changelog/0.11.0.html | 1 +
changelog/0.12.0.html | 1 +
changelog/0.5.0.html | 1 +
changelog/0.6.0.html | 1 +
changelog/0.7.0.html | 1 +
changelog/0.8.0.html | 1 +
changelog/0.9.0.html | 1 +
changelog/43.0.0.html | 1 +
changelog/44.0.0.html | 1 +
changelog/45.0.0.html | 1 +
changelog/46.0.0.html | 1 +
changelog/47.0.0.html | 1 +
changelog/48.0.0.html | 1 +
changelog/49.0.0.html | 1 +
changelog/50.0.0.html | 1 +
changelog/51.0.0.html | 1 +
changelog/52.0.0.html | 1 +
changelog/53.0.0.html | 1 +
changelog/54.0.0.html | 1 +
changelog/54.1.0.html | 1 +
changelog/55.0.0.html | 1 +
changelog/index.html | 1 +
community/adopters.html | 1 +
community/communication.html | 1 +
contributors-guide/architecture.html | 1 +
contributors-guide/benchmarking/ballista.html | 1 +
contributors-guide/benchmarking/comet.html | 1 +
contributors-guide/benchmarking/environment.html | 1 +
contributors-guide/benchmarking/index.html | 1 +
contributors-guide/benchmarking/spark.html | 1 +
contributors-guide/benchmarking/trino.html | 1 +
contributors-guide/code-organization.html | 1 +
contributors-guide/dataframe-cache.html | 1 +
contributors-guide/development.html | 1 +
contributors-guide/shuffle.html | 1 +
contributors-guide/user-personas.html | 1 +
genindex.html | 1 +
index.html | 1 +
objects.inv | Bin 1367 -> 1372 bytes
search.html | 1 +
searchindex.js | 2 +-
upgrading/54.0.0.html | 1 +
upgrading/55.0.0.html | 5 +-
.../communication.html => upgrading/56.0.0.html | 141 ++++++++----
upgrading/index.html | 6 +-
user-guide/cli.html | 1 +
user-guide/configs.html | 1 +
user-guide/deployment/cargo-install.html | 6 +
user-guide/deployment/docker-compose.html | 1 +
user-guide/deployment/docker.html | 5 +
user-guide/deployment/index.html | 2 +
user-guide/deployment/kubernetes.html | 227 +++++++++++++++++++
user-guide/deployment/quick-start.html | 1 +
user-guide/extending-components.html | 1 +
user-guide/extensions-example.html | 1 +
user-guide/faq.html | 1 +
user-guide/flightsql.html | 1 +
user-guide/history-server.html | 1 +
user-guide/introduction.html | 1 +
user-guide/metrics.html | 1 +
user-guide/python/cloud-storage.html | 1 +
user-guide/python/index.html | 1 +
user-guide/python/jupyter.html | 1 +
user-guide/python/quickstart.html | 1 +
user-guide/rust.html | 1 +
user-guide/scheduler.html | 87 +++++---
user-guide/tuning-guide.html | 1 +
74 files changed, 853 insertions(+), 102 deletions(-)
diff --git a/_sources/upgrading/56.0.0.md.txt b/_sources/upgrading/56.0.0.md.txt
new file mode 100644
index 000000000..8e9897cc9
--- /dev/null
+++ b/_sources/upgrading/56.0.0.md.txt
@@ -0,0 +1,90 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# Upgrade Guides
+
+## Ballista 56.0.0
+
+**Note:** Ballista `56.0.0` has not been released yet. The information provided
+in this section pertains to features and changes that have already been merged
+to the main branch and are awaiting release in this version.
+
+### Deprecations
+
+#### The scheduler's embedded Flight proxy is deprecated
+
+`--enable-embedded-flight-proxy` (and `SchedulerConfig::enable_embedded_flight_proxy`)
+is deprecated and will be removed in 57.0.0. The proxy puts result traffic on the
+scheduler's process, where it competes with query planning and task scheduling,
+and it scales only with the scheduler.
+
+Deployments where clients cannot reach the executors should instead run the new
+`ballista-result-service` binary, a stateless Arrow Flight service that forwards
+each result fetch to the executor holding the partition, and point the
+scheduler's `--advertise-flight-endpoint` at it, or at a load balancer in front
+of several replicas:
+
+```bash
+ballista-result-service --bind-port 50055
+ballista-scheduler --advertise-flight-endpoint ballista-results.example.com:50055
+```
+
+For Kubernetes, see the example manifests in
+[Serving Results to Clients Outside the Cluster](../user-guide/deployment/kubernetes.md#serving-results-to-clients-outside-the-cluster).
+
+Clients need no changes. The embedded proxy keeps working throughout 56.x, so
+you can compare the two and roll back without downgrading Ballista; the
+scheduler logs a warning at startup while it is enabled.
+
+If you embed the scheduler and configure the proxy's connections to executors
+with `SchedulerConfig::use_tls` and `override_create_grpc_client_endpoint`, for
+example for mTLS, run the result service from your own binary with
+`ballista_result_service::serve`. Its `ResultServiceConfig` takes the same two
+settings. The `ballista-result-service` binary connects to executors in
+plaintext, as the stock scheduler binary does.
+
+Also deprecated:
+
+- `SchedulerConfig::with_enable_embedded_flight_proxy`
+- passing `--advertise-flight-endpoint` with no value, which starts the
+ embedded proxy
+
+### Behavior changes
+
+#### Containers stop on SIGTERM
+
+The images' entrypoints now `exec` their binaries, and each binary handles
+SIGTERM, so `docker stop` and pod termination reach the Ballista process.
+Before, the signal never did, and containers waited out their grace period to
+be killed.
+
+- Executors now drain their running tasks before exiting. If tasks can run
+ longer than 30 seconds, raise the executor pods'
+ `terminationGracePeriodSeconds` and the scheduler's
+ `--executor-termination-grace-period` to match.
+- The scheduler exits on SIGTERM or Ctrl-C, without draining its connections.
+- The CLI exits on SIGTERM.
+- The standalone image stops its executor first, then its scheduler.
+
+#### Undecodable fetch tickets are rejected as `InvalidArgument`
+
+Executors, the embedded proxy, and the result service now share one
+implementation of partition fetches. A `DoGet` ticket that does not decode as a
+Ballista action is reported with the gRPC status `InvalidArgument` everywhere;
+executors used to report it as `Internal`.
diff --git a/_sources/upgrading/index.rst.txt b/_sources/upgrading/index.rst.txt
index 1d490ea4a..e651933fc 100644
--- a/_sources/upgrading/index.rst.txt
+++ b/_sources/upgrading/index.rst.txt
@@ -21,5 +21,6 @@ Upgrade Guides
.. toctree::
:maxdepth: 1
+ Ballista 56.0.0 <56.0.0>
Ballista 55.0.0 <55.0.0>
Ballista 54.0.0 <54.0.0>
diff --git a/_sources/user-guide/deployment/cargo-install.md.txt b/_sources/user-guide/deployment/cargo-install.md.txt
index 2cdbaeb7c..c3e4261f8 100644
--- a/_sources/user-guide/deployment/cargo-install.md.txt
+++ b/_sources/user-guide/deployment/cargo-install.md.txt
@@ -27,6 +27,13 @@ cargo install --locked ballista-scheduler
cargo install --locked ballista-executor
```
+If clients cannot reach the executors, also install the result service, which serves them query
+results instead (see [Fetching Query Results](../scheduler.md#fetching-query-results)):
+
+```bash
+cargo install --locked ballista-result-service
+```
+
With these crates installed, it is now possible to start a scheduler process.
```bash
diff --git a/_sources/user-guide/deployment/docker.md.txt b/_sources/user-guide/deployment/docker.md.txt
index 46d4130a0..aeb5b851e 100644
--- a/_sources/user-guide/deployment/docker.md.txt
+++ b/_sources/user-guide/deployment/docker.md.txt
@@ -26,6 +26,7 @@ The official Docker images are published to the GitHub Container Registry:
- [`ghcr.io/apache/datafusion-ballista-standalone`](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-standalone)
- [`ghcr.io/apache/datafusion-ballista-scheduler`](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-scheduler)
- [`ghcr.io/apache/datafusion-ballista-executor`](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-executor)
+- [`ghcr.io/apache/datafusion-ballista-result-service`](https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-result-service)
Pull the images needed for your deployment with the following commands:
@@ -33,6 +34,8 @@ Pull the images needed for your deployment with the following commands:
docker pull ghcr.io/apache/datafusion-ballista-standalone:latest
docker pull ghcr.io/apache/datafusion-ballista-scheduler:latest
docker pull ghcr.io/apache/datafusion-ballista-executor:latest
+# Only needed if clients cannot reach the executors:
+docker pull ghcr.io/apache/datafusion-ballista-result-service:latest
```
Alternatively run the following commands to clone the source repository and build the Docker images from source:
@@ -48,6 +51,7 @@ This will create the following images:
- `apache/datafusion-ballista-benchmarks:latest`
- `apache/datafusion-ballista-cli:latest`
- `apache/datafusion-ballista-executor:latest`
+- `apache/datafusion-ballista-result-service:latest`
- `apache/datafusion-ballista-scheduler:latest`
- `apache/datafusion-ballista-standalone:latest`
diff --git a/_sources/user-guide/deployment/kubernetes.md.txt b/_sources/user-guide/deployment/kubernetes.md.txt
index 524384447..515665abf 100644
--- a/_sources/user-guide/deployment/kubernetes.md.txt
+++ b/_sources/user-guide/deployment/kubernetes.md.txt
@@ -54,6 +54,14 @@ docker push <your-repo>/datafusion-ballista-scheduler:latest
docker push <your-repo>/datafusion-ballista-executor:latest
```
+If clients will fetch results from outside the cluster, also publish the result service image (see
+[Serving Results to Clients Outside the Cluster](#serving-results-to-clients-outside-the-cluster)):
+
+```bash
+docker tag apache/datafusion-ballista-result-service:latest <your-repo>/datafusion-ballista-result-service:latest
+docker push <your-repo>/datafusion-ballista-result-service:latest
+```
+
## Create Persistent Volume and Persistent Volume Claim
Copy the following yaml to a `pv.yaml` file and apply to the cluster to create a persistent volume and a persistent
@@ -316,6 +324,12 @@ so the operator (or Deployment controller) can bring a matching version up.
Bump `BALLISTA_PROTOCOL_VERSION` only on releases that change the
executor↔scheduler wire format. Most releases will not bump it.
+### Stopping executors
+
+Executors finish their running tasks before exiting on SIGTERM. If tasks can run longer than 30
+seconds, raise the executor pods' `terminationGracePeriodSeconds` and the scheduler's
+`--executor-termination-grace-period` to match.
+
### Rolling both Deployments together
The scheduler and executor live in separate Deployments; there is no single
@@ -392,6 +406,225 @@ Use the following command to set up port-forwarding.
kubectl port-forward service/ballista-scheduler 50050:50050
```
+## Serving Results to Clients Outside the Cluster
+
+Port forwarding the scheduler is enough to submit queries, but by default clients fetch query
+results directly from the executors, and executor pods are not reachable from outside the cluster.
+Deploy a result service that clients can reach, and have the scheduler point them at it. See
+[Fetching Query Results](../scheduler.md#fetching-query-results) for how it works.
+
+Save the following as `result-service.yaml`:
+
+```yaml
+apiVersion: v1
+kind: Service
+metadata:
+ name: ballista-result-service
+spec:
+ ports:
+ - port: 50055
+ name: flight
+ # Plaintext HTTP/2, so a Gateway API route speaks gRPC to the pods.
+ appProtocol: kubernetes.io/h2c
+ selector:
+ app: ballista-result-service
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: ballista-result-service
+spec:
+ # Replicas are interchangeable: each forwards any fetch to the executor that
+ # holds the partition and keeps no state, so scale with result traffic.
+ replicas: 2
+ # Start a replacement before stopping an old pod, so rollouts keep capacity.
+ strategy:
+ type: RollingUpdate
+ rollingUpdate:
+ maxSurge: 1
+ maxUnavailable: 0
+ selector:
+ matchLabels:
+ app: ballista-result-service
+ template:
+ metadata:
+ labels:
+ app: ballista-result-service
+ ballista-cluster: ballista
+ spec:
+ # Prefer placing replicas on different nodes, so they don't share a
+ # node's network bandwidth.
+ topologySpreadConstraints:
+ - maxSkew: 1
+ topologyKey: kubernetes.io/hostname
+ whenUnsatisfiable: ScheduleAnyway
+ labelSelector:
+ matchLabels:
+ app: ballista-result-service
+ # Leaves 10s of headroom after the preStop sleep (5s) and the drain
+ # bound (45s).
+ terminationGracePeriodSeconds: 60
+ containers:
+ - name: ballista-result-service
+ image: <your-repo>/datafusion-ballista-result-service:latest
+ args:
+ - "--bind-port=50055"
+ # After SIGTERM, stop accepting connections and give in-flight
+ # result streams up to 45s to finish.
+ - "--graceful-shutdown-timeout-seconds=45"
+ lifecycle:
+ # Kubernetes removes a terminating pod from the Service's endpoints
+ # in parallel with shutting it down, and kube-proxy and gateways
+ # learn of the removal asynchronously, so new connections can still
+ # arrive for a few seconds. Sleeping keeps the pod accepting them
+ # until routing catches up; SIGTERM follows once the hook returns.
+ preStop:
+ sleep:
+ seconds: 5
+ ports:
+ - containerPort: 50055
+ name: flight
+ livenessProbe:
+ httpGet:
+ path: /healthz
+ port: 50055
+ failureThreshold: 3
+ periodSeconds: 10
+ readinessProbe:
+ httpGet:
+ path: /readyz
+ port: 50055
+ failureThreshold: 3
+ periodSeconds: 5
+---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+ name: ballista-result-service
+spec:
+ # Node drains evict at most one replica at a time.
+ minAvailable: 1
+ selector:
+ matchLabels:
+ app: ballista-result-service
+```
+
+When a pod is replaced or moved to another node, Kubernetes removes it from the Service's
+endpoints at the same time as it starts shutting the pod down. kube-proxy and any gateway pick up
+that removal asynchronously, so for a few seconds new connections can still be routed to the pod.
+The `preStop` sleep keeps the pod accepting connections during that window. On SIGTERM, which
+follows the sleep, it stops accepting connections and lets in-flight result streams finish, for up
+to `--graceful-shutdown-timeout-seconds` (10 by default), before exiting. The rolling-update
+strategy and the PodDisruptionBudget keep at least one replica serving throughout.
+
+Then add `--advertise-flight-endpoint` to the scheduler container's `args` in `cluster.yaml`. Which
+address to advertise depends on how clients reach the cluster.
+
+### Through port forwarding
+
+For a client on your own machine, forward a local port to the result service as well as to the
+scheduler, and advertise the local address:
+
+```yaml
+args: ["--bind-port=50050", "--advertise-flight-endpoint=127.0.0.1:50055"]
+```
+
+```bash
+kubectl apply -f result-service.yaml
+kubectl apply -f cluster.yaml
+kubectl port-forward service/ballista-scheduler 50050:50050 &
+kubectl port-forward service/ballista-result-service 50055:50055 &
+```
+
+`kubectl port-forward` sends all traffic to a single pod, so use this for testing only.
+
+### Through a TLS-terminating gateway
+
+The result service doesn't authenticate result fetches, and it connects to executors in
+plaintext. Make the gateway reachable only by trusted clients, for example through an internal
+load balancer or the gateway's access restrictions, and limit what the service can reach as
+described in [Restricting what the result service can reach](#restricting-what-the-result-service-can-reach).
+
+In production, expose the result service through the [Gateway API] with a Gateway that terminates
+TLS and a `GRPCRoute` to the result service, and advertise the gateway's address. TLS covers the
+connections from clients to the gateway. Behind it, the result service pods serve plaintext and
+connect to executors in plaintext.
+
+```yaml
+args:
+ - "--bind-port=50050"
+ - "--advertise-flight-endpoint=ballista-results.example.com:443"
+```
+
+```yaml
+apiVersion: gateway.networking.k8s.io/v1
+kind: Gateway
+metadata:
+ name: ballista-results
+spec:
+ gatewayClassName: <your-gateway-class>
+ listeners:
+ - name: grpc
+ protocol: HTTPS
+ port: 443
+ hostname: ballista-results.example.com
+ tls:
+ mode: Terminate
+ certificateRefs:
+ - name: ballista-results-tls
+---
+apiVersion: gateway.networking.k8s.io/v1
+kind: GRPCRoute
+metadata:
+ name: ballista-result-service
+spec:
+ parentRefs:
+ - name: ballista-results
+ hostnames:
+ - ballista-results.example.com
+ rules:
+ - backendRefs:
+ - name: ballista-result-service
+ port: 50055
+```
+
+Result streams are long-lived, so make sure the gateway implementation's idle and stream timeouts
+allow for the largest result. Clients fetch results over TLS by setting `ballista.client.use_tls`
+to `true`, with TLS roots for the gateway's certificate supplied through a gRPC endpoint override.
+That takes a client built with one of tonic's TLS features, as in the [mTLS cluster example].
+
+[gateway api]: https://gateway-api.sigs.k8s.io/
+[mtls cluster example]: https://github.com/apache/datafusion-ballista/blob/main/examples/examples/mtls-cluster.rs
+
+### Restricting what the result service can reach
+
+The result service checks no credentials, and it dials whichever executor address a fetch ticket
+names without checking that the address belongs to the cluster, so a forged ticket can make it
+connect to an arbitrary host. Keep it on networks you trust and, where your cluster enforces
+network policies, allow it to connect only to the executors' Arrow Flight port:
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+ name: ballista-result-service-egress
+spec:
+ podSelector:
+ matchLabels:
+ app: ballista-result-service
+ policyTypes: ["Egress"]
+ egress:
+ - to:
+ - podSelector:
+ matchLabels:
+ app: ballista-executor
+ ports:
+ - port: 50051
+ protocol: TCP
+```
+
+If executors register with host names rather than pod IPs (`--external-host`), also allow DNS.
+
## Deleting the Ballista Cluster
Run the following kubectl command to delete the cluster.
@@ -400,6 +633,12 @@ Run the following kubectl command to delete the cluster.
kubectl delete -f cluster.yaml
```
+If you deployed a result service, delete it too:
+
+```bash
+kubectl delete -f result-service.yaml
+```
+
## Autoscaling Executors
Ballista supports autoscaling for executors through [Keda](http://keda.sh). Keda allows for the scaling of a
diff --git a/_sources/user-guide/scheduler.md.txt b/_sources/user-guide/scheduler.md.txt
index f2dfc44ef..544f5ec7d 100644
--- a/_sources/user-guide/scheduler.md.txt
+++ b/_sources/user-guide/scheduler.md.txt
@@ -41,48 +41,67 @@ from `--external-host` and `--bind-port`.
By default a client fetches the result partitions of a query directly from the executors that
produced them, over Arrow Flight. This keeps the scheduler out of the data path, but it requires
every client to have network access to every executor — which is not the case in isolated
-environments where only the scheduler is reachable.
+environments such as Kubernetes, where clients can usually reach only a few entry points.
-For those deployments the scheduler can advertise a different address for clients to fetch results
-from, and can optionally host an Arrow Flight proxy itself.
+For those deployments, run a result service (`ballista-result-service`) that clients can reach, and
+have the scheduler advertise its address:
-> Note: this is plain Arrow Flight, used to move result partitions. It is not Flight SQL, which was
-> removed in Ballista 46.0.0 — the proxy only serves Ballista's `FetchPartition` action, so generic
-> Flight SQL or JDBC clients cannot use this endpoint.
-
-Two independent options control this:
+```bash
+ballista-result-service --bind-port 50055
+ballista-scheduler --advertise-flight-endpoint ballista-results.example.com:50055
+```
-| Option | Description |
-| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `--enable-embedded-flight-proxy` | Runs an Arrow Flight proxy inside the scheduler process, on the scheduler's own host and port. The proxy forwards each fetch to the executor that owns the partition. |
-| `--advertise-flight-endpoint` | The `HOST:PORT` address clients are told to fetch results from, instead of the executors. Use it to point clients at a load balancer or a standalone proxy. |
+Clients then fetch each partition from the result service, which forwards the fetch to the executor
+that holds it, so the scheduler serves no result data. The result service holds no state: scale it
+by running more replicas behind a gRPC-aware load balancer and advertising the load balancer's
+address. For Kubernetes manifests, see
+[Serving Results to Clients Outside the Cluster](deployment/kubernetes.md#serving-results-to-clients-outside-the-cluster).
-The first controls whether a proxy _runs_; the second controls what clients are _told_. Setting
-both is a supported combination: the embedded proxy runs, and clients are pointed at the advertised
-address, which is how you put a load balancer in front of one or more schedulers.
+### Behind a TLS-terminating gateway
-To let clients fetch results through the scheduler itself:
+To serve results through a gateway or load balancer that terminates TLS, advertise its address. TLS
+covers the connections from clients to the gateway. The result service replicas behind it serve
+plaintext and connect to executors in plaintext, and they don't authenticate fetches, so make the
+gateway reachable only by trusted clients:
```bash
-ballista-scheduler --enable-embedded-flight-proxy
+ballista-result-service --bind-port 50055
+ballista-scheduler --advertise-flight-endpoint ballista-results.example.com:443
```
-To point clients at a load balancer that fronts the schedulers:
+Clients choose TLS for result fetches themselves, with `ballista.client.use_tls`. That takes a client
+built with one of tonic's TLS features, and TLS roots for the gateway's certificate supplied through
+a gRPC endpoint override, as the [mTLS cluster example] does. `ballista.client.use_tls` applies to every result fetch a client
+makes, including direct fetches from executors. The gateway must forward gRPC (HTTP/2) to the
+replicas, for example with a Gateway API `GRPCRoute`, and allow long-lived streams.
-```bash
-ballista-scheduler --enable-embedded-flight-proxy \
- --advertise-flight-endpoint ballista-flight.example.com:50050
+[mtls cluster example]: https://github.com/apache/datafusion-ballista/blob/main/examples/examples/mtls-cluster.rs
+
+> Note: the advertised endpoint serves plain Arrow Flight `DoGet` for Ballista's own partition-fetch
+> tickets. Generic Flight SQL, JDBC, and ADBC clients connect to the scheduler's
+> [Flight SQL frontend](flightsql.md) instead.
+
+```{warning}
+The result service checks no credentials, and it dials whichever executor address a fetch ticket
+names without checking that the address belongs to the cluster. Keep it on a trusted network.
```
-Both options are disabled by default, and the embedded proxy should be enabled deliberately: it
-puts result traffic on the scheduler's process and thread pool, competing with query planning and
-task scheduling. Under load this is a known source of scheduler congestion, so prefer a separate
-proxy — advertised with `--advertise-flight-endpoint` — for clusters where result volume is
-significant.
+| Option | Description |
+| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `--advertise-flight-endpoint` | The `HOST:PORT` address clients are told to fetch results from, instead of the executors: a result service, or a load balancer in front of one. |
+| `--enable-embedded-flight-proxy` | **Deprecated, to be removed in 57.0.0.** Runs an Arrow Flight proxy inside the scheduler process, on the scheduler's own host and port, and points clients at the scheduler for results. |
+
+### The embedded proxy (deprecated)
+
+The scheduler can still proxy results itself with `--enable-embedded-flight-proxy`. This puts result
+traffic on the scheduler's process and thread pool, competing with query planning and task
+scheduling, and it scales only with the scheduler. It is deprecated in 56.0.0 and will be removed in
+57.0.0; the scheduler logs a warning at startup while it is enabled. To migrate, deploy a result
+service and set `--advertise-flight-endpoint` to its address.
> `--advertise-flight-sql-endpoint` is accepted as a deprecated alias of
-> `--advertise-flight-endpoint`. Passing either flag with no value used to start the embedded proxy;
-> that is deprecated too and logs a warning — use `--enable-embedded-flight-proxy` instead.
+> `--advertise-flight-endpoint`. Passing either flag with no value starts the deprecated embedded
+> proxy, is itself deprecated, and logs a warning.
## REST API
diff --git a/changelog/0.10.0.html b/changelog/0.10.0.html
index 324e1ceb9..5d54b9fd3 100644
--- a/changelog/0.10.0.html
+++ b/changelog/0.10.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.11.0.html b/changelog/0.11.0.html
index a484f0606..1b3fc3b17 100644
--- a/changelog/0.11.0.html
+++ b/changelog/0.11.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.12.0.html b/changelog/0.12.0.html
index dda7d8056..04a463aed 100644
--- a/changelog/0.12.0.html
+++ b/changelog/0.12.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.5.0.html b/changelog/0.5.0.html
index 4bacd3850..dd2853bb1 100644
--- a/changelog/0.5.0.html
+++ b/changelog/0.5.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.6.0.html b/changelog/0.6.0.html
index e852c3144..dde35207c 100644
--- a/changelog/0.6.0.html
+++ b/changelog/0.6.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.7.0.html b/changelog/0.7.0.html
index 53790f168..8558a0ed7 100644
--- a/changelog/0.7.0.html
+++ b/changelog/0.7.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.8.0.html b/changelog/0.8.0.html
index 2b63ca42f..3484152e7 100644
--- a/changelog/0.8.0.html
+++ b/changelog/0.8.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/0.9.0.html b/changelog/0.9.0.html
index cd216be13..25f09d7ec 100644
--- a/changelog/0.9.0.html
+++ b/changelog/0.9.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/43.0.0.html b/changelog/43.0.0.html
index e297628fd..72e5d7fa4 100644
--- a/changelog/43.0.0.html
+++ b/changelog/43.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/44.0.0.html b/changelog/44.0.0.html
index 16a3b511d..4845c03a4 100644
--- a/changelog/44.0.0.html
+++ b/changelog/44.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/45.0.0.html b/changelog/45.0.0.html
index 1d3d81101..57844466b 100644
--- a/changelog/45.0.0.html
+++ b/changelog/45.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/46.0.0.html b/changelog/46.0.0.html
index cd873a5aa..6d235b70f 100644
--- a/changelog/46.0.0.html
+++ b/changelog/46.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/47.0.0.html b/changelog/47.0.0.html
index 0585b81d2..f3a9ac3ec 100644
--- a/changelog/47.0.0.html
+++ b/changelog/47.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/48.0.0.html b/changelog/48.0.0.html
index 23903a03c..43059a47d 100644
--- a/changelog/48.0.0.html
+++ b/changelog/48.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/49.0.0.html b/changelog/49.0.0.html
index 308ddd384..ed790d0d9 100644
--- a/changelog/49.0.0.html
+++ b/changelog/49.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/50.0.0.html b/changelog/50.0.0.html
index e9bee1b63..808b8e282 100644
--- a/changelog/50.0.0.html
+++ b/changelog/50.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/51.0.0.html b/changelog/51.0.0.html
index 36d0e8825..05c9f9ea6 100644
--- a/changelog/51.0.0.html
+++ b/changelog/51.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/52.0.0.html b/changelog/52.0.0.html
index 0ca54877a..3e764b9de 100644
--- a/changelog/52.0.0.html
+++ b/changelog/52.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/53.0.0.html b/changelog/53.0.0.html
index 0ca9d7ce5..242487f8e 100644
--- a/changelog/53.0.0.html
+++ b/changelog/53.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/54.0.0.html b/changelog/54.0.0.html
index fbb66574d..578f1d844 100644
--- a/changelog/54.0.0.html
+++ b/changelog/54.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/54.1.0.html b/changelog/54.1.0.html
index 040a18140..757c74735 100644
--- a/changelog/54.1.0.html
+++ b/changelog/54.1.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/55.0.0.html b/changelog/55.0.0.html
index 85d9d8087..da261b5e6 100644
--- a/changelog/55.0.0.html
+++ b/changelog/55.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/changelog/index.html b/changelog/index.html
index 38e898d29..efa486934 100644
--- a/changelog/index.html
+++ b/changelog/index.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/community/adopters.html b/community/adopters.html
index 181dd3656..466a99144 100644
--- a/community/adopters.html
+++ b/community/adopters.html
@@ -295,6 +295,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/community/communication.html b/community/communication.html
index 6f4023e93..5f7377f62 100644
--- a/community/communication.html
+++ b/community/communication.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/architecture.html b/contributors-guide/architecture.html
index 64c1ca071..ba17f708e 100644
--- a/contributors-guide/architecture.html
+++ b/contributors-guide/architecture.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/benchmarking/ballista.html b/contributors-guide/benchmarking/ballista.html
index b85f785a6..33d551b64 100644
--- a/contributors-guide/benchmarking/ballista.html
+++ b/contributors-guide/benchmarking/ballista.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/benchmarking/comet.html b/contributors-guide/benchmarking/comet.html
index 373724351..593e24591 100644
--- a/contributors-guide/benchmarking/comet.html
+++ b/contributors-guide/benchmarking/comet.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/benchmarking/environment.html b/contributors-guide/benchmarking/environment.html
index b4997d85d..d4b29e4d5 100644
--- a/contributors-guide/benchmarking/environment.html
+++ b/contributors-guide/benchmarking/environment.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/benchmarking/index.html b/contributors-guide/benchmarking/index.html
index 9c5c365eb..f34232fd1 100644
--- a/contributors-guide/benchmarking/index.html
+++ b/contributors-guide/benchmarking/index.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/benchmarking/spark.html b/contributors-guide/benchmarking/spark.html
index 4b1851524..29733e5c5 100644
--- a/contributors-guide/benchmarking/spark.html
+++ b/contributors-guide/benchmarking/spark.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/benchmarking/trino.html b/contributors-guide/benchmarking/trino.html
index 1b4ff1f21..f7e5a528f 100644
--- a/contributors-guide/benchmarking/trino.html
+++ b/contributors-guide/benchmarking/trino.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/code-organization.html b/contributors-guide/code-organization.html
index 46f0ac618..94d47f462 100644
--- a/contributors-guide/code-organization.html
+++ b/contributors-guide/code-organization.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/dataframe-cache.html b/contributors-guide/dataframe-cache.html
index 9f945f3a4..b90d00360 100644
--- a/contributors-guide/dataframe-cache.html
+++ b/contributors-guide/dataframe-cache.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/development.html b/contributors-guide/development.html
index c70135af9..87e0a6ddf 100644
--- a/contributors-guide/development.html
+++ b/contributors-guide/development.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/shuffle.html b/contributors-guide/shuffle.html
index 4d7955da1..192009bb4 100644
--- a/contributors-guide/shuffle.html
+++ b/contributors-guide/shuffle.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/contributors-guide/user-personas.html b/contributors-guide/user-personas.html
index 5d6c9a46d..dca803692 100644
--- a/contributors-guide/user-personas.html
+++ b/contributors-guide/user-personas.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/genindex.html b/genindex.html
index 59894f91f..3469096b7 100644
--- a/genindex.html
+++ b/genindex.html
@@ -291,6 +291,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/index.html b/index.html
index 28ec133c5..048bc7fa1 100644
--- a/index.html
+++ b/index.html
@@ -297,6 +297,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/objects.inv b/objects.inv
index 645ae924f..9c2d96137 100644
Binary files a/objects.inv and b/objects.inv differ
diff --git a/search.html b/search.html
index 0c9746cd7..3ae4bfd44 100644
--- a/search.html
+++ b/search.html
@@ -290,6 +290,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/searchindex.js b/searchindex.js
index 4fb3b203f..ac6ea124e 100644
--- a/searchindex.js
+++ b/searchindex.js
@@ -1 +1 @@
-Search.setIndex({"alltitles":{"API changes":[[39,"api-changes"]],"AQE join behavior (opt-in)":[[38,"aqe-join-behavior-opt-in"]],"Adaptive Query Execution":[[62,"adaptive-query-execution"]],"Adaptive Query Execution (AQE)":[[25,"adaptive-query-execution-aqe"]],"Adding a persona":[[36,"adding-a-persona"]],"Apache DataFusion Ballista":[[37,null]],"Apache DataFusion Ballista 0.10.0 Changelog":[[0,null]],"Apache DataFusion Ballista 0.11.0 Changelog":[[1,null]],"Apache DataFusion Ballista 0.12 [...]
\ No newline at end of file
+Search.setIndex({"alltitles":{"API changes":[[39,"api-changes"]],"AQE join behavior (opt-in)":[[38,"aqe-join-behavior-opt-in"]],"Adaptive Query Execution":[[63,"adaptive-query-execution"]],"Adaptive Query Execution (AQE)":[[25,"adaptive-query-execution-aqe"]],"Adding a persona":[[36,"adding-a-persona"]],"Apache DataFusion Ballista":[[37,null]],"Apache DataFusion Ballista 0.10.0 Changelog":[[0,null]],"Apache DataFusion Ballista 0.11.0 Changelog":[[1,null]],"Apache DataFusion Ballista 0.12 [...]
\ No newline at end of file
diff --git a/upgrading/54.0.0.html b/upgrading/54.0.0.html
index 2fb99de13..d6823d91a 100644
--- a/upgrading/54.0.0.html
+++ b/upgrading/54.0.0.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="current nav bd-sidenav">
<li class="toctree-l1 current active has-children"><a class="reference internal" href="index.html">Upgrade Guides</a><details open="open"><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul class="current">
+<li class="toctree-l2"><a class="reference internal" href="56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2 current active"><a aria-current="page" class="current reference internal" href="#">Ballista 54.0.0</a></li>
</ul>
diff --git a/upgrading/55.0.0.html b/upgrading/55.0.0.html
index daac0ac80..97f15ab6f 100644
--- a/upgrading/55.0.0.html
+++ b/upgrading/55.0.0.html
@@ -46,7 +46,7 @@
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Upgrade Guides" href="54.0.0.html" />
- <link rel="prev" title="Upgrade Guides" href="index.html" />
+ <link rel="prev" title="Upgrade Guides" href="56.0.0.html" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="color-scheme" content="light dark"/>
<meta name="docsearch:language" content="en"/>
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="current nav bd-sidenav">
<li class="toctree-l1 current active has-children"><a class="reference internal" href="index.html">Upgrade Guides</a><details open="open"><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul class="current">
+<li class="toctree-l2"><a class="reference internal" href="56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2 current active"><a aria-current="page" class="current reference internal" href="#">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -702,7 +703,7 @@ struct.</p>
<div class="prev-next-area">
<a class="left-prev"
- href="index.html"
+ href="56.0.0.html"
title="previous page">
<i class="fa-solid fa-angle-left"></i>
<div class="prev-next-info">
diff --git a/community/communication.html b/upgrading/56.0.0.html
similarity index 72%
copy from community/communication.html
copy to upgrading/56.0.0.html
index 6f4023e93..e36b0684e 100644
--- a/community/communication.html
+++ b/upgrading/56.0.0.html
@@ -7,7 +7,7 @@
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /><meta name="viewport" content="width=device-width, initial-scale=1" />
- <title>Communication — Apache DataFusion Ballista documentation</title>
+ <title>Upgrade Guides — Apache DataFusion Ballista documentation</title>
@@ -41,12 +41,12 @@
<script src="../_static/documentation_options.js?v=5929fcd5"></script>
<script src="../_static/doctools.js?v=fd6eb6e6"></script>
<script src="../_static/sphinx_highlight.js?v=6ffebe34"></script>
- <script>DOCUMENTATION_OPTIONS.pagename = 'community/communication';</script>
+ <script>DOCUMENTATION_OPTIONS.pagename = 'upgrading/56.0.0';</script>
<script>DOCUMENTATION_OPTIONS.search_as_you_type = false;</script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Who Uses Ballista" href="adopters.html" />
- <link rel="prev" title="Trino Benchmark Configuration" href="../contributors-guide/benchmarking/trino.html" />
+ <link rel="next" title="Upgrade Guides" href="55.0.0.html" />
+ <link rel="prev" title="Upgrade Guides" href="index.html" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="color-scheme" content="light dark"/>
<meta name="docsearch:language" content="en"/>
@@ -294,10 +294,11 @@
</details></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
-<ul class="nav bd-sidenav">
-<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
-<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
-<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
+<ul class="current nav bd-sidenav">
+<li class="toctree-l1 current active has-children"><a class="reference internal" href="index.html">Upgrade Guides</a><details open="open"><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul class="current">
+<li class="toctree-l2 current active"><a aria-current="page" class="current reference internal" href="#">Ballista 56.0.0</a></li>
+<li class="toctree-l2"><a class="reference internal" href="55.0.0.html">Ballista 55.0.0</a></li>
+<li class="toctree-l2"><a class="reference internal" href="54.0.0.html">Ballista 54.0.0</a></li>
</ul>
</details></li>
</ul>
@@ -320,9 +321,9 @@
<li class="toctree-l1"><a class="reference external" href="https://github.com/apache/datafusion-ballista/">Source code</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Community</span></p>
-<ul class="current nav bd-sidenav">
-<li class="toctree-l1 current active"><a aria-current="page" class="current reference internal" href="#">Communication</a></li>
-<li class="toctree-l1"><a class="reference internal" href="adopters.html">Who Uses Ballista</a></li>
+<ul class="nav bd-sidenav">
+<li class="toctree-l1"><a class="reference internal" href="../community/communication.html">Communication</a></li>
+<li class="toctree-l1"><a class="reference internal" href="../community/adopters.html">Who Uses Ballista</a></li>
<li class="toctree-l1"><a class="reference external" href="https://github.com/apache/datafusion-ballista/issues">Issue tracker</a></li>
<li class="toctree-l1"><a class="reference external" href="https://github.com/apache/datafusion-ballista/blob/main/CODE_OF_CONDUCT.md">Code of conduct</a></li>
</ul>
@@ -367,7 +368,10 @@
<i class="fa-solid fa-home"></i>
</a>
</li>
- <li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Communication</span></li>
+
+ <li class="breadcrumb-item"><a href="index.html" class="nav-link">Upgrade Guides</a></li>
+
+ <li class="breadcrumb-item active" aria-current="page"><span class="ellipsis">Upgrade Guides</span></li>
</ul>
</nav>
</div>
@@ -402,25 +406,75 @@
specific language governing permissions and limitations
under the License.
-->
-<section id="communication">
-<h1>Communication<a class="headerlink" href="#communication" title="Link to this heading">#</a></h1>
-<p>We welcome participation from everyone and encourage you to join us, ask
-questions, and get involved.</p>
-<p>All participation in the Apache DataFusion Ballista project is governed by the
-Apache Software Foundation’s <a class="reference external" href="https://www.apache.org/foundation/policies/conduct.html">code of
-conduct</a>.</p>
-<p>We use the same communication channels as the main DataFusion project:</p>
-<p><a class="reference external" href="https://datafusion.apache.org/contributor-guide/communication.html">https://datafusion.apache.org/contributor-guide/communication.html</a></p>
-<section id="contributing">
-<h2>Contributing<a class="headerlink" href="#contributing" title="Link to this heading">#</a></h2>
-<p>Our source code is hosted on
-<a class="reference external" href="https://github.com/apache/datafusion-ballista">GitHub</a>. More information on contributing is in
-the <a class="reference external" href="https://github.com/apache/datafusion-ballista/blob/main/CONTRIBUTING.md">Contribution Guide</a>
-, and we have curated a <a class="reference external" href="https://github.com/apache/datafusion-ballista/contribute">good-first-issue</a>
-list to help you get started.</p>
-<p>We use GitHub issues for maintaining a queue of development work and as the
-public record. We often use Google docs, Github issues and pull requests for
-quick and small design discussions. For major design change proposals, we encourage you to write a rfc.</p>
+<section id="upgrade-guides">
+<h1>Upgrade Guides<a class="headerlink" href="#upgrade-guides" title="Link to this heading">#</a></h1>
+<section id="ballista-56-0-0">
+<h2>Ballista 56.0.0<a class="headerlink" href="#ballista-56-0-0" title="Link to this heading">#</a></h2>
+<p><strong>Note:</strong> Ballista <code class="docutils literal notranslate"><span class="pre">56.0.0</span></code> has not been released yet. The information provided
+in this section pertains to features and changes that have already been merged
+to the main branch and are awaiting release in this version.</p>
+<section id="deprecations">
+<h3>Deprecations<a class="headerlink" href="#deprecations" title="Link to this heading">#</a></h3>
+<section id="the-scheduler-s-embedded-flight-proxy-is-deprecated">
+<h4>The scheduler’s embedded Flight proxy is deprecated<a class="headerlink" href="#the-scheduler-s-embedded-flight-proxy-is-deprecated" title="Link to this heading">#</a></h4>
+<p><code class="docutils literal notranslate"><span class="pre">--enable-embedded-flight-proxy</span></code> (and <code class="docutils literal notranslate"><span class="pre">SchedulerConfig::enable_embedded_flight_proxy</span></code>)
+is deprecated and will be removed in 57.0.0. The proxy puts result traffic on the
+scheduler’s process, where it competes with query planning and task scheduling,
+and it scales only with the scheduler.</p>
+<p>Deployments where clients cannot reach the executors should instead run the new
+<code class="docutils literal notranslate"><span class="pre">ballista-result-service</span></code> binary, a stateless Arrow Flight service that forwards
+each result fetch to the executor holding the partition, and point the
+scheduler’s <code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code> at it, or at a load balancer in front
+of several replicas:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ballista-result-service<span class="w"> </span>--bind-port<span class="w"> </span><span class="m">50055</span>
+ballista-scheduler<span class="w"> </span>--advertise-flight-endpoint<span class="w"> </span>ballista-results.example.com:50055
+</pre></div>
+</div>
+<p>For Kubernetes, see the example manifests in
+<a class="reference internal" href="../user-guide/deployment/kubernetes.html#serving-results-to-clients-outside-the-cluster"><span class="std std-ref">Serving Results to Clients Outside the Cluster</span></a>.</p>
+<p>Clients need no changes. The embedded proxy keeps working throughout 56.x, so
+you can compare the two and roll back without downgrading Ballista; the
+scheduler logs a warning at startup while it is enabled.</p>
+<p>If you embed the scheduler and configure the proxy’s connections to executors
+with <code class="docutils literal notranslate"><span class="pre">SchedulerConfig::use_tls</span></code> and <code class="docutils literal notranslate"><span class="pre">override_create_grpc_client_endpoint</span></code>, for
+example for mTLS, run the result service from your own binary with
+<code class="docutils literal notranslate"><span class="pre">ballista_result_service::serve</span></code>. Its <code class="docutils literal notranslate"><span class="pre">ResultServiceConfig</span></code> takes the same two
+settings. The <code class="docutils literal notranslate"><span class="pre">ballista-result-service</span></code> binary connects to executors in
+plaintext, as the stock scheduler binary does.</p>
+<p>Also deprecated:</p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">SchedulerConfig::with_enable_embedded_flight_proxy</span></code></p></li>
+<li><p>passing <code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code> with no value, which starts the
+embedded proxy</p></li>
+</ul>
+</section>
+</section>
+<section id="behavior-changes">
+<h3>Behavior changes<a class="headerlink" href="#behavior-changes" title="Link to this heading">#</a></h3>
+<section id="containers-stop-on-sigterm">
+<h4>Containers stop on SIGTERM<a class="headerlink" href="#containers-stop-on-sigterm" title="Link to this heading">#</a></h4>
+<p>The images’ entrypoints now <code class="docutils literal notranslate"><span class="pre">exec</span></code> their binaries, and each binary handles
+SIGTERM, so <code class="docutils literal notranslate"><span class="pre">docker</span> <span class="pre">stop</span></code> and pod termination reach the Ballista process.
+Before, the signal never did, and containers waited out their grace period to
+be killed.</p>
+<ul class="simple">
+<li><p>Executors now drain their running tasks before exiting. If tasks can run
+longer than 30 seconds, raise the executor pods’
+<code class="docutils literal notranslate"><span class="pre">terminationGracePeriodSeconds</span></code> and the scheduler’s
+<code class="docutils literal notranslate"><span class="pre">--executor-termination-grace-period</span></code> to match.</p></li>
+<li><p>The scheduler exits on SIGTERM or Ctrl-C, without draining its connections.</p></li>
+<li><p>The CLI exits on SIGTERM.</p></li>
+<li><p>The standalone image stops its executor first, then its scheduler.</p></li>
+</ul>
+</section>
+<section id="undecodable-fetch-tickets-are-rejected-as-invalidargument">
+<h4>Undecodable fetch tickets are rejected as <code class="docutils literal notranslate"><span class="pre">InvalidArgument</span></code><a class="headerlink" href="#undecodable-fetch-tickets-are-rejected-as-invalidargument" title="Link to this heading">#</a></h4>
+<p>Executors, the embedded proxy, and the result service now share one
+implementation of partition fetches. A <code class="docutils literal notranslate"><span class="pre">DoGet</span></code> ticket that does not decode as a
+Ballista action is reported with the gRPC status <code class="docutils literal notranslate"><span class="pre">InvalidArgument</span></code> everywhere;
+executors used to report it as <code class="docutils literal notranslate"><span class="pre">Internal</span></code>.</p>
+</section>
+</section>
</section>
</section>
@@ -435,20 +489,20 @@ quick and small design discussions. For major design change proposals, we encour
<div class="prev-next-area">
<a class="left-prev"
- href="../contributors-guide/benchmarking/trino.html"
+ href="index.html"
title="previous page">
<i class="fa-solid fa-angle-left"></i>
<div class="prev-next-info">
<p class="prev-next-subtitle">previous</p>
- <p class="prev-next-title">Trino Benchmark Configuration</p>
+ <p class="prev-next-title">Upgrade Guides</p>
</div>
</a>
<a class="right-next"
- href="adopters.html"
+ href="55.0.0.html"
title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
- <p class="prev-next-title">Who Uses Ballista</p>
+ <p class="prev-next-title">Upgrade Guides</p>
</div>
<i class="fa-solid fa-angle-right"></i>
</a>
@@ -471,7 +525,18 @@ quick and small design discussions. For major design change proposals, we encour
</div>
<nav id="pst-page-toc-nav" class="page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="pst-show_toc_level nav section-nav flex-column">
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#contributing">Contributing</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#ballista-56-0-0">Ballista 56.0.0</a><ul class="nav section-nav flex-column">
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#deprecations">Deprecations</a><ul class="nav section-nav flex-column">
+<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#the-scheduler-s-embedded-flight-proxy-is-deprecated">The scheduler’s embedded Flight proxy is deprecated</a></li>
+</ul>
+</li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#behavior-changes">Behavior changes</a><ul class="nav section-nav flex-column">
+<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#containers-stop-on-sigterm">Containers stop on SIGTERM</a></li>
+<li class="toc-h4 nav-item toc-entry"><a class="reference internal nav-link" href="#undecodable-fetch-tickets-are-rejected-as-invalidargument">Undecodable fetch tickets are rejected as <code class="docutils literal notranslate"><span class="pre">InvalidArgument</span></code></a></li>
+</ul>
+</li>
+</ul>
+</li>
</ul>
</nav></div>
@@ -479,7 +544,7 @@ quick and small design discussions. For major design change proposals, we encour
<div class="tocsection editthispage">
- <a href="https://github.com/apache/datafusion-ballista/edit/main/docs/source/community/communication.md">
+ <a href="https://github.com/apache/datafusion-ballista/edit/main/docs/source/upgrading/56.0.0.md">
<i class="fa-solid fa-pencil"></i>
@@ -494,7 +559,7 @@ quick and small design discussions. For major design change proposals, we encour
<div class="sidebar-secondary-item">
<div class="tocsection sourcelink">
- <a href="../_sources/community/communication.md.txt">
+ <a href="../_sources/upgrading/56.0.0.md.txt">
<i class="fa-solid fa-file-lines"></i> Show Source
</a>
</div>
diff --git a/upgrading/index.html b/upgrading/index.html
index 4e4a55e3e..58909faff 100644
--- a/upgrading/index.html
+++ b/upgrading/index.html
@@ -45,7 +45,7 @@
<script>DOCUMENTATION_OPTIONS.search_as_you_type = false;</script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
- <link rel="next" title="Upgrade Guides" href="55.0.0.html" />
+ <link rel="next" title="Upgrade Guides" href="56.0.0.html" />
<link rel="prev" title="Apache DataFusion Ballista 0.5.0 Changelog" href="../changelog/0.5.0.html" />
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<meta name="color-scheme" content="light dark"/>
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="current nav bd-sidenav">
<li class="toctree-l1 current active has-children"><a aria-current="page" class="current reference internal" href="#">Upgrade Guides</a><details open="open"><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -388,6 +389,7 @@
<h1>Upgrade Guides<a class="headerlink" href="#upgrade-guides" title="Link to this heading">#</a></h1>
<div class="toctree-wrapper compound">
<ul>
+<li class="toctree-l1"><a class="reference internal" href="56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l1"><a class="reference internal" href="55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l1"><a class="reference internal" href="54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -414,7 +416,7 @@
</div>
</a>
<a class="right-next"
- href="55.0.0.html"
+ href="56.0.0.html"
title="next page">
<div class="prev-next-info">
<p class="prev-next-subtitle">next</p>
diff --git a/user-guide/cli.html b/user-guide/cli.html
index 7e2d76fe7..e2fae4c3f 100644
--- a/user-guide/cli.html
+++ b/user-guide/cli.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/configs.html b/user-guide/configs.html
index 06b1f603c..bd5b64154 100644
--- a/user-guide/configs.html
+++ b/user-guide/configs.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/deployment/cargo-install.html b/user-guide/deployment/cargo-install.html
index e2257659d..a2741d379 100644
--- a/user-guide/deployment/cargo-install.html
+++ b/user-guide/deployment/cargo-install.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -413,6 +414,11 @@ the scheduler and executor crates.</p>
cargo<span class="w"> </span>install<span class="w"> </span>--locked<span class="w"> </span>ballista-executor
</pre></div>
</div>
+<p>If clients cannot reach the executors, also install the result service, which serves them query
+results instead (see <a class="reference internal" href="../scheduler.html#fetching-query-results"><span class="std std-ref">Fetching Query Results</span></a>):</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>cargo<span class="w"> </span>install<span class="w"> </span>--locked<span class="w"> </span>ballista-result-service
+</pre></div>
+</div>
<p>With these crates installed, it is now possible to start a scheduler process.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span><span class="nv">RUST_LOG</span><span class="o">=</span>info<span class="w"> </span>ballista-scheduler
</pre></div>
diff --git a/user-guide/deployment/docker-compose.html b/user-guide/deployment/docker-compose.html
index f3673b76a..509899b69 100644
--- a/user-guide/deployment/docker-compose.html
+++ b/user-guide/deployment/docker-compose.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/deployment/docker.html b/user-guide/deployment/docker.html
index c8b652462..6f57c5788 100644
--- a/user-guide/deployment/docker.html
+++ b/user-guide/deployment/docker.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -414,11 +415,14 @@
<li><p><a class="reference external" href="https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-standalone"><code class="docutils literal notranslate"><span class="pre">ghcr.io/apache/datafusion-ballista-standalone</span></code></a></p></li>
<li><p><a class="reference external" href="https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-scheduler"><code class="docutils literal notranslate"><span class="pre">ghcr.io/apache/datafusion-ballista-scheduler</span></code></a></p></li>
<li><p><a class="reference external" href="https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-executor"><code class="docutils literal notranslate"><span class="pre">ghcr.io/apache/datafusion-ballista-executor</span></code></a></p></li>
+<li><p><a class="reference external" href="https://github.com/apache/datafusion-ballista/pkgs/container/datafusion-ballista-result-service"><code class="docutils literal notranslate"><span class="pre">ghcr.io/apache/datafusion-ballista-result-service</span></code></a></p></li>
</ul>
<p>Pull the images needed for your deployment with the following commands:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>docker<span class="w"> </span>pull<span class="w"> </span>ghcr.io/apache/datafusion-ballista-standalone:latest
docker<span class="w"> </span>pull<span class="w"> </span>ghcr.io/apache/datafusion-ballista-scheduler:latest
docker<span class="w"> </span>pull<span class="w"> </span>ghcr.io/apache/datafusion-ballista-executor:latest
+<span class="c1"># Only needed if clients cannot reach the executors:</span>
+docker<span class="w"> </span>pull<span class="w"> </span>ghcr.io/apache/datafusion-ballista-result-service:latest
</pre></div>
</div>
<p>Alternatively run the following commands to clone the source repository and build the Docker images from source:</p>
@@ -432,6 +436,7 @@ docker<span class="w"> </span>pull<span class="w"> </span>ghcr.io/apache/datafus
<li><p><code class="docutils literal notranslate"><span class="pre">apache/datafusion-ballista-benchmarks:latest</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">apache/datafusion-ballista-cli:latest</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">apache/datafusion-ballista-executor:latest</span></code></p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">apache/datafusion-ballista-result-service:latest</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">apache/datafusion-ballista-scheduler:latest</span></code></p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">apache/datafusion-ballista-standalone:latest</span></code></p></li>
</ul>
diff --git a/user-guide/deployment/index.html b/user-guide/deployment/index.html
index 1c8f2f7b1..d5c959986 100644
--- a/user-guide/deployment/index.html
+++ b/user-guide/deployment/index.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -414,6 +415,7 @@
<li class="toctree-l2"><a class="reference internal" href="kubernetes.html#deploying-a-ballista-cluster">Deploying a Ballista Cluster</a></li>
<li class="toctree-l2"><a class="reference internal" href="kubernetes.html#health-probes-and-rolling-upgrades">Health Probes and Rolling Upgrades</a></li>
<li class="toctree-l2"><a class="reference internal" href="kubernetes.html#port-forwarding">Port Forwarding</a></li>
+<li class="toctree-l2"><a class="reference internal" href="kubernetes.html#serving-results-to-clients-outside-the-cluster">Serving Results to Clients Outside the Cluster</a></li>
<li class="toctree-l2"><a class="reference internal" href="kubernetes.html#deleting-the-ballista-cluster">Deleting the Ballista Cluster</a></li>
<li class="toctree-l2"><a class="reference internal" href="kubernetes.html#autoscaling-executors">Autoscaling Executors</a></li>
</ul>
diff --git a/user-guide/deployment/kubernetes.html b/user-guide/deployment/kubernetes.html
index d7271ef7e..c60eeb7e4 100644
--- a/user-guide/deployment/kubernetes.html
+++ b/user-guide/deployment/kubernetes.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -438,6 +439,12 @@ docker<span class="w"> </span>push<span class="w"> </span><your-repo>/data
docker<span class="w"> </span>push<span class="w"> </span><your-repo>/datafusion-ballista-executor:latest
</pre></div>
</div>
+<p>If clients will fetch results from outside the cluster, also publish the result service image (see
+<a class="reference internal" href="#serving-results-to-clients-outside-the-cluster"><span class="xref myst">Serving Results to Clients Outside the Cluster</span></a>):</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>docker<span class="w"> </span>tag<span class="w"> </span>apache/datafusion-ballista-result-service:latest<span class="w"> </span><your-repo>/datafusion-ballista-result-service:latest
+docker<span class="w"> </span>push<span class="w"> </span><your-repo>/datafusion-ballista-result-service:latest
+</pre></div>
+</div>
</section>
<section id="create-persistent-volume-and-persistent-volume-claim">
<h2>Create Persistent Volume and Persistent Volume Claim<a class="headerlink" href="#create-persistent-volume-and-persistent-volume-claim" title="Link to this heading">#</a></h2>
@@ -686,6 +693,12 @@ so the operator (or Deployment controller) can bring a matching version up.</p>
<p>Bump <code class="docutils literal notranslate"><span class="pre">BALLISTA_PROTOCOL_VERSION</span></code> only on releases that change the
executor↔scheduler wire format. Most releases will not bump it.</p>
</section>
+<section id="stopping-executors">
+<h3>Stopping executors<a class="headerlink" href="#stopping-executors" title="Link to this heading">#</a></h3>
+<p>Executors finish their running tasks before exiting on SIGTERM. If tasks can run longer than 30
+seconds, raise the executor pods’ <code class="docutils literal notranslate"><span class="pre">terminationGracePeriodSeconds</span></code> and the scheduler’s
+<code class="docutils literal notranslate"><span class="pre">--executor-termination-grace-period</span></code> to match.</p>
+</section>
<section id="rolling-both-deployments-together">
<h3>Rolling both Deployments together<a class="headerlink" href="#rolling-both-deployments-together" title="Link to this heading">#</a></h3>
<p>The scheduler and executor live in separate Deployments; there is no single
@@ -767,12 +780,219 @@ ballista-scheduler<span class="w"> </span>ClusterIP<span class="w"> </span><
</pre></div>
</div>
</section>
+<section id="serving-results-to-clients-outside-the-cluster">
+<h2>Serving Results to Clients Outside the Cluster<a class="headerlink" href="#serving-results-to-clients-outside-the-cluster" title="Link to this heading">#</a></h2>
+<p>Port forwarding the scheduler is enough to submit queries, but by default clients fetch query
+results directly from the executors, and executor pods are not reachable from outside the cluster.
+Deploy a result service that clients can reach, and have the scheduler point them at it. See
+<a class="reference internal" href="../scheduler.html#fetching-query-results"><span class="std std-ref">Fetching Query Results</span></a> for how it works.</p>
+<p>Save the following as <code class="docutils literal notranslate"><span class="pre">result-service.yaml</span></code>:</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">v1</span>
+<span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Service</span>
+<span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">ports</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">50055</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">flight</span>
+<span class="w"> </span><span class="c1"># Plaintext HTTP/2, so a Gateway API route speaks gRPC to the pods.</span>
+<span class="w"> </span><span class="nt">appProtocol</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">kubernetes.io/h2c</span>
+<span class="w"> </span><span class="nt">selector</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="nn">---</span>
+<span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">apps/v1</span>
+<span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Deployment</span>
+<span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="c1"># Replicas are interchangeable: each forwards any fetch to the executor that</span>
+<span class="w"> </span><span class="c1"># holds the partition and keeps no state, so scale with result traffic.</span>
+<span class="w"> </span><span class="nt">replicas</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">2</span>
+<span class="w"> </span><span class="c1"># Start a replacement before stopping an old pod, so rollouts keep capacity.</span>
+<span class="w"> </span><span class="nt">strategy</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">type</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">RollingUpdate</span>
+<span class="w"> </span><span class="nt">rollingUpdate</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">maxSurge</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">1</span>
+<span class="w"> </span><span class="nt">maxUnavailable</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">0</span>
+<span class="w"> </span><span class="nt">selector</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">matchLabels</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="w"> </span><span class="nt">template</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">labels</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="w"> </span><span class="nt">ballista-cluster</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista</span>
+<span class="w"> </span><span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="c1"># Prefer placing replicas on different nodes, so they don't share a</span>
+<span class="w"> </span><span class="c1"># node's network bandwidth.</span>
+<span class="w"> </span><span class="nt">topologySpreadConstraints</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">maxSkew</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">1</span>
+<span class="w"> </span><span class="nt">topologyKey</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">kubernetes.io/hostname</span>
+<span class="w"> </span><span class="nt">whenUnsatisfiable</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ScheduleAnyway</span>
+<span class="w"> </span><span class="nt">labelSelector</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">matchLabels</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="w"> </span><span class="c1"># Leaves 10s of headroom after the preStop sleep (5s) and the drain</span>
+<span class="w"> </span><span class="c1"># bound (45s).</span>
+<span class="w"> </span><span class="nt">terminationGracePeriodSeconds</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">60</span>
+<span class="w"> </span><span class="nt">containers</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain"><your-repo>/datafusion-ballista-result-service:latest</span>
+<span class="w"> </span><span class="nt">args</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="s">"--bind-port=50055"</span>
+<span class="w"> </span><span class="c1"># After SIGTERM, stop accepting connections and give in-flight</span>
+<span class="w"> </span><span class="c1"># result streams up to 45s to finish.</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="s">"--graceful-shutdown-timeout-seconds=45"</span>
+<span class="w"> </span><span class="nt">lifecycle</span><span class="p">:</span>
+<span class="w"> </span><span class="c1"># Kubernetes removes a terminating pod from the Service's endpoints</span>
+<span class="w"> </span><span class="c1"># in parallel with shutting it down, and kube-proxy and gateways</span>
+<span class="w"> </span><span class="c1"># learn of the removal asynchronously, so new connections can still</span>
+<span class="w"> </span><span class="c1"># arrive for a few seconds. Sleeping keeps the pod accepting them</span>
+<span class="w"> </span><span class="c1"># until routing catches up; SIGTERM follows once the hook returns.</span>
+<span class="w"> </span><span class="nt">preStop</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">sleep</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">seconds</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
+<span class="w"> </span><span class="nt">ports</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">containerPort</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">50055</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">flight</span>
+<span class="w"> </span><span class="nt">livenessProbe</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">httpGet</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/healthz</span>
+<span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">50055</span>
+<span class="w"> </span><span class="nt">failureThreshold</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">3</span>
+<span class="w"> </span><span class="nt">periodSeconds</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">10</span>
+<span class="w"> </span><span class="nt">readinessProbe</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">httpGet</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">path</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">/readyz</span>
+<span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">50055</span>
+<span class="w"> </span><span class="nt">failureThreshold</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">3</span>
+<span class="w"> </span><span class="nt">periodSeconds</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">5</span>
+<span class="nn">---</span>
+<span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">policy/v1</span>
+<span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">PodDisruptionBudget</span>
+<span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="c1"># Node drains evict at most one replica at a time.</span>
+<span class="w"> </span><span class="nt">minAvailable</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">1</span>
+<span class="w"> </span><span class="nt">selector</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">matchLabels</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+</pre></div>
+</div>
+<p>When a pod is replaced or moved to another node, Kubernetes removes it from the Service’s
+endpoints at the same time as it starts shutting the pod down. kube-proxy and any gateway pick up
+that removal asynchronously, so for a few seconds new connections can still be routed to the pod.
+The <code class="docutils literal notranslate"><span class="pre">preStop</span></code> sleep keeps the pod accepting connections during that window. On SIGTERM, which
+follows the sleep, it stops accepting connections and lets in-flight result streams finish, for up
+to <code class="docutils literal notranslate"><span class="pre">--graceful-shutdown-timeout-seconds</span></code> (10 by default), before exiting. The rolling-update
+strategy and the PodDisruptionBudget keep at least one replica serving throughout.</p>
+<p>Then add <code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code> to the scheduler container’s <code class="docutils literal notranslate"><span class="pre">args</span></code> in <code class="docutils literal notranslate"><span class="pre">cluster.yaml</span></code>. Which
+address to advertise depends on how clients reach the cluster.</p>
+<section id="through-port-forwarding">
+<h3>Through port forwarding<a class="headerlink" href="#through-port-forwarding" title="Link to this heading">#</a></h3>
+<p>For a client on your own machine, forward a local port to the result service as well as to the
+scheduler, and advertise the local address:</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">args</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">[</span><span class="s">"--bind-port=50050"</span><span class="p p-Indicator">,</span><span class="w"> </span><span class="s">"--advertise-flight-endpoint=127.0.0.1:50055"</span><span class="p p-Indicator">]</span>
+</pre></div>
+</div>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kubectl<span class="w"> </span>apply<span class="w"> </span>-f<span class="w"> </span>result-service.yaml
+kubectl<span class="w"> </span>apply<span class="w"> </span>-f<span class="w"> </span>cluster.yaml
+kubectl<span class="w"> </span>port-forward<span class="w"> </span>service/ballista-scheduler<span class="w"> </span><span class="m">50050</span>:50050<span class="w"> </span><span class="p">&</span>
+kubectl<span class="w"> </span>port-forward<span class="w"> </span>service/ballista-result-service<span class="w"> </span><span class="m">50055</span>:50055<span class="w"> </span><span class="p">&</span>
+</pre></div>
+</div>
+<p><code class="docutils literal notranslate"><span class="pre">kubectl</span> <span class="pre">port-forward</span></code> sends all traffic to a single pod, so use this for testing only.</p>
+</section>
+<section id="through-a-tls-terminating-gateway">
+<h3>Through a TLS-terminating gateway<a class="headerlink" href="#through-a-tls-terminating-gateway" title="Link to this heading">#</a></h3>
+<p>The result service doesn’t authenticate result fetches, and it connects to executors in
+plaintext. Make the gateway reachable only by trusted clients, for example through an internal
+load balancer or the gateway’s access restrictions, and limit what the service can reach as
+described in <a class="reference internal" href="#restricting-what-the-result-service-can-reach"><span class="xref myst">Restricting what the result service can reach</span></a>.</p>
+<p>In production, expose the result service through the <a class="reference external" href="https://gateway-api.sigs.k8s.io/">Gateway API</a> with a Gateway that terminates
+TLS and a <code class="docutils literal notranslate"><span class="pre">GRPCRoute</span></code> to the result service, and advertise the gateway’s address. TLS covers the
+connections from clients to the gateway. Behind it, the result service pods serve plaintext and
+connect to executors in plaintext.</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">args</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="s">"--bind-port=50050"</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="s">"--advertise-flight-endpoint=ballista-results.example.com:443"</span>
+</pre></div>
+</div>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gateway.networking.k8s.io/v1</span>
+<span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Gateway</span>
+<span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-results</span>
+<span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">gatewayClassName</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain"><your-gateway-class></span>
+<span class="w"> </span><span class="nt">listeners</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">grpc</span>
+<span class="w"> </span><span class="nt">protocol</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">HTTPS</span>
+<span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">443</span>
+<span class="w"> </span><span class="nt">hostname</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-results.example.com</span>
+<span class="w"> </span><span class="nt">tls</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">mode</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">Terminate</span>
+<span class="w"> </span><span class="nt">certificateRefs</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-results-tls</span>
+<span class="nn">---</span>
+<span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">gateway.networking.k8s.io/v1</span>
+<span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">GRPCRoute</span>
+<span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">parentRefs</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-results</span>
+<span class="w"> </span><span class="nt">hostnames</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-results.example.com</span>
+<span class="w"> </span><span class="nt">rules</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">backendRefs</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">50055</span>
+</pre></div>
+</div>
+<p>Result streams are long-lived, so make sure the gateway implementation’s idle and stream timeouts
+allow for the largest result. Clients fetch results over TLS by setting <code class="docutils literal notranslate"><span class="pre">ballista.client.use_tls</span></code>
+to <code class="docutils literal notranslate"><span class="pre">true</span></code>, with TLS roots for the gateway’s certificate supplied through a gRPC endpoint override.
+That takes a client built with one of tonic’s TLS features, as in the <a class="reference external" href="https://github.com/apache/datafusion-ballista/blob/main/examples/examples/mtls-cluster.rs">mTLS cluster example</a>.</p>
+</section>
+<section id="restricting-what-the-result-service-can-reach">
+<h3>Restricting what the result service can reach<a class="headerlink" href="#restricting-what-the-result-service-can-reach" title="Link to this heading">#</a></h3>
+<p>The result service checks no credentials, and it dials whichever executor address a fetch ticket
+names without checking that the address belongs to the cluster, so a forged ticket can make it
+connect to an arbitrary host. Keep it on networks you trust and, where your cluster enforces
+network policies, allow it to connect only to the executors’ Arrow Flight port:</p>
+<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">apiVersion</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">networking.k8s.io/v1</span>
+<span class="nt">kind</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">NetworkPolicy</span>
+<span class="nt">metadata</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service-egress</span>
+<span class="nt">spec</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">podSelector</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">matchLabels</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-result-service</span>
+<span class="w"> </span><span class="nt">policyTypes</span><span class="p">:</span><span class="w"> </span><span class="p p-Indicator">[</span><span class="s">"Egress"</span><span class="p p-Indicator">]</span>
+<span class="w"> </span><span class="nt">egress</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">to</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">podSelector</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">matchLabels</span><span class="p">:</span>
+<span class="w"> </span><span class="nt">app</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">ballista-executor</span>
+<span class="w"> </span><span class="nt">ports</span><span class="p">:</span>
+<span class="w"> </span><span class="p p-Indicator">-</span><span class="w"> </span><span class="nt">port</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">50051</span>
+<span class="w"> </span><span class="nt">protocol</span><span class="p">:</span><span class="w"> </span><span class="l l-Scalar l-Scalar-Plain">TCP</span>
+</pre></div>
+</div>
+<p>If executors register with host names rather than pod IPs (<code class="docutils literal notranslate"><span class="pre">--external-host</span></code>), also allow DNS.</p>
+</section>
+</section>
<section id="deleting-the-ballista-cluster">
<h2>Deleting the Ballista Cluster<a class="headerlink" href="#deleting-the-ballista-cluster" title="Link to this heading">#</a></h2>
<p>Run the following kubectl command to delete the cluster.</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kubectl<span class="w"> </span>delete<span class="w"> </span>-f<span class="w"> </span>cluster.yaml
</pre></div>
</div>
+<p>If you deployed a result service, delete it too:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>kubectl<span class="w"> </span>delete<span class="w"> </span>-f<span class="w"> </span>result-service.yaml
+</pre></div>
+</div>
</section>
<section id="autoscaling-executors">
<h2>Autoscaling Executors<a class="headerlink" href="#autoscaling-executors" title="Link to this heading">#</a></h2>
@@ -871,10 +1091,17 @@ scale the executors.</p>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#deploying-a-ballista-cluster">Deploying a Ballista Cluster</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#health-probes-and-rolling-upgrades">Health Probes and Rolling Upgrades</a><ul class="nav section-nav flex-column">
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#protocol-version-and-forced-upgrades">Protocol version and forced upgrades</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#stopping-executors">Stopping executors</a></li>
<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#rolling-both-deployments-together">Rolling both Deployments together</a></li>
</ul>
</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#port-forwarding">Port Forwarding</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#serving-results-to-clients-outside-the-cluster">Serving Results to Clients Outside the Cluster</a><ul class="nav section-nav flex-column">
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#through-port-forwarding">Through port forwarding</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#through-a-tls-terminating-gateway">Through a TLS-terminating gateway</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#restricting-what-the-result-service-can-reach">Restricting what the result service can reach</a></li>
+</ul>
+</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#deleting-the-ballista-cluster">Deleting the Ballista Cluster</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#autoscaling-executors">Autoscaling Executors</a></li>
</ul>
diff --git a/user-guide/deployment/quick-start.html b/user-guide/deployment/quick-start.html
index 4fbb7aec9..d0e183ae7 100644
--- a/user-guide/deployment/quick-start.html
+++ b/user-guide/deployment/quick-start.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/extending-components.html b/user-guide/extending-components.html
index 39cc45a8a..b8da29300 100644
--- a/user-guide/extending-components.html
+++ b/user-guide/extending-components.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/extensions-example.html b/user-guide/extensions-example.html
index a8a673b46..e0f85f8e1 100644
--- a/user-guide/extensions-example.html
+++ b/user-guide/extensions-example.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/faq.html b/user-guide/faq.html
index 8bbf27b41..1e0c75eaa 100644
--- a/user-guide/faq.html
+++ b/user-guide/faq.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/flightsql.html b/user-guide/flightsql.html
index 66d5adfc8..9df959663 100644
--- a/user-guide/flightsql.html
+++ b/user-guide/flightsql.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/history-server.html b/user-guide/history-server.html
index 327db1a0a..a361cf999 100644
--- a/user-guide/history-server.html
+++ b/user-guide/history-server.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/introduction.html b/user-guide/introduction.html
index ded265692..536d325c6 100644
--- a/user-guide/introduction.html
+++ b/user-guide/introduction.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/metrics.html b/user-guide/metrics.html
index 362d95c16..f49bbf070 100644
--- a/user-guide/metrics.html
+++ b/user-guide/metrics.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/python/cloud-storage.html b/user-guide/python/cloud-storage.html
index 6366c1938..497e25e45 100644
--- a/user-guide/python/cloud-storage.html
+++ b/user-guide/python/cloud-storage.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/python/index.html b/user-guide/python/index.html
index 73742a7e7..ac1f31cb0 100644
--- a/user-guide/python/index.html
+++ b/user-guide/python/index.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/python/jupyter.html b/user-guide/python/jupyter.html
index ebb891141..32e4f3df0 100644
--- a/user-guide/python/jupyter.html
+++ b/user-guide/python/jupyter.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/python/quickstart.html b/user-guide/python/quickstart.html
index eb3401a19..05badcdc3 100644
--- a/user-guide/python/quickstart.html
+++ b/user-guide/python/quickstart.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/rust.html b/user-guide/rust.html
index 9d0e3c566..b0fcddfdf 100644
--- a/user-guide/rust.html
+++ b/user-guide/rust.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
diff --git a/user-guide/scheduler.html b/user-guide/scheduler.html
index 4235e3890..1ff2613ee 100644
--- a/user-guide/scheduler.html
+++ b/user-guide/scheduler.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
@@ -424,15 +425,43 @@ from <code class="docutils literal notranslate"><span class="pre">--external-hos
<p>By default a client fetches the result partitions of a query directly from the executors that
produced them, over Arrow Flight. This keeps the scheduler out of the data path, but it requires
every client to have network access to every executor — which is not the case in isolated
-environments where only the scheduler is reachable.</p>
-<p>For those deployments the scheduler can advertise a different address for clients to fetch results
-from, and can optionally host an Arrow Flight proxy itself.</p>
+environments such as Kubernetes, where clients can usually reach only a few entry points.</p>
+<p>For those deployments, run a result service (<code class="docutils literal notranslate"><span class="pre">ballista-result-service</span></code>) that clients can reach, and
+have the scheduler advertise its address:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ballista-result-service<span class="w"> </span>--bind-port<span class="w"> </span><span class="m">50055</span>
+ballista-scheduler<span class="w"> </span>--advertise-flight-endpoint<span class="w"> </span>ballista-results.example.com:50055
+</pre></div>
+</div>
+<p>Clients then fetch each partition from the result service, which forwards the fetch to the executor
+that holds it, so the scheduler serves no result data. The result service holds no state: scale it
+by running more replicas behind a gRPC-aware load balancer and advertising the load balancer’s
+address. For Kubernetes manifests, see
+<a class="reference internal" href="deployment/kubernetes.html#serving-results-to-clients-outside-the-cluster"><span class="std std-ref">Serving Results to Clients Outside the Cluster</span></a>.</p>
+<section id="behind-a-tls-terminating-gateway">
+<h3>Behind a TLS-terminating gateway<a class="headerlink" href="#behind-a-tls-terminating-gateway" title="Link to this heading">#</a></h3>
+<p>To serve results through a gateway or load balancer that terminates TLS, advertise its address. TLS
+covers the connections from clients to the gateway. The result service replicas behind it serve
+plaintext and connect to executors in plaintext, and they don’t authenticate fetches, so make the
+gateway reachable only by trusted clients:</p>
+<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ballista-result-service<span class="w"> </span>--bind-port<span class="w"> </span><span class="m">50055</span>
+ballista-scheduler<span class="w"> </span>--advertise-flight-endpoint<span class="w"> </span>ballista-results.example.com:443
+</pre></div>
+</div>
+<p>Clients choose TLS for result fetches themselves, with <code class="docutils literal notranslate"><span class="pre">ballista.client.use_tls</span></code>. That takes a client
+built with one of tonic’s TLS features, and TLS roots for the gateway’s certificate supplied through
+a gRPC endpoint override, as the <a class="reference external" href="https://github.com/apache/datafusion-ballista/blob/main/examples/examples/mtls-cluster.rs">mTLS cluster example</a> does. <code class="docutils literal notranslate"><span class="pre">ballista.client.use_tls</span></code> applies to every result fetch a client
+makes, including direct fetches from executors. The gateway must forward gRPC (HTTP/2) to the
+replicas, for example with a Gateway API <code class="docutils literal notranslate"><span class="pre">GRPCRoute</span></code>, and allow long-lived streams.</p>
<blockquote>
-<div><p>Note: this is plain Arrow Flight, used to move result partitions. It is not Flight SQL, which was
-removed in Ballista 46.0.0 — the proxy only serves Ballista’s <code class="docutils literal notranslate"><span class="pre">FetchPartition</span></code> action, so generic
-Flight SQL or JDBC clients cannot use this endpoint.</p>
+<div><p>Note: the advertised endpoint serves plain Arrow Flight <code class="docutils literal notranslate"><span class="pre">DoGet</span></code> for Ballista’s own partition-fetch
+tickets. Generic Flight SQL, JDBC, and ADBC clients connect to the scheduler’s
+<a class="reference internal" href="flightsql.html"><span class="std std-doc">Flight SQL frontend</span></a> instead.</p>
</div></blockquote>
-<p>Two independent options control this:</p>
+<div class="admonition warning">
+<p class="admonition-title">Warning</p>
+<p>The result service checks no credentials, and it dials whichever executor address a fetch ticket
+names without checking that the address belongs to the cluster. Keep it on a trusted network.</p>
+</div>
<div class="pst-scrollable-table-container"><table class="table">
<thead>
<tr class="row-odd"><th class="head"><p>Option</p></th>
@@ -440,38 +469,30 @@ Flight SQL or JDBC clients cannot use this endpoint.</p>
</tr>
</thead>
<tbody>
-<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">--enable-embedded-flight-proxy</span></code></p></td>
-<td><p>Runs an Arrow Flight proxy inside the scheduler process, on the scheduler’s own host and port. The proxy forwards each fetch to the executor that owns the partition.</p></td>
+<tr class="row-even"><td><p><code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code></p></td>
+<td><p>The <code class="docutils literal notranslate"><span class="pre">HOST:PORT</span></code> address clients are told to fetch results from, instead of the executors: a result service, or a load balancer in front of one.</p></td>
</tr>
-<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code></p></td>
-<td><p>The <code class="docutils literal notranslate"><span class="pre">HOST:PORT</span></code> address clients are told to fetch results from, instead of the executors. Use it to point clients at a load balancer or a standalone proxy.</p></td>
+<tr class="row-odd"><td><p><code class="docutils literal notranslate"><span class="pre">--enable-embedded-flight-proxy</span></code></p></td>
+<td><p><strong>Deprecated, to be removed in 57.0.0.</strong> Runs an Arrow Flight proxy inside the scheduler process, on the scheduler’s own host and port, and points clients at the scheduler for results.</p></td>
</tr>
</tbody>
</table>
</div>
-<p>The first controls whether a proxy <em>runs</em>; the second controls what clients are <em>told</em>. Setting
-both is a supported combination: the embedded proxy runs, and clients are pointed at the advertised
-address, which is how you put a load balancer in front of one or more schedulers.</p>
-<p>To let clients fetch results through the scheduler itself:</p>
-<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ballista-scheduler<span class="w"> </span>--enable-embedded-flight-proxy
-</pre></div>
-</div>
-<p>To point clients at a load balancer that fronts the schedulers:</p>
-<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>ballista-scheduler<span class="w"> </span>--enable-embedded-flight-proxy<span class="w"> </span><span class="se">\</span>
-<span class="w"> </span>--advertise-flight-endpoint<span class="w"> </span>ballista-flight.example.com:50050
-</pre></div>
-</div>
-<p>Both options are disabled by default, and the embedded proxy should be enabled deliberately: it
-puts result traffic on the scheduler’s process and thread pool, competing with query planning and
-task scheduling. Under load this is a known source of scheduler congestion, so prefer a separate
-proxy — advertised with <code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code> — for clusters where result volume is
-significant.</p>
+</section>
+<section id="the-embedded-proxy-deprecated">
+<h3>The embedded proxy (deprecated)<a class="headerlink" href="#the-embedded-proxy-deprecated" title="Link to this heading">#</a></h3>
+<p>The scheduler can still proxy results itself with <code class="docutils literal notranslate"><span class="pre">--enable-embedded-flight-proxy</span></code>. This puts result
+traffic on the scheduler’s process and thread pool, competing with query planning and task
+scheduling, and it scales only with the scheduler. It is deprecated in 56.0.0 and will be removed in
+57.0.0; the scheduler logs a warning at startup while it is enabled. To migrate, deploy a result
+service and set <code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code> to its address.</p>
<blockquote>
<div><p><code class="docutils literal notranslate"><span class="pre">--advertise-flight-sql-endpoint</span></code> is accepted as a deprecated alias of
-<code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code>. Passing either flag with no value used to start the embedded proxy;
-that is deprecated too and logs a warning — use <code class="docutils literal notranslate"><span class="pre">--enable-embedded-flight-proxy</span></code> instead.</p>
+<code class="docutils literal notranslate"><span class="pre">--advertise-flight-endpoint</span></code>. Passing either flag with no value starts the deprecated embedded
+proxy, is itself deprecated, and logs a warning.</p>
</div></blockquote>
</section>
+</section>
<section id="rest-api">
<h2>REST API<a class="headerlink" href="#rest-api" title="Link to this heading">#</a></h2>
<p>The scheduler also provides a REST API that allows jobs to be monitored.</p>
@@ -645,7 +666,11 @@ scheduler with <code class="docutils literal notranslate"><span class="pre">--di
<nav id="pst-page-toc-nav" class="page-toc" aria-labelledby="pst-page-navigation-heading-2">
<ul class="pst-show_toc_level nav section-nav flex-column">
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#scheduler-identity">Scheduler Identity</a></li>
-<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#fetching-query-results">Fetching Query Results</a></li>
+<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#fetching-query-results">Fetching Query Results</a><ul class="nav section-nav flex-column">
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#behind-a-tls-terminating-gateway">Behind a TLS-terminating gateway</a></li>
+<li class="toc-h3 nav-item toc-entry"><a class="reference internal nav-link" href="#the-embedded-proxy-deprecated">The embedded proxy (deprecated)</a></li>
+</ul>
+</li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#rest-api">REST API</a></li>
<li class="toc-h2 nav-item toc-entry"><a class="reference internal nav-link" href="#web-tui-configuration">Web TUI Configuration</a></li>
</ul>
diff --git a/user-guide/tuning-guide.html b/user-guide/tuning-guide.html
index 53910869b..6f5fea98a 100644
--- a/user-guide/tuning-guide.html
+++ b/user-guide/tuning-guide.html
@@ -296,6 +296,7 @@
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Upgrade Guides</span></p>
<ul class="nav bd-sidenav">
<li class="toctree-l1 has-children"><a class="reference internal" href="../upgrading/index.html">Upgrade Guides</a><details><summary><span class="toctree-toggle" role="presentation"><i class="fa-solid fa-chevron-down"></i></span></summary><ul>
+<li class="toctree-l2"><a class="reference internal" href="../upgrading/56.0.0.html">Ballista 56.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/55.0.0.html">Ballista 55.0.0</a></li>
<li class="toctree-l2"><a class="reference internal" href="../upgrading/54.0.0.html">Ballista 54.0.0</a></li>
</ul>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@datafusion.apache.org
For additional commands, e-mail: commits-help@datafusion.apache.org