You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by lh...@apache.org on 2026/09/18 16:40:04 UTC
(pulsar-site) 02/04: [docs] Document REST API automation, the OpenAPI specifications and client generation
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
commit 80565101d14437f0d6b8519ca53640382af02e8e
Author: Lari Hotari <lh...@users.noreply.github.com>
AuthorDate: Fri Sep 18 18:16:12 2026 +0300
[docs] Document REST API automation, the OpenAPI specifications and client generation
reference-rest-api-overview (now "Pulsar REST APIs and OpenAPI
specifications"):
- "Automate with the REST API": every admin tool is a client of the REST
API; curl equivalents of pulsar-admin commands; the multipart exception
for functions, sources and sinks.
- The OpenAPI 3 documents published per release under /openapi/<version>/
(one per API with its base path), with the layout explained: the root
openapi.json is the /admin/v2 API (there is no v1), the v2/ and v3/
subdirectories are byte-identical copies grouped by base path, and the
v3 APIs are described only by their own documents. The example URL uses
the latest 5.x release token so it never regresses to a 4.x release.
- A generated download table (src/components/RestApiSpecTable.tsx) of master
plus the maintained release lines, reusing the release policy's support
windows: buildSupportedVersionList() and isMaintained() are extracted,
unchanged in behaviour, from SupportedVersionsTable.
- Client generation with OpenAPI Generator, and a "Related REST endpoints"
footer for the multipart function/connector creation and for producing
messages over HTTP (a data path that is neither part of the admin API nor
of the specifications).
admin-api-functions: "Create a function" gets worked examples that keep
the configuration in a file: a YAML file for pulsar-admin
--function-config-file and, in the REST API tab, a curl multipart request
with the functionConfig part typed as application/json, package upload,
package URL, and built-in functions via jar=builtin://...; the package URL
allow-list of functions_worker.yml (functionsDirectory /
enableReferencingFunctionsDirectoryFiles /
additionalEnabledFunctionsUrlPatterns and the connector counterparts,
enforced by PackageUrlValidator); the deprecated autoAck dropped from the
examples. "Update a function" refers to the create examples and explains
the differences: merge semantics, the settings an update cannot change,
the optional package, "Update contains no change", and updateAuthData;
its Java example used `new UpdateOptions()` on an interface and now uses
UpdateOptionsImpl.
client-libraries/rest: states that it covers producing messages and points
at the REST API and OpenAPI page for the admin API. The load-balance and
rolling-restart pages and the landing pages link to the REST API section.
Assisted-by: Claude Code (Opus 5)
---
client-libraries/rest.md | 2 +
docs/admin-api-functions.md | 179 ++++++++++++++++++++++++++----
docs/administration-load-balance.md | 2 +
docs/administration-rolling-restart.md | 2 +
docs/developers-landing.md | 2 +-
docs/pulsar-api-overview.md | 2 +-
docs/reference-landing.md | 2 +-
docs/reference-rest-api-overview.md | 71 +++++++++++-
src/components/RestApiSpecTable.tsx | 136 +++++++++++++++++++++++
src/components/SupportedVersionsTable.tsx | 28 +++--
10 files changed, 389 insertions(+), 37 deletions(-)
diff --git a/client-libraries/rest.md b/client-libraries/rest.md
index 62c7eeb3257..92b3cf920b4 100644
--- a/client-libraries/rest.md
+++ b/client-libraries/rest.md
@@ -7,6 +7,8 @@ description: Learn how to use Pulsar REST to interact with Pulsar without using
Pulsar not only provides REST endpoints to manage resources in Pulsar clusters, but also provides methods to query the state for those resources. In addition, Pulsar REST provides a simple way to interact with Pulsar **without using client libraries**, which is convenient for applications to use HTTP to interact with Pulsar.
+This page covers the endpoints for **producing messages** over HTTP. The endpoints for managing a Pulsar instance (clusters, tenants, namespaces, topics, schemas, functions, connectors and so on) are the admin REST API, described in [REST APIs and OpenAPI specifications](/docs/@pulsar:version_reference@/reference-rest-api-overview); since Pulsar 5.0 it comes with OpenAPI 3 specifications from which clients for other languages can be generated. The produce endpoints below are served by th [...]
+
## Connection
To connect to Pulsar, you need to specify a URL.
diff --git a/docs/admin-api-functions.md b/docs/admin-api-functions.md
index 124c27b20b8..88bdd0eab2d 100644
--- a/docs/admin-api-functions.md
+++ b/docs/admin-api-functions.md
@@ -26,7 +26,23 @@ Category|Method|If you want to manage functions...
You can perform the following operations on [functions](functions-overview.md/#what-are-pulsar-functions).
## Create a function
-You can create a Pulsar function in cluster mode (deploy it on a Pulsar cluster) using Admin CLI, REST API or Java Admin API.
+You can create a Pulsar function in cluster mode (deploy it on a Pulsar cluster) using the Admin CLI, the REST API or the Java admin API. Every interface takes the same two inputs:
+
+- the **configuration**: the fields of [`FunctionConfig`](@pulsar:javadoc:admin@/org/apache/pulsar/common/functions/FunctionConfig.html) (`tenant`, `namespace`, `name`, `className`, `inputs`, `output`, `parallelism`, `userConfig`, `resources`, ...). Every field is available in every interface under the same name: as command-line options or the keys of the YAML file for the CLI, the keys of the JSON `functionConfig` part for the REST API, and the setters of the `FunctionConfig` object in [...]
+- the **package**, in one of three forms:
+
+| Package | How to pass it | Notes |
+| --- | --- | --- |
+| A file | CLI: `--jar`, `--py` or `--go`; REST: the `data` file part; Java: the file name argument | Uploaded to the function worker. |
+| A URL | CLI: `jar: <url>` in the configuration file; REST: the `url` form field; Java: `createFunctionWithUrl` | Fetched by the function worker; the URL must be allowed, see below. |
+| A [built-in function](functions-deploy-cluster-builtin.md) | `jar: builtin://<function name>` in the configuration, no package | The worker uses the function from its `functionsDirectory`. |
+
+A package URL is fetched by the function worker and must be allowed by its configuration in `conf/functions_worker.yml`; a URL that is not allowed fails with `400 Function Package url is not valid`:
+
+- `file:///path/on/the/worker`: the path must lie inside the worker's `functionsDirectory`, with `enableReferencingFunctionsDirectoryFiles: true` (the default).
+- `http://...` or `https://...`: the URL must match one of the regular expressions in `additionalEnabledFunctionsUrlPatterns` (empty by default). A `file://` path outside the functions directory can be allowed the same way.
+- `function://tenant/namespace/name@version`: a package uploaded to [package management](admin-api-packages.md); requires `functionsWorkerEnablePackageManagement: true`.
+- Sources and sinks use `connectorsDirectory`, `enableReferencingConnectorDirectoryFiles` and `additionalEnabledConnectorUrlPatterns` instead.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -34,26 +50,93 @@ You can create a Pulsar function in cluster mode (deploy it on a Pulsar cluster)
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
-Use the [`create`](/reference/#/@pulsar:version_reference@/pulsar-admin/functions?id=create) subcommand.
-
-**Example**
+Use the [`create`](/reference/#/@pulsar:version_reference@/pulsar-admin/functions?id=create) subcommand. The configuration can be given as command-line options:
```shell
pulsar-admin functions create \
--tenant public \
--namespace default \
- --name (the name of Pulsar Functions) \
- --inputs test-input-topic \
- --output persistent://public/default/test-output-topic \
+ --name exclamation \
--classname org.apache.pulsar.functions.api.examples.ExclamationFunction \
+ --inputs persistent://public/default/test-input-topic \
+ --output persistent://public/default/test-output-topic \
+ --parallelism 1 \
+ --jar $PWD/examples/api-examples.jar
+```
+
+or kept in a YAML file passed with `--function-config-file`, which is easier to maintain: it can live in version control, and the same file serves [`update`](#update-a-function) later. Command-line options override the values in the file.
+
+```shell
+cat > exclamation.yaml <<EOF
+tenant: public
+namespace: default
+name: exclamation
+className: org.apache.pulsar.functions.api.examples.ExclamationFunction
+inputs:
+ - persistent://public/default/test-input-topic
+output: persistent://public/default/test-output-topic
+parallelism: 1
+EOF
+
+pulsar-admin functions create \
+ --function-config-file exclamation.yaml \
--jar $PWD/examples/api-examples.jar
```
+For a package URL or a built-in function, put it in the `jar` key of the file (`jar: https://...` or `jar: builtin://<function name>`) and drop `--jar`.
+
</TabItem>
<TabItem value="REST API">
[](swagger:/admin/v3/functions/registerFunction)
+The request is a `multipart/form-data` `POST` with the JSON configuration in the `functionConfig` part, which must be sent with the content type `application/json`, and the package as the `data` file part or the `url` form field (`-F "url=https://..."`). With `curl`:
+
+```shell
+cat > /tmp/functionconfig.json <<EOF
+{
+ "tenant": "public",
+ "namespace": "default",
+ "name": "exclamation",
+ "className": "org.apache.pulsar.functions.api.examples.ExclamationFunction",
+ "runtime": "JAVA",
+ "inputs": ["persistent://public/default/test-input-topic"],
+ "output": "persistent://public/default/test-output-topic",
+ "parallelism": 1
+}
+EOF
+
+curl -X POST \
+ -H "Authorization: Bearer $(cat token)" \
+ -F "functionConfig=@/tmp/functionconfig.json;type=application/json" \
+ -F "data=@$PWD/examples/api-examples.jar;type=application/octet-stream" \
+ http://localhost:8080/admin/v3/functions/public/default/exclamation
+```
+
+For a built-in function, send no package and set `jar` in the configuration:
+
+```shell
+cat > /tmp/functionconfig.json <<EOF
+{
+ "tenant": "public",
+ "namespace": "default",
+ "name": "myfunction",
+ "jar": "builtin://builtin-function-name",
+ "runtime": "JAVA",
+ "inputs": ["persistent://public/default/input-topic"],
+ "output": "persistent://public/default/output-topic",
+ "parallelism": 1
+}
+EOF
+
+curl -X POST \
+ -H "Authorization: Bearer $(cat token)" \
+ -F "functionConfig=@/tmp/functionconfig.json;type=application/json" \
+ http://localhost:8080/admin/v3/functions/public/default/myfunction
+```
+
+Send the request to the broker's web service port (8080) when the function worker [runs with the brokers](functions-worker-corun.md), or to the worker's own port (`workerPort`, 6750 by default) when it [runs separately](functions-worker-run-separately.md). Sources and sinks use the same shape with a `sourceConfig` or `sinkConfig` part.
+
</TabItem>
<TabItem value="Java">
@@ -68,7 +151,6 @@ functionConfig.setClassName("org.apache.pulsar.functions.api.examples.Exclamatio
functionConfig.setProcessingGuarantees(FunctionConfig.ProcessingGuarantees.ATLEAST_ONCE);
functionConfig.setTopicsPattern(sourceTopicPattern);
functionConfig.setSubName(subscriptionName);
-functionConfig.setAutoAck(true);
functionConfig.setOutput(sinkTopic);
admin.functions().createFunction(functionConfig, fileName);
```
@@ -80,7 +162,13 @@ admin.functions().createFunction(functionConfig, fileName);
## Update a function
-You can update a Pulsar function that has been deployed to a Pulsar cluster using Admin CLI, REST API or Java Admin API.
+You can update a function that is already deployed using the Admin CLI, the REST API or the Java admin API. An update takes the same configuration and package as [create](#create-a-function) and uses the same requests, so the examples above apply; what differs is how the function worker treats them:
+
+- The configuration is **merged** into the deployed one: settings you leave out keep their current values, and the tenant, namespace and name must match the deployed function. You can therefore send either the complete configuration the function was created with, or only its identity (tenant, namespace, name) and the settings to change.
+- Some settings **cannot be changed** by an update: the input topics, the subscription name, the processing guarantees, the ordering guarantees and the runtime. To change those, delete the function and create it again.
+- The **package is optional**: leave it out to keep the deployed code, or provide it to roll out a new build.
+- An update that changes neither the configuration nor the package is rejected with `400 Update contains no change`.
+- The update-only option `--update-auth-data` (`updateOptions.updateAuthData` in the REST and Java APIs) makes the worker replace the authentication data stored for the function, for example the token the function uses to connect to Pulsar, with the credentials of the caller.
````mdx-code-block
<Tabs groupId="api-choice"
@@ -88,17 +176,27 @@ You can update a Pulsar function that has been deployed to a Pulsar cluster usin
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
-Use the [`update`](/reference/#/@pulsar:version_reference@/pulsar-admin/functions?id=update) subcommand.
+Use the [`update`](/reference/#/@pulsar:version_reference@/pulsar-admin/functions?id=update) subcommand. Update merges into the deployed configuration, so you can pass the same configuration file as in the create example, a file with only `tenant`, `namespace`, `name` and the settings to change, or just command-line options; either way, pass `--jar` (or `--py`, `--go`) to also roll out a new implementation, and leave it out to keep the deployed one.
**Example**
```shell
+# roll out a new build of the function (with whatever the file contains, changed or not)
+pulsar-admin functions update \
+ --function-config-file exclamation.yaml \
+ --jar $PWD/examples/api-examples-2.jar
+
+# change only the configuration, for example after setting parallelism: 2 in the file;
+# the deployed implementation is kept
+pulsar-admin functions update \
+ --function-config-file exclamation.yaml
+
+# the same change with command-line options only: the function's identity and the settings to change
pulsar-admin functions update \
--tenant public \
--namespace default \
- --name (the name of Pulsar Functions) \
- --output persistent://public/default/update-output-topic \
- # other options
+ --name exclamation \
+ --parallelism 2
```
</TabItem>
@@ -106,22 +204,57 @@ pulsar-admin functions update \
[](swagger:/admin/v3/functions/updateFunction)
+The same `multipart/form-data` request as for create, sent as a `PUT`. Update merges into the deployed configuration, so the `functionConfig` part can be the complete configuration or only the function's identity and the settings to change; include the `data` part (or the `url` field) to also roll out a new implementation, leave it out to keep the deployed one, and add the optional `updateOptions` part to replace the stored authentication data:
+
+```shell
+# roll out a new build of the function (with whatever the file contains, changed or not)
+curl -X PUT \
+ -H "Authorization: Bearer $(cat token)" \
+ -F "functionConfig=@/tmp/functionconfig.json;type=application/json" \
+ -F "data=@$PWD/examples/api-examples-2.jar;type=application/octet-stream" \
+ http://localhost:8080/admin/v3/functions/public/default/exclamation
+
+# change only the configuration, for example after setting "parallelism": 2 in the file;
+# the deployed implementation is kept. Also replace the stored authentication data with the caller's
+curl -X PUT \
+ -H "Authorization: Bearer $(cat token)" \
+ -F "functionConfig=@/tmp/functionconfig.json;type=application/json" \
+ -F 'updateOptions={"updateAuthData":true};type=application/json' \
+ http://localhost:8080/admin/v3/functions/public/default/exclamation
+
+# the same change with a minimal configuration: the function's identity and the settings to change
+curl -X PUT \
+ -H "Authorization: Bearer $(cat token)" \
+ -F 'functionConfig={"tenant":"public","namespace":"default","name":"exclamation","parallelism":2};type=application/json' \
+ http://localhost:8080/admin/v3/functions/public/default/exclamation
+```
+
</TabItem>
<TabItem value="Java">
```java
-FunctionConfig functionConfig = new FunctionConfig();
-functionConfig.setTenant(tenant);
-functionConfig.setNamespace(namespace);
-functionConfig.setName(functionName);
-functionConfig.setRuntime(FunctionConfig.Runtime.JAVA);
-functionConfig.setParallelism(1);
-functionConfig.setClassName("org.apache.pulsar.functions.api.examples.ExclamationFunction");
-UpdateOptions updateOptions = new UpdateOptions();
-updateOptions.setUpdateAuthData(updateAuthData);
-admin.functions().updateFunction(functionConfig, userCodeFile, updateOptions);
+// Update merges into the deployed configuration, so the function's identity and the settings
+// to change are enough; the complete configuration from the create example works as well.
+FunctionConfig update = new FunctionConfig();
+update.setTenant(tenant);
+update.setNamespace(namespace);
+update.setName(functionName);
+
+// roll out a new build of the function, keeping the deployed configuration
+admin.functions().updateFunction(update, "/path/to/api-examples-2.jar", new UpdateOptionsImpl());
+
+// change only the configuration, here the parallelism; the deployed implementation is kept
+update.setParallelism(2);
+admin.functions().updateFunction(update, null, new UpdateOptionsImpl());
+
+// also replace the stored authentication data with the caller's credentials
+UpdateOptionsImpl updateOptions = new UpdateOptionsImpl();
+updateOptions.setUpdateAuthData(true);
+admin.functions().updateFunction(update, null, updateOptions);
```
+`UpdateOptionsImpl` (`org.apache.pulsar.common.functions`, in `pulsar-common`) implements the `UpdateOptions` interface that `updateFunction` takes.
+
</TabItem>
</Tabs>
diff --git a/docs/administration-load-balance.md b/docs/administration-load-balance.md
index 426aac8c801..2bbbc802b40 100644
--- a/docs/administration-load-balance.md
+++ b/docs/administration-load-balance.md
@@ -11,6 +11,8 @@ You can use multiple settings and tools to control the traffic distribution whic
The following sections introduce how load-balanced assignments work across Pulsar brokers and how you can leverage the framework to adjust. Pulsar ships two load managers: the **modular** load manager (`loadManagerClassName=org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl`, the default) and the **extensible** load manager (`org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl`). Unless a setting is marked otherwise, this page applies to both; the diff [...]
+The `pulsar-admin` commands on this page are thin wrappers around the admin [REST API](reference-rest-api-overview.md); every operation shown here can be called directly from your own automation, see [Automate with the REST API](reference-rest-api-overview.md#automate-with-the-rest-api).
+
## Dynamic assignments
Topics are dynamically assigned to brokers based on the load conditions of all brokers in the cluster. The assignment of topics to brokers is not done at the topic level but the **bundle** level (a higher level). Instead of individual topic assignments, each broker takes ownership of a subset of the topics for a namespace. This subset is called a bundle and effectively this subset is a sharding mechanism.
diff --git a/docs/administration-rolling-restart.md b/docs/administration-rolling-restart.md
index b0fde4556d0..7936b93a598 100644
--- a/docs/administration-rolling-restart.md
+++ b/docs/administration-rolling-restart.md
@@ -7,6 +7,8 @@ description: Learn what happens when a broker is restarted, how to drain brokers
A rolling restart of the brokers, whether for an [upgrade](administration-upgrade.md) or for a configuration change, is the largest reassignment of topics a cluster goes through. Every broker that stops hands all of its bundles to the others, and every broker that starts comes back empty. This page explains what happens at each step and how to keep the restart, and the rebalancing after it, under control. Bookies, the metadata store and proxies are covered by the [upgrade guide](administ [...]
+The `pulsar-admin` commands below are shown for readability; each of them is a call to the admin [REST API](reference-rest-api-overview.md#automate-with-the-rest-api), which is what an operator or a restart script would use.
+
## What happens when a broker stops
When a broker receives `SIGTERM`, or when you run `pulsar-admin brokers shutdown`, it goes through the following steps:
diff --git a/docs/developers-landing.md b/docs/developers-landing.md
index 01a862d5fa3..79baf0989d3 100644
--- a/docs/developers-landing.md
+++ b/docs/developers-landing.md
@@ -12,7 +12,7 @@ If you want to read top-viewed documents for developers, check out the following
- [Pulsar clients](/docs/client-libraries/)
- [Pulsar APIs](pulsar-api-overview.md)
- [Pulsar admin APIs](admin-api-overview.md)
- - [Pulsar REST APIs](reference-rest-api-overview.md)
+ - [Pulsar REST APIs and OpenAPI specifications](reference-rest-api-overview.md)
- [Pulsar contribution guide](/contribute)
## Advanced topics
diff --git a/docs/pulsar-api-overview.md b/docs/pulsar-api-overview.md
index 878be5eb2dc..d91ab0fda71 100644
--- a/docs/pulsar-api-overview.md
+++ b/docs/pulsar-api-overview.md
@@ -46,5 +46,5 @@ Audiences|Developers|DevOps
Goals|Build applications with Pulsar|Administer Pulsar clusters
Use cases|Pulsar client APIs help you create applications that rely on real-time data. <br/><br/> For example, you can build a financial application to handle fraud alerts or an eCommerce application that creates recommendations based on user activities.| Pulsar administration APIs let you administer the entire Pulsar instance, including clusters, tenants, namespaces, and topics, from a single endpoint. <br/><br/> For example, you can configure security and compliance, or get information [...]
Key features|- Process data with producers, consumers, readers, and TableView <br/><br/> - Secure data with authentication and authorization <br/><br/> - Protect data with transactions and schema <br/><br/> - Stabilize data with cluster-level auto failover | - Configure authentication and authorization <br/><br/> - Set data retention and resource isolation policies <br/><br/> - Facilitate workflow of application development<br/><br/> - Troubleshoot Pulsar
-Interfaces | - [Java client API](@pulsar:javadoc:client@/) <br/><br/> - [C++ client API](@pulsar:apidoc:cpp@) <br/><br/> - [Python client API](@pulsar:apidoc:python@) <br/><br/> - [Go client API](https://pkg.go.dev/github.com/apache/pulsar-client-go/pulsar) <br/><br/> - [Node.js client API](/docs/client-libraries/node) <br/><br/> - [WebSocket client API](/docs/client-libraries/websocket#api-reference) <br/><br/> - [C# client API](/docs/client-libraries/dotnet) | - [Java admin API](admin [...]
+Interfaces | - [Java client API](@pulsar:javadoc:client@/) <br/><br/> - [C++ client API](@pulsar:apidoc:cpp@) <br/><br/> - [Python client API](@pulsar:apidoc:python@) <br/><br/> - [Go client API](https://pkg.go.dev/github.com/apache/pulsar-client-go/pulsar) <br/><br/> - [Node.js client API](/docs/client-libraries/node) <br/><br/> - [WebSocket client API](/docs/client-libraries/websocket#api-reference) <br/><br/> - [C# client API](/docs/client-libraries/dotnet) | - [Java admin API](admin [...]
diff --git a/docs/reference-landing.md b/docs/reference-landing.md
index 95b31dec5b2..63a8aa7fb50 100644
--- a/docs/reference-landing.md
+++ b/docs/reference-landing.md
@@ -7,7 +7,7 @@ sidebar_label: "Reference"
This reference section provides an overview of Apache Pulsar's key concepts, terminology, and features. You'll find more detailed information in the resources listed below. Happy streaming!
- [Pulsar APIs](pulsar-api-overview.md)
- - [REST APIs](reference-rest-api-overview.md)
+ - [REST APIs and OpenAPI specifications](reference-rest-api-overview.md)
- [Terminology](reference-terminology.md)
- [Pulsar CLI tools](reference-cli-tools.md)
- [Pulsar configuration](reference-configuration.md)
diff --git a/docs/reference-rest-api-overview.md b/docs/reference-rest-api-overview.md
index c10b748e8d4..432b1425746 100644
--- a/docs/reference-rest-api-overview.md
+++ b/docs/reference-rest-api-overview.md
@@ -1,10 +1,14 @@
---
id: reference-rest-api-overview
-title: Pulsar REST APIs
-sidebar_label: "REST APIs"
-description: Get a comprehensive understanding of Pulsar REST APIs.
+title: Pulsar REST APIs and OpenAPI specifications
+sidebar_label: "REST API / OpenAPI clients"
+description: The Pulsar REST APIs, their OpenAPI specifications per release, and how to generate client bindings for other languages.
---
+````mdx-code-block
+import RestApiSpecTable from "@site/src/components/RestApiSpecTable";
+````
+
A REST API (also known as RESTful API, REpresentational State Transfer Application Programming Interface) is a set of definitions and protocols for building and integrating application software, using HTTP requests to GET, PUT, POST, and DELETE data following the REST standards. In essence, REST API is a set of remote calls using standard methods to request and return data in a specific format between two systems.
Pulsar provides a variety of REST APIs that enable you to interact with Pulsar to retrieve information or perform an action.
@@ -19,3 +23,64 @@ Pulsar provides a variety of REST APIs that enable you to interact with Pulsar t
| [Transactions](/transactions-rest-api/?version=@pulsar:rest_api_version@) | REST APIs for transaction-specific operations.|
| [Lookup](/lookup-rest-api/?version=@pulsar:rest_api_version@) | REST APIs for lookup-specific operations, such as getting the owner broker of a topic, getting the namespace bundle that a topic belongs to, and so on.|
+## Automate with the REST API
+
+The `pulsar-admin` CLI and the Java admin client are both clients of the REST API: everything they do is an HTTP request to a broker (or to a proxy in front of the brokers), so any tool that speaks HTTP can drive a Pulsar cluster without starting a JVM. This is the way to go for automation written in other languages, for operators and controllers, and for scripts that run inside a Kubernetes cluster, where a `curl` against the broker's web service port is cheaper than a `pulsar-admin` in [...]
+
+The [admin API tutorial](admin-api-tutorial.md) shows how to call the endpoints with `curl`, including authentication. Most endpoints take JSON or nothing at all; the ones that upload a package (creating or updating functions, sources and sinks) take a `multipart/form-data` request whose configuration part must carry the `application/json` content type, see [Create a function](admin-api-functions.md#create-a-function) for a worked `curl` example. For example, the following requests unloa [...]
+
+```shell
+# equivalent to: pulsar-admin namespaces unload my-tenant/my-namespace --bundle 0x00000000_0x08000000 --destinationBroker broker-2.example.com:8080
+curl -X PUT "http://broker.example.com:8080/admin/v2/namespaces/my-tenant/my-namespace/0x00000000_0x08000000/unload?destinationBroker=broker-2.example.com:8080"
+
+# equivalent to: pulsar-admin brokers update-dynamic-config --config loadBalancerSheddingEnabled --value false
+curl -X POST "http://broker.example.com:8080/admin/v2/brokers/configuration/loadBalancerSheddingEnabled/false"
+```
+
+### OpenAPI specifications
+
+Since Pulsar 5.0, the REST API is described by [OpenAPI 3](https://spec.openapis.org/oas/v3.0.1) documents that are generated from the broker source code for every release and published together with this site. Each Pulsar version has its own directory, `https://pulsar.apache.org/openapi/<version>/` (for the version this page documents: [`@pulsar:rest_api_spec_dir@`](pathname://@pulsar:rest_api_spec_dir@); the `master` directory follows the development branch), with one document per REST [...]
+
+| Document | REST API | Base path |
+| --- | --- | --- |
+| [`openapi.json`](pathname://@pulsar:rest_api_spec_dir@openapi.json) | Admin | `/admin/v2` |
+| [`openapilookup.json`](pathname://@pulsar:rest_api_spec_dir@openapilookup.json) | Lookup | `/lookup/v2` |
+| [`openapifunctions.json`](pathname://@pulsar:rest_api_spec_dir@openapifunctions.json) | Functions | `/admin/v3` |
+| [`openapisource.json`](pathname://@pulsar:rest_api_spec_dir@openapisource.json) | Sources | `/admin/v3` |
+| [`openapisink.json`](pathname://@pulsar:rest_api_spec_dir@openapisink.json) | Sinks | `/admin/v3` |
+| [`openapipackages.json`](pathname://@pulsar:rest_api_spec_dir@openapipackages.json) | Packages | `/admin/v3` |
+| [`openapitransactions.json`](pathname://@pulsar:rest_api_spec_dir@openapitransactions.json) | Transactions | `/admin/v3` |
+
+For example, the admin API of Pulsar @pulsar:version:latest-v5plus@ is [`https://pulsar.apache.org/openapi/@pulsar:version:latest-v5plus@/openapi.json`](https://pulsar.apache.org/openapi/@pulsar:version:latest-v5plus@/openapi.json). This document at the root of the version directory describes the current `/admin/v2` admin API; there is no separate v1 document, as the v1 admin API no longer exists. The functions, sources, sinks, packages and transactions APIs are described only by their o [...]
+
+The specifications are the same documents that render the REST API reference pages listed above, so you can also download the spec for a version from the reference page. Tools such as Postman or Insomnia can import them directly to explore and test the API.
+
+### Download the specifications
+
+The development branch and the newest release of every [maintained release line](https://pulsar.apache.org/contribute/release-policy/#supported-versions). Older releases keep their documents under the same directory layout.
+
+<RestApiSpecTable />
+
+### Generate a client from the specification
+
+With an OpenAPI 3 document, a client library for the admin API can be generated for most languages instead of being written by hand. [OpenAPI Generator](https://openapi-generator.tech/) supports [more than fifty client generators](https://openapi-generator.tech/docs/generators#client-generators), including Go, Python, TypeScript, Rust, C#, Kotlin and Java. For example, to generate a Python client for the admin API of the version this page documents:
+
+```shell
+openapi-generator-cli generate \
+ -i https://pulsar.apache.org@pulsar:rest_api_spec_dir@openapi.json \
+ -g python \
+ -o ./pulsar-admin-python
+```
+
+The generated client takes the broker's web service URL as its base path and lets you add the authentication header of your [authentication provider](security-overview.md), for example a bearer token. Regenerate it when you upgrade Pulsar, so that new endpoints and parameters become available; the REST API keeps backward compatibility across versions, so a client generated from an older specification keeps working against newer brokers.
+
+## Related REST endpoints
+
+### Create functions, sources and sinks
+
+Creating or updating a function, source or sink is the one part of the admin API that is not a plain JSON request: the endpoint takes a `multipart/form-data` request with the configuration as a JSON part (which must carry the `application/json` content type) and the package as a file part or a package URL, or `builtin://<name>` in the configuration for a built-in function or connector. See [Create a function](admin-api-functions.md#create-a-function) for a worked `curl` example and the d [...]
+
+### Produce messages over HTTP
+
+The brokers also accept messages over HTTP on the same web service port: `POST /topics/persistent/{tenant}/{namespace}/{topic}` (or `/partitions/{n}` for one partition of a partitioned topic) with a JSON body that carries the messages and, optionally, the schema to encode them with. This is a data-path endpoint for applications that cannot use a client library, not part of the admin API, and it is not included in the OpenAPI documents above. It is documented in [Pulsar REST](/docs/client [...]
+
diff --git a/src/components/RestApiSpecTable.tsx b/src/components/RestApiSpecTable.tsx
new file mode 100644
index 00000000000..0208762f11d
--- /dev/null
+++ b/src/components/RestApiSpecTable.tsx
@@ -0,0 +1,136 @@
+import React, { FC } from "react";
+import semver from "semver/preload";
+import useBaseUrl from "@docusaurus/useBaseUrl";
+
+import restApiVersions from "@site/static/swagger/restApiVersions.json";
+import { buildSupportedVersionList, isMaintained } from "./SupportedVersionsTable";
+
+// One entry of static/swagger/restApiVersions.json: the spec documents published for a
+// Pulsar version, grouped by REST API base path version (v2 or v3).
+type SpecGroup = { fileName: string[]; version: string };
+
+// Human-readable name of a spec document, keyed by the file name without its
+// "openapi" (5.0.0+) or "swagger" (pre-5.0) prefix.
+const DOCUMENT_LABELS: Record<string, string> = {
+ "": "Admin",
+ lookup: "Lookup",
+ functions: "Functions",
+ source: "Sources",
+ sink: "Sinks",
+ packages: "Packages",
+ transactions: "Transactions",
+};
+
+const DOCUMENT_ORDER = ["", "lookup", "functions", "source", "sink", "packages", "transactions"];
+
+type SpecRow = {
+ // "master" or the release line, e.g. "4.0 (LTS)" or "5.0 milestone"
+ line: string;
+ // the Pulsar version whose spec documents are linked, e.g. "4.0.13" or "5.0.0-M2"
+ version: string;
+ openApi3: boolean;
+ // document key (see DOCUMENT_LABELS) -> published file name without extension
+ documents: Map<string, string>;
+};
+
+// Pulsar 5.0.0+ (starting with 5.0.0-M1) and master publish OpenAPI 3 documents under
+// /openapi/; earlier releases keep their Swagger 2.0 documents under /swagger/. Mirrors
+// usesOpenApi3() in src/pages/RestApi/RestApi.tsx and src/config/pulsarVariables.ts.
+function usesOpenApi3(version: string): boolean {
+ if (version === "master") return true;
+ const v = semver.coerce(version);
+ return v !== null && v.major >= 5;
+}
+
+function toRow(line: string, version: string, groups: SpecGroup[]): SpecRow {
+ const documents = new Map<string, string>();
+ for (const group of groups) {
+ for (const fileName of group.fileName) {
+ documents.set(fileName.replace(/^(openapi|swagger)/, ""), fileName);
+ }
+ }
+ return { line, version, openApi3: usesOpenApi3(version), documents };
+}
+
+// The versions with published spec documents, newest first.
+function publishedSpecVersions(): string[] {
+ return Object.keys(restApiVersions as Record<string, SpecGroup[]>)
+ .filter((version) => semver.valid(version) !== null)
+ .sort(semver.rcompare);
+}
+
+// master plus one row per maintained release line (active or security support, or the
+// current milestone), each linking the newest release of the line that has published
+// spec documents. Unmaintained lines are left out; their documents remain available
+// under the same directory layout.
+function buildRows(): SpecRow[] {
+ const specs = restApiVersions as Record<string, SpecGroup[]>;
+ const published = publishedSpecVersions();
+ const rows: SpecRow[] = [];
+ if (specs["master"]) {
+ rows.push(toRow("master (development branch)", "master", specs["master"]));
+ }
+ for (const line of buildSupportedVersionList().filter((v) => isMaintained(v))) {
+ const lineName = `${line.version.major}.${line.version.minor}`;
+ const label = line.milestone
+ ? `${lineName} milestone`
+ : line.version.minor === 0 && line.version.major >= 3
+ ? `${lineName} (LTS)`
+ : lineName;
+ // Prefer the exact newest release of the line; a milestone line otherwise matches
+ // the newest published milestone of that major.minor.
+ const version =
+ published.find((v) => v === line.latestDisplay) ??
+ published.find((v) => {
+ const c = semver.coerce(v);
+ return c !== null && c.major === line.version.major && c.minor === line.version.minor
+ && (!line.milestone || /-M\d+/i.test(v));
+ });
+ if (version) {
+ rows.push(toRow(label, version, specs[version]));
+ }
+ }
+ return rows;
+}
+
+const RestApiSpecTable: FC = () => {
+ const rows = buildRows();
+ const baseUrl = useBaseUrl("/");
+
+ return (
+ <table>
+ <thead>
+ <tr>
+ <th>Release line</th>
+ <th>Newest release</th>
+ <th>Format</th>
+ <th>Documents</th>
+ </tr>
+ </thead>
+ <tbody>
+ {rows.map((row) => {
+ const dir = `${baseUrl}${row.openApi3 ? "openapi" : "swagger"}/${row.version}/`;
+ return (
+ <tr key={row.version}>
+ <td>{row.line}</td>
+ <td>
+ <a href={dir}>{row.version}</a>
+ </td>
+ <td>{row.openApi3 ? "OpenAPI 3" : "Swagger 2.0"}</td>
+ <td>
+ {DOCUMENT_ORDER.filter((key) => row.documents.has(key)).map((key, index) => (
+ <React.Fragment key={key}>
+ {index > 0 ? " ยท " : ""}
+ <a href={`${dir}${row.documents.get(key)}.json`}>{DOCUMENT_LABELS[key]}</a>
+ </React.Fragment>
+ ))}
+ </td>
+ </tr>
+ );
+ })}
+ </tbody>
+ </table>
+ );
+};
+
+export default RestApiSpecTable;
diff --git a/src/components/SupportedVersionsTable.tsx b/src/components/SupportedVersionsTable.tsx
index c8e27252b23..2ec482fa8e6 100644
--- a/src/components/SupportedVersionsTable.tsx
+++ b/src/components/SupportedVersionsTable.tsx
@@ -17,7 +17,7 @@ type SimpleReleaseData = {
releaseNoteLink: string,
};
-type SupportedVersionData = {
+export type SupportedVersionData = {
version: semver.SemVer,
milestone: boolean,
released: moment.Moment,
@@ -135,7 +135,10 @@ type SupportedVersionsTableProps = {
isHideUnmaintained?: boolean
};
-const SupportedVersionsTable: FC<SupportedVersionsTableProps> = (props) => {
+// One entry per feature release line (major.minor), newest first, with the support
+// windows derived from the release dates in data/release-pulsar. Shared with other
+// components that need to know which release lines are still maintained.
+export function buildSupportedVersionList(): SupportedVersionData[] {
let releaseList: SimpleReleaseData[] = releases.map(r => ({
version: semver.coerce(r.tagName),
displayVersion: r.tagName.replace(/^v/, ''),
@@ -145,7 +148,7 @@ const SupportedVersionsTable: FC<SupportedVersionsTableProps> = (props) => {
}))
releaseList.sort((o1, o2) => semver.rcompare(o1.version, o2.version))
- let supportedVersionList: SupportedVersionData[] = []
+ const supportedVersionList: SupportedVersionData[] = []
for (const release of releaseList) {
const version = release.version
const released = release.released
@@ -171,13 +174,22 @@ const SupportedVersionsTable: FC<SupportedVersionsTableProps> = (props) => {
latestReleaseNoteLink: release.releaseNoteLink,
})
}
+ return supportedVersionList
+}
+
+// A release line is maintained while it is in active or security support; a milestone
+// counts as maintained until the release it precedes ships.
+export function isMaintained(v: SupportedVersionData, now: Date = new Date()): boolean {
+ return v.milestone ||
+ (v.activeSupport && v.activeSupport.isAfter(now)) ||
+ (v.securitySupport && v.securitySupport.isAfter(now))
+}
+
+const SupportedVersionsTable: FC<SupportedVersionsTableProps> = (props) => {
+ let supportedVersionList = buildSupportedVersionList()
if (props.isHideUnmaintained) {
- supportedVersionList = supportedVersionList.filter(v =>
- v.milestone ||
- (v.activeSupport && v.activeSupport.isAfter(new Date())) ||
- (v.securitySupport && v.securitySupport.isAfter(new Date()))
- );
+ supportedVersionList = supportedVersionList.filter(v => isMaintained(v));
}
const TableHeaderCell = styled(TableCell)({fontWeight: "bold"})