You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "david-streamlio (via GitHub)" <gi...@apache.org> on 2026/02/09 21:14:25 UTC
[I] Add --state filter flag to pulsar-admin functions list [pulsar]
david-streamlio opened a new issue, #25235:
URL: https://github.com/apache/pulsar/issues/25235
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Motivation
IMHO, it's kind of annoying that pulsar-admin functions list doesn't distinguish between running functions and stopped ones. If you have a namespace with a mix of active and stopped functions, there's no way to quickly see which ones are actually running without checking each one individually.
This bash one-liner solves that, but it shouldn't be necessary:
```
for function in $(pulsar-admin functions list --namespace $NAMESPACE --tenant $TENANT); do
echo $function
pulsar-admin functions status --namespace $NAMESPACE --tenant $TENANT --name $function | fgrep running
done
```
This is slow and noisy — it makes N+1 API calls just to answer "what's currently running?" It would be much more useful if functions list supported filtering by runtime state directly.
### Solution
Add optional filter flags to pulsar-admin functions list:
--state flag
Filter functions by their running state.
```
# Show only running functions
pulsar-admin functions list --tenant public --namespace default --state RUNNING
# Show only stopped functions
pulsar-admin functions list --tenant public --namespace default --state STOPPED
```
Accepted values: RUNNING, STOPPED (and potentially FAILED, UNKNOWN to cover all states returned by functions status).
--long / -l flag (optional, nice-to-have)
Show extended output with status inline, similar to ls -l:
```
NAME STATE INSTANCES
add-offset-function RUNNING 1/1
topic-filter-key STOPPED 0/1
my-dedup-function RUNNING 3/3
```
### Alternatives
### Benefit
Eliminates the N+1 API call pattern shown above
Makes it practical to monitor function health from scripts and CI/CD pipelines
Consistent with other pulsar-admin subcommands that support filtering (e.g., topics list)
Reduces operational friction for namespaces with many functions
### Compatibility
This is a purely additive change. Without any flags, functions list behaves exactly as it does today.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "onceMisery (via GitHub)" <gi...@apache.org>.
onceMisery commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-4020507160
@david-streamlio Could you please help me review the code? Thanks! [25235](https://github.com/apache/pulsar/pull/25297)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "onceMisery (via GitHub)" <gi...@apache.org>.
onceMisery commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-3949226905
Could you assign it to me?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "onceMisery (via GitHub)" <gi...@apache.org>.
onceMisery commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-4066580198
Could you help review the PIP? [25299](https://github.com/apache/pulsar/pull/25299 )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "david-streamlio (via GitHub)" <gi...@apache.org>.
david-streamlio commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-3977351597
Thanks for the detailed implementation plan. I am ok with either approach. Looking forward to this contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat closed issue #25235: Add --state filter flag to pulsar-admin functions list
URL: https://github.com/apache/pulsar/issues/25235
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-6388675630
Thanks @david-streamlio and @onceMisery. This needs a new REST endpoint, admin API and CLI options, so it goes through a PIP. PIP-459 (https://github.com/apache/pulsar/pull/25299) was discussed on dev@ but never went to a vote, and it was closed during the open-PR cleanup. To move it forward, reopen #25299 and start a [VOTE] (or new [DISCUSS]) thread on dev@, as described in https://github.com/apache/pulsar/blob/master/pip/README.md. I'm closing this issue for now; the PIP will track the work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "onceMisery (via GitHub)" <gi...@apache.org>.
onceMisery commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-3976255362
Thank you for your reply. Here is my implementation plan. Which one do you prefer?
### Approach B: Server-Side Aggregation (New API Endpoint)
Add a new REST endpoint that returns function names with status summaries. The CLI makes a single API call for both
filtering and extended output.
#### Implementation Steps
**Step 1: Add DTO data model**
File: `pulsar-client-admin-api/.../policies/data/FunctionSummary.java` (new)
```java
@Data
public class FunctionSummary {
private String name;
private String state; // RUNNING, STOPPED, PARTIAL
private int numInstances;
private int numRunning;
}
```
**Step 2: Extend Admin API interface**
File: `pulsar-client-admin-api/.../Functions.java`
```java
/**
* Get the list of functions with status summary under a namespace.
*/
List<FunctionSummary> getFunctionsWithStatus(String tenant, String namespace)
throws PulsarAdminException;
CompletableFuture<List<FunctionSummary>> getFunctionsWithStatusAsync(
String tenant, String namespace);
```
**Step 3: Implement Admin API client**
File: `pulsar-client-admin/.../FunctionsImpl.java`
```java
@Override
public List<FunctionSummary> getFunctionsWithStatus(String tenant, String namespace)
throws PulsarAdminException {
return sync(() -> getFunctionsWithStatusAsync(tenant, namespace));
}
@Override
public CompletableFuture<List<FunctionSummary>> getFunctionsWithStatusAsync(
String tenant, String namespace) {
WebTarget path = functions.path(tenant).path(namespace).path("status-summary");
return asyncGetRequest(path, new GenericType<List<FunctionSummary>>() {
});
}
```
New REST path: `GET /admin/v3/functions/{tenant}/{namespace}/status-summary`
**Step 4: Add REST endpoint**
File: `pulsar-functions/worker/.../v3/FunctionsApiV3Resource.java`
```java
@GET
@ApiOperation(
value = "Displays the list of Pulsar Functions with status summary",
response = FunctionSummary.class,
responseContainer = "List"
)
@Produces(MediaType.APPLICATION_JSON)
@Path("/{tenant}/{namespace}/status-summary")
public List<FunctionSummary> listFunctionsWithStatus(
final @PathParam("tenant") String tenant,
final @PathParam("namespace") String namespace) {
return functions().listFunctionsWithStatus(
tenant, namespace, uri.getRequestUri(), authParams());
}
```
**Step 5: Implement Worker service logic**
File: `pulsar-functions/worker/.../ComponentImpl.java`
```java
public List<FunctionSummary> listFunctionsWithStatus(
final String tenant, final String namespace,
final URI uri, final AuthenticationParameters authParams) {
// impl listFunctionsWithStatus
}
```
> Note: `getComponentStatus()` is an existing method in `ComponentImpl` that handles inter-worker RPCs internally. By
> calling it server-side, we avoid N external HTTP round-trips.
**Step 6: Modify CLI command**
File: `pulsar-client-tools/.../CmdFunctions.java` — `ListFunctions` class
```java
@Command(description = "List all Pulsar Functions running under a specific tenant and namespace")
class ListFunctions extends NamespaceCommand {
@Option(names = "--state", description = "Filter by state: RUNNING, STOPPED, PARTIAL")
private String state;
@Option(names = {"-l", "--long"}, description = "Show extended output with status")
private boolean longFormat;
@Override
void runCmd() throws Exception {
List<String> functions = getAdmin().functions().getFunctions(tenant, namespace);
if (state == null && !longFormat) {
print(functions); // Original behavior unchanged
return;
}
for (String fn : functions) {
FunctionStatus status = getAdmin().functions()
.getFunctionStatus(tenant, namespace, fn);
String fnState = deriveState(status);
if (state != null && !state.equalsIgnoreCase(fnState)) {
continue;
}
if (longFormat) {
System.out.printf("%-30s %-10s %d/%d%n",
fn, fnState, status.numRunning, status.numInstances);
} else {
print(fn);
}
}
}
}
```
---
### Approach C: Hybrid (Server Batch Retrieval + Client-Side Filtering) — Recommended
Server provides a single batch API that returns all functions with their status summaries. The CLI consumes this API and
handles all filtering (`--state`) and formatting (`--long`) logic on the client side.
**Key difference from Approach B:** Approach B frames the server as the "aggregation + filtering" layer. Approach C
explicitly separates concerns — the server is a **data provider** only (batch status retrieval), while the client owns *
*all presentation logic** (filtering, sorting, formatting). This keeps the server endpoint generic and reusable.
#### Architecture
```
┌─────────────────────────────────────────────────────┐
│ CLI (Client) │
│ ┌───────────┐ ┌──────────┐ ┌───────────────┐ │
│ │ --state │──▶│ Filter │──▶│ Print (plain │ │
│ │ --long │ │ by state │ │ or table fmt) │ │
│ └───────────┘ └──────────┘ └───────────────┘ │
│ ▲ │
│ │ Single API call │
└────────┼────────────────────────────────────────────┘
│
┌────────┼────────────────────────────────────────────┐
│ Server (Worker) │
│ ┌─────────────────────────────────────────────┐ │
│ │ GET /status-summary │ │
│ │ → list metadata → batch getComponentStatus │ │
│ │ → return List<FunctionSummary> (ALL funcs) │ │
│ └─────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────┘
```
#### Implementation Steps
**Step 1: Add DTO data model** (same as Approach B)
**Step 2: Extend Admin API interface** (same as Approach B)
**Step 3: Implement Admin API client** (same as Approach B)
**Step 4: Add REST endpoint** (same as Approach B)
**Step 5: Implement Worker service logic** (same as Approach B)
**Step 6: Modify CLI command — filtering and formatting on client side**
File: `pulsar-client-tools/.../CmdFunctions.java` — `ListFunctions` class
This is where Approach C differs from B: **all filtering and formatting logic lives in the CLI**, not on the server.
```java
@Command(description = "List all Pulsar Functions running under a specific tenant and namespace")
class ListFunctions extends NamespaceCommand {
@Option(names = "--state",
description = "Filter by state: RUNNING, STOPPED, PARTIAL")
private String state;
@Option(names = {"-l", "--long"},
description = "Show extended output with status")
private boolean longFormat;
@Override
void runCmd() throws Exception {
// No flags → original behavior, original API
if (state == null && !longFormat) {
print(getAdmin().functions().getFunctions(tenant, namespace));
return;
}
// Single batch call to get all summaries
List<FunctionSummary> summaries = getAdmin().functions()
.getFunctionsWithStatus(tenant, namespace);
// Client-side filtering
if (state != null) {
summaries = summaries.stream()
.filter(s -> state.equalsIgnoreCase(s.getState()))
.collect(Collectors.toList());
}
// Client-side formatting
if (longFormat) {
System.out.printf("%-30s %-10s %s%n",
"NAME", "STATE", "INSTANCES");
for (FunctionSummary s : summaries) {
System.out.printf("%-30s %-10s %d/%d%n",
s.getName(), s.getState(),
s.getNumRunning(), s.getNumInstances());
}
} else {
summaries.forEach(s -> print(s.getName()));
}
}
}
```
## 5. Side-by-Side Comparison
| Dimension | B: Server-Side | C: Hybrid (Recommended) |
|-------------------------------|------------------------|-------------------------|
| New API endpoints | 1 | 1 |
| Files changed | 7 | 7 |
| Network calls (client→server) | 1 | 1 |
| Status aggregation | Worker | Worker |
| Filtering location | Server | **CLI** |
| Formatting location | Server | **CLI** |
| Non-CLI clients benefit | Yes | Yes |
| Server endpoint reusability | Coupled to CLI needs | **Generic** |
| Performance (100 functions) | 1 HTTP + internal RPCs | 1 HTTP + internal RPCs |
| Backward compatibility | Full | Full |
| Complexity | Medium | Medium |
## 6. Recommendation
**Approach C (Hybrid) is recommended** for the following reasons:
1. **Clean separation of concerns** — Server is a pure data provider; CLI owns all presentation logic. This makes the
API reusable for other consumers with different filtering/formatting needs.
2. **Performance** — Server-side `getComponentStatus()` uses internal worker RPCs, far faster than N external HTTP calls
from the client.
3. **Extensibility** — Adding new CLI filters (e.g., `--name-pattern`) requires zero server changes.
4. **Precedent** — Consistent with Pulsar's existing API patterns (dedicated endpoints for enriched data).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "onceMisery (via GitHub)" <gi...@apache.org>.
onceMisery commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-3956653924
@david-streamlio Hello！I have some implementation details that I need your help to confirm.
### Server-side Filtering
Add a state query parameter to the Worker REST endpoint. The server performs filtering and returns the result. This involves a single API call and delivers optimal performance, but requires changes to the API interface, Worker implementation, and REST endpoint.
### Client-side Filtering
The CLI layer first retrieves the function list, then queries the status one by one for filtering. No server-side changes are needed, but it results in N+1 API calls, which is slow under large namespaces.
### Hybrid Approach
Add a new server-side API for batch status retrieval (e.g., listFunctionsWithStatus). The CLI fetches all functions and their statuses in one call, then performs filtering and formatting on the client side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add --state filter flag to pulsar-admin functions list [pulsar]
Posted by "onceMisery (via GitHub)" <gi...@apache.org>.
onceMisery commented on issue #25235:
URL: https://github.com/apache/pulsar/issues/25235#issuecomment-3970207022
> [@david-streamlio](https://github.com/david-streamlio) Hello！I have some implementation details that I need your help to confirm.
@david-streamlio Please help me see which implementation method is more reliable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org