You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "merlimat (via GitHub)" <gi...@apache.org> on 2026/10/16 17:45:52 UTC
[PR] [fix][client] Surface V5 client failures as the typed exception of their error category [pulsar]
merlimat opened a new pull request, #26950:
URL: https://github.com/apache/pulsar/pull/26950
### Motivation
The V5 client hands almost every failure to the application as a plain `org.apache.pulsar.client.api.v5.PulsarClientException`, so an application can't branch on its cause: an authorization failure, a busy producer, an incompatible schema, a send timeout and a transaction conflict all look the same. The scalable-topics error model (`spec/scalable-topics/error-model.md` §1, §2 and §4) requires a typed error category for every fallible operation, and `java-mapping.md` §6 names the V5 exception of each. Async operations are worse off: their futures complete with the raw v4 exception, which isn't a V5 type at all, though `ProducerBuilder.createAsync()` documents that it completes with a `PulsarClientException`.
The type is lost in two places:
- The v4 layer builds untyped exceptions for three scalable-topic errors: `ClientCnx` for a `ScalableTopicSubscribe` error, `ScalableTopicsWatcher` for the namespace watch, and `DagWatchClient` for the topic lookup, which only typed `TopicNotFound`.
- The V5 layer wraps each v4 exception in a plain V5 one, or hands it over as is:
- the producer, stream, queue and checkpoint consumer builders: `create()` and `subscribe()` wrap it (only the producer kept `NotFound`), `createAsync()` and `subscribeAsync()` hand it over;
- the per-segment producer creation and the blocking send wrap it, the async send hands it over;
- the client build, the authentication plugin loading, `newTransaction()` and the client close wrap it;
- transaction commit and abort wrap it, their async variants hand it over;
- the blocking consumer close wraps it, even a V5 failure, and the async close hands it over.
### Example
A role with only `consume` permission on the namespace creates a producer on a scalable topic (the lookup is allowed) and sends:
```java
producer.newMessage().value("x").send();
// org.apache.pulsar.client.api.v5.PulsarClientException: {"errorMsg":"Client is not authorized to Produce", ...}
```
`ScalableTopicProducer.getOrCreateSegmentProducer` wraps the v4 `AuthorizationException` of the segment producer:
```java
if (cause instanceof org.apache.pulsar.client.api.PulsarClientException v4Exc) {
throw new PulsarClientException(v4Exc.getMessage(), v4Exc);
}
```
In the same way:
- a role without permission gets `PulsarClientException: org.apache.pulsar.client.api.PulsarClientException$AuthorizationException: ... not authorized to Lookup` from `create()` or `subscribe()`, and a produce-only role gets `Scalable topic subscribe failed: AuthorizationError - ...` from a stream consumer's `subscribe()`;
- a second exclusive producer fails with a plain exception: `V5ProducerAccessModeTest` matches "Busy" or "Fenced" in its message;
- `commit()` of an aborted transaction throws a plain exception caused by `InvalidTxnStatusException`, and `async().commit()` completes with the `InvalidTxnStatusException` itself.
### Modifications
- `V5Utils.toV5Exception(Throwable)` converts a failure to the exception the application gets. A table maps each v4 exception to the V5 type of its error category, with the v4 exception as the cause. Any other failure becomes a plain V5 `PulsarClientException` caused by it, and a V5 exception is returned as is.
| Category | v4 exceptions | V5 exception |
|---|---|---|
| NotFound | `NotFoundException`, `TopicDoesNotExistException` | `NotFoundException` |
| Authentication | `AuthenticationException`, `GettingAuthenticationDataException`, `UnsupportedAuthenticationException` | the V5 types of the same names |
| Authorization | `AuthorizationException` | `AuthorizationException` |
| Timeout | `TimeoutException` | `TimeoutException` |
| AlreadyClosed | `AlreadyClosedException` | `AlreadyClosedException` |
| ProducerBusy / Fenced | `ProducerBusyException`, `ProducerFencedException` | `ProducerBusyException` |
| ConsumerBusy | `ConsumerBusyException`, `ConsumerAssignException` | `ConsumerBusyException` |
| IncompatibleSchema | `IncompatibleSchemaException` | `IncompatibleSchemaException` |
| TopicTerminated | `TopicTerminatedException` | `TopicTerminatedException` |
| Connect / NotConnected | `ConnectException`, `NotConnectedException` | `ConnectException`, `NotConnectedException` |
| ProducerQueueFull | `ProducerQueueIsFullError` | `ProducerQueueIsFullException` |
| MemoryBufferFull | `MemoryBufferIsFullError` | `MemoryBufferIsFullException` |
| TransactionConflict | `TransactionConflictException`, `TransactionHasOperationFailedException`, `InvalidTxnStatusException`, `TransactionNotFoundException` | `TransactionConflictException` |
| Crypto | `CryptoException` | `CryptoException` |
| InvalidConfiguration | `InvalidConfigurationException` | `InvalidConfigurationException` |
| InvalidServiceURL | `InvalidServiceURL` | `InvalidServiceURLException` |
- `ConsumerAssignException`, overlapping Key_Shared hash ranges, is a subscription constraint that keeps the consumer out, which is how the error model defines ConsumerBusy.
- A transaction with a failed operation, no longer open, or no longer known to its coordinator (timed out) can't be committed or aborted, and `client-api.md` requires a typed error for that.
- The v4 exceptions that no category covers (lookup, not allowed, not supported, quota, broker persistence or metadata, ...) stay plain V5 exceptions.
- Every place listed above uses it: the builders' futures with `.exceptionallyCompose(V5Utils::failWithV5Exception)`, the blocking calls with `throw V5Utils.toV5Exception(e)`. The async send converts its failure in `handleAsyncSegmentFailure`, so the future handed to the caller is still the one `flush()` awaits.
- The three scalable-topic errors are built with `ClientCnx.getPulsarClientException(error, message)`, as the other v4 responses are. The lookup keeps `NotFoundException` for `TopicNotFound`.
- The producer still decides whether to retry a send from the raw v4 exception, so `TopicTerminated` and `AlreadyClosed` reach the application only once the retries for a gone segment are exhausted.
- Consumers now get `NotFoundException` for a topic that doesn't exist and can't be auto-created, as the producer did. The message of a failure that stays plain no longer starts with the v4 exception's class name.
### Verifying this change
- `V5AuthErrorTest` (new) runs a broker with token authentication and `PulsarAuthorizationProvider`, with roles that have no permission, produce only, or consume only on the namespace, and an invalid token. Producer `create()`, `createAsync()` and exclusive `create()`, blocking and async `send()`, the stream, queue and checkpoint (with and without a consumer group) consumers' `subscribe()`/`create()`, and the namespace stream and queue consumers' `subscribe()` fail with `AuthorizationException`; the invalid token fails with `AuthenticationException`. The produce-only and consume-only roles still do what they are permitted to. All 5 tests fail on master.
- `V5ProducerAccessModeTest` and `V5SchemaEvolutionTest` assert `ProducerBusyException` and `IncompatibleSchemaException` instead of matching messages. `V5ScalableTopicAutoCreateTest` asserts `NotFoundException` for the consumers too. `V5TransactionTest` asserts `TransactionConflictException` from the commit of an aborted transaction, blocking and async, and `InvalidConfigurationException` from `newTransaction()` without a transaction policy. `PulsarClientBuilderV5Test` asserts `InvalidServiceURLException` for a malformed service URL and `UnsupportedAuthenticationException` for a missing authentication plugin. These all fail on master.
- `V5UtilsTest` covers each row of the table, and the plain and V5 cases. `DagWatchClientTest` covers the lookup errors, which `V5AuthErrorTest` can't reach: for a role without permission, the v4 connection lookup that precedes the scalable-topic lookup rejects it first.
- The pulsar-client-v5 tests (208), the pulsar-broker `org.apache.pulsar.client.api.v5` and `org.apache.pulsar.client.impl.v5` tests (78 classes, 279 tests), `ProxyScalableTopicsTest` and `quickCheck` pass.
### Does this pull request potentially affect one of the following parts:
*If the box was checked, please highlight the changes*
- [ ] Dependencies (add or upgrade a dependency)
- [ ] The public API
- [ ] The schema
- [ ] The default values of configurations
- [ ] The threading model
- [ ] The binary protocol
- [ ] The REST endpoints
- [ ] The admin CLI options
- [ ] The metrics
- [ ] Anything that affects deployment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][client] Surface V5 client failures as the typed exception of their error category [pulsar]
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui commented on code in PR #26950:
URL: https://github.com/apache/pulsar/pull/26950#discussion_r4298486284
##########
pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/ScalableTopicProducer.java:
##########
@@ -440,9 +440,9 @@ MessageIdV5 sendInternal(
} catch (org.apache.pulsar.client.api.PulsarClientException e) {
// Thrown by the v4 producer's send().
if (!isSegmentGoneError(e)) {
- throw new PulsarClientException(e.getMessage(), e);
+ throw V5Utils.toV5Exception(e);
Review Comment:
[P2] Preserve the transaction error before the blocking-send unwrap
For a send using an already-aborted transaction, `TransactionImpl.checkIfOpen()` completes the v4 send future with `InvalidTxnStatusException`. However, `sendV4()` first calls the v4 `PulsarClientException.unwrap(e)`, whose fallback wraps this coordinator exception in a plain v4 `PulsarClientException`. By the time this converter runs, it can only produce a generic V5 exception. The async send path receives the original exception and correctly maps it to `TransactionConflictException`, so callers still cannot handle the same failure consistently across blocking and async sends.
Could we translate the original completion failure before the lossy v4 unwrap, retaining its cause for segment-retry classification, and add a blocking/async send-after-abort regression? This is an incomplete fix in this path; the scenario was traced in source, not runtime-reproduced.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][client] Surface V5 client failures as the typed exception of their error category [pulsar]
Posted by "codelipenghui (via GitHub)" <gi...@apache.org>.
codelipenghui merged PR #26950:
URL: https://github.com/apache/pulsar/pull/26950
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][client] Surface V5 client failures as the typed exception of their error category [pulsar]
Posted by "merlimat (via GitHub)" <gi...@apache.org>.
merlimat commented on code in PR #26950:
URL: https://github.com/apache/pulsar/pull/26950#discussion_r4298855087
##########
pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/V5Utils.java:
##########
@@ -58,4 +123,36 @@ static TopicName parseScalableTopicInput(String topic) {
}
return tn;
}
+
+ /**
+ * The V5 exception to hand the application for the failure of an operation that the V5 client runs on the
+ * v4 client, so that it can branch on the error category (error-model.md §2): the V5 type of its category,
+ * or the plain {@link PulsarClientException} for a failure outside them, either one caused by the original
+ * failure. A V5 exception is returned as is. A {@code CompletionException} or {@code ExecutionException} is
+ * unwrapped first.
+ */
+ static PulsarClientException toV5Exception(Throwable failure) {
+ Throwable cause = FutureUtil.unwrapCompletionException(failure);
+ if (cause instanceof PulsarClientException v5Exception) {
+ return v5Exception;
+ }
+ for (V5Type type : V5_TYPES) {
+ if (type.v4Type().isInstance(cause)) {
Review Comment:
Done in 32ce958f4b: `V5ExceptionMappings` looks the class of the failure up in a map first, and otherwise takes the closest mapped supertype from the table sorted by class depth (each type before its supertypes), so the order of the table no longer matters. `V5ExceptionMappingsTest` covers an unlisted subtype.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
##########
pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/V5Utils.java:
##########
@@ -18,14 +18,79 @@
*/
package org.apache.pulsar.client.impl.v5;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.function.Function;
+import org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException;
+import org.apache.pulsar.client.api.v5.PulsarClientException;
import org.apache.pulsar.common.naming.TopicDomain;
import org.apache.pulsar.common.naming.TopicName;
+import org.apache.pulsar.common.util.FutureUtil;
/**
* Internal utilities for the V5 client implementation.
*/
final class V5Utils {
Review Comment:
Done in 32ce958f4b: the table, `toV5Exception` and `failWithV5Exception` moved to the new `V5ExceptionMappings`, and `V5Utils` is back to what it is on master.
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
##########
pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/ScalableTopicProducer.java:
##########
@@ -440,9 +440,9 @@ MessageIdV5 sendInternal(
} catch (org.apache.pulsar.client.api.PulsarClientException e) {
// Thrown by the v4 producer's send().
if (!isSegmentGoneError(e)) {
- throw new PulsarClientException(e.getMessage(), e);
+ throw V5Utils.toV5Exception(e);
Review Comment:
Fixed in 32ce958f4b: `sendV4` converts the send's own failure with `V5ExceptionMappings.toV5Exception` instead of the v4 `unwrap`, so a blocking send in an aborted transaction now fails with `TransactionConflictException`, like the async one. The segment-gone retry still finds the v4 cause. The new `V5TransactionTest.testSendInAbortedTransactionFailsWithTransactionConflict` covers both, and fails without the fix (plain `PulsarClientException` caused by `InvalidTxnStatusException`).
_🤖 Addressed by [Claude Code](https://claude.com/claude-code)_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][client] Surface V5 client failures as the typed exception of their error category [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #26950:
URL: https://github.com/apache/pulsar/pull/26950#discussion_r4298457955
##########
pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/V5Utils.java:
##########
@@ -58,4 +123,36 @@ static TopicName parseScalableTopicInput(String topic) {
}
return tn;
}
+
+ /**
+ * The V5 exception to hand the application for the failure of an operation that the V5 client runs on the
+ * v4 client, so that it can branch on the error category (error-model.md §2): the V5 type of its category,
+ * or the plain {@link PulsarClientException} for a failure outside them, either one caused by the original
+ * failure. A V5 exception is returned as is. A {@code CompletionException} or {@code ExecutionException} is
+ * unwrapped first.
+ */
+ static PulsarClientException toV5Exception(Throwable failure) {
+ Throwable cause = FutureUtil.unwrapCompletionException(failure);
+ if (cause instanceof PulsarClientException v5Exception) {
+ return v5Exception;
+ }
+ for (V5Type type : V5_TYPES) {
+ if (type.v4Type().isInstance(cause)) {
Review Comment:
would it be useful to have a lookup map to first check if there's an exact match from a Class instance and only after that iterate all entries?
Another problem with the isInstance check could be that the ordering matters if the base type is before the sub types in the mapping. Perhaps the mappings could be ordered programmatically at instantiation type to avoid mistakes? "deepest" subtypes would come first in order. Iterating the list could be useful in cases where there's no direct mapping so that lookup map + list iteration could be complementory.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [fix][client] Surface V5 client failures as the typed exception of their error category [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari commented on code in PR #26950:
URL: https://github.com/apache/pulsar/pull/26950#discussion_r4298472135
##########
pulsar-client-v5/src/main/java/org/apache/pulsar/client/impl/v5/V5Utils.java:
##########
@@ -18,14 +18,79 @@
*/
package org.apache.pulsar.client.impl.v5;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+import java.util.function.Function;
+import org.apache.pulsar.client.api.transaction.TransactionCoordinatorClientException;
+import org.apache.pulsar.client.api.v5.PulsarClientException;
import org.apache.pulsar.common.naming.TopicDomain;
import org.apache.pulsar.common.naming.TopicName;
+import org.apache.pulsar.common.util.FutureUtil;
/**
* Internal utilities for the V5 client implementation.
*/
final class V5Utils {
Review Comment:
Would it make sense to put the exception mappings in a dedicated class? V5ExceptionMappings etc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org